Necessary Maximum Principle of Stochastic Optimal Control with Delay and Jump Di ff usion

XING LEI1,2AND ZHAO PENG-FEI3,4,5,

(1.College of Basic Sciences,Changchun University of Technology,Changchun,130022)

(2.Institute of Mathematics,Jilin University,Changchun,130012)

(3.College of Geoexploration Science and Technology,Jilin University,Changchun,130026)

(4.Post-doctoral Flow Station of Computer Science and Technology,College of Computer Science and Technology,Jilin University,Changchun,130012)

(5.Department of Scienti fi c Computing,Florida State University,Tallahassee,FL 32306,USA)

Necessary Maximum Principle of Stochastic Optimal Control with Delay and Jump Di ff usion

XING LEI1,2AND ZHAO PENG-FEI3,4,5,*

(1.College of Basic Sciences,Changchun University of Technology,Changchun,130022)

(2.Institute of Mathematics,Jilin University,Changchun,130012)

(3.College of Geoexploration Science and Technology,Jilin University,Changchun,130026)

(4.Post-doctoral Flow Station of Computer Science and Technology,College of Computer Science and Technology,Jilin University,Changchun,130012)

(5.Department of Scienti fi c Computing,Florida State University,Tallahassee,FL 32306,USA)

Communicated by Li Yong

In this paper,we have studied the necessary maximum principle of stochastic optimal control problem with delay and jump di ff usion.

stochastic di ff erential equation,jump di ff usion,delay,necessary maximum principle

1 Introduction

The optimal control theory of stochastic problem has been widely used in economy, fi nanceetc.The delay variables and jump di ff usions make the problem more complex than the one with only Brownian motion.Actually,delay and jump are very common in real world,and its optimal control problem is very challenging and meaningful.In this paper,we study the necessary maximum principle of stochastic optimal control problem with delay and jump di ff usion.

In last forty years,the maximum principle of stochastic problem has been studied by many researchers.The research started by the work of Kushner[1]and Bismut[2],and improved by Bensoussan[3],Haussmann[4],and Peng[5],Yong and Zhou[6]reviewed and improved the previous work based on these famous works.These works are mainly about the Wiener process.

In last decade,the studies about stochastic optimal problem with delay have developed very fast.In 2000,Øksendal and Sulem[7]obtained two sufficient maximum principles of stochastic optimal problem with delay,and cooperated with Elsanousiet al.[8]to give the solvability result of optimal control for random harvest problem with delay.Later,Larssen[9]applied the dynamic programming method to solve stochastic optimal control problem with delay.In 2010,based on Peng and Yang’s work(see[10]),Chen and Wu[11]gave the necessary and sufficient principles of the stochastic problem with delay.There were also many works for the optimal control problems with the jump di ff usion.Tang and Li[12]obtained the necessary maximum principle about this problem,Framstadet al.[13]gave the sufficient maximum principle.Also,in 2011,Øksendalet al.[14]gave a result about maximum principle on stochastic system with delay and jump,and our result is di ff erent with this result in model and proof of the theorems,such as higher dimensions,di ff erent parameters,and the proof details.

In Section 2,we introduce the preliminaries about the stochastic di ff erential equation with delay and jump di ff usion,and give several assumptions which should be satis fi ed in the following results.In Section 3,with the aid of Itˆo’s formula and variational technique, we obtain the necessary maximum principle for the stochastic problem with delay and jump di ff usion.

2 Preliminaries

Firstly,we introduce the stochastic di ff erential equation with delay and jump di ff usion.Let the state variable X(t)be a jump di ff usion which is a ff ected by the control variable v(t)and its delay variable v(t−δ).Then the equation of X(t)can be described as follows∶

dX(t)=b(t,X(t),X(t−δ),v(t),v(t−δ))dt+σ(t,X(t),X(t−δ),v(t),v(t−δ))dB(t)

Here,b∶[0,T]×Rn×Rn×Rk×Rk−→Rn,σ∶[0,T]×Rn×Rn×Rk×Rk−→Rn×dand θ∶[0,T]×Rn×Rn×Rk×Rk×R−→Rn×lare known functions,(B(t))t∈[0,T]is a d dimensional Brownian motion on

We de fi ne

The state functional about the problem(2.1)-(2.2)is de fi ned as

where L∶[0,T]×Rn×Rk×Rk→R,Φ∶Rn→Rare known functions.Suppose

Secondly,we de fi ne the Hamiltonian(see[15]) where r∶Rn+1→Rn×lsuch that the integral in H(·)is convergent.

Before de fi ning the adjoint equation of the optimal control pair,we give some expressions, and make the descriptions of the equations much easier.

or

where f=b,σ,˜f=θij,ω=x,xδ,v,vδ.The adjoint equation of(X,u)can be described as

Finally,we give some assumptions as follows∶

(A1)b,σ,θ,L,Φare continuously di ff erentiable with respect to(X(t),X(t−δ),v(t), v(t−δ)),and their di ff erential coefficients are bounded.

(A2)Assume that(2.6)has a unique solution if(A1)and some other conditions are satis fi ed.

We may assume that the variablesatis fi es the condition u(·)+∈Sinceis convex,one has uρ(·)∈where uρ(·)=u(·)+0≤ρ≤1.For the stochastic optimal control problem(2.1)–(2.4),the corresponding trajectory of uρ(·)is Xρ(·).Also, we may have some expressions of uρas follows∶

and

3 Main Results

In this section,we obtain the necessary maximum principle of stochastic optimal control problem with delay and jump di ff usion.Before introducing the main results,we prove a convergence lemma.With the aid of this lemma,we get the variational inequality which is very important for the necessary maximum principle.

In the following,we discuss the convergence relations among Xρ,X(t)and q(t).De fi ne the variational equation as follows∶

Lemma 3.1When(3.1)satis fi es the conditions(A1)–(A3),we have

where

Proof.According to the expression of˜X(t),we obtain

Note the above equation,we have

where the symbols are de fi ned as

Assume that the conditions(A1)–(A3)are established.Applying Itˆo formula to|˜X(t)|2,we have

where C1,C2,C3＞0.Since

we have

Therefore,

where

By Gronwall’s inequality,we have

The proof is completed.

Lemma 3.2Suppose that(A1)–(A3)are established.Then we have the inequality

Proof.For u(·)is the optimal control variable,we have

Dividing by ρ＞0 in the both sides,we have

When ρ→0,we get the limitation of two sides.Then

According to Lemma 3.1,we know that

Then

The proof is completed.

Now we give the necessary maximum principle of stochastic optimal control problem (2.1)–(2.4)with delay and jump di ff usion.

Theorem 3.1If(X(·),u(·))is the optimal control pair of stochastic optimal control problem(2.1)–(2.4)with delay and jump di ff usion,then we have

where

Proof.With the aid of Itˆo formula,we have E〈p(T),q(T)〉

In order to make the whole proof simple,we divide the right side of the above equation into eight parts,and they are de fi ned as follows∶

When t∈[−δ,0]∪[T,T+δ],q(t)=0.Then Δ4=0.

When t∈[−δ,0],q(t)=0.That is,when t∈[0,δ],q(t−δ)=0.Therefore,

When t∈(T,T+δ],p(t)=0.Therefore,

Then we have Δ5=0.

When t∈[−δ,0],q(t)=0.That is,when t∈[0,δ],q(t−δ)=0.Then we have

When t∈[T,T+δ],k(t)=0.So

Therefore,Δ6=0.

In summary,

Further,from the above equation,we have

Add the left hand side of(3.3)to the right hand side of(3.2),we have

By(2.6),we have p(T)=Φx(X(T)).Therefore,the inequality can be described as

that is,

In order to get the inequality condition which is satis fi ed by the optimal control variable u(·),we make aciculate variation to the feasible control variable

where t∈[0,T],v(·)is a feasible control variable.Therefore,

and

From(3.4),we have

When ε→0,we have

Assume that v∈U is a deterministic variable,A is an arbitrary element of σ-algebra Ft. We have

And then

By the arbitrariness of A,we have

The proof is completed.

AcknowledgementsThe authors express sincere thanks to Professor Gong Fu-zhou for his instructions and many invaluable suggestions.Especially,we would like to thank Professors Li Yong,Han Yue-cai and Dong Xiao-gang for their enthusiastic assistance.

[1]Kushner H J.Necessary conditions for continuous parameter stochastic optimization problems. SIAM J.Control Optim.,1972,10:550–565.

[2]Bismut J M.Conjuagate convex functions in optimal stochastic control.J.Math.Anal.Appl., 1973,44:384–404.

[3]Benssoussan A.Maximum principle and dynamic programming approaches of the optimal control of partially observed di ff usions.Stochastics,1983,9:169–222.

[4]Haussmann U G.A stochastic Maximum Principle for Optimal Control of Di ff usions.Harlow: Longman Scienti fi c and Technical,1986.

[5]Peng S G.A general stochastic maximum principle for optimal control problems.SIAM J. Control Optim.,1990,28:966–979.

[6]Yong J,Zhou X.Stochastic Controls-Hamiltonian Systems and HJB Equations.New York: Springer-Verlag,1999.

[8]Elsanousi I,Øksendal B,Sulem A.Some solvable stochastic control problems with delay. Stochastics,2000,7:69–89.

[9]Larssen B.Dynamic programming in stochastic ontrol of systems with delay.Stochastics,2002,74:651–673.

[10]Peng S,Yang Z.Anticipated backward stochastic di ff erential equation.Ann.Probab.,2009,27:877–902.

[11]Chen L,Wu Z.Maximum principle for the stochastic optimal control problem with delay and application.Automatica,2010,46:1074–1080.

[12]Tang S J,Li X J.Neccessary conditions for optimal control of stochastic systems with random jumps.SIAM.J.Control Optim.,1994,32:1447–1475.

[13]Framstad N C,Øksendal B,Sulem A.A sufficient stochastic maximum principle for optimal control of jump di ff usions and applications to fi nance.J.Optim.Theory Appl.,2004,121: 77–98.

[14]Øksendal B,Sulem A,Zhang T.Optimal control of stochastic delay equations and timeadvanced backward stochastic di ff erential equations.Adv.Appl.Probab.,2011:43,572–596.

[15]Øksendal B,Sulem A.Applied Stochastic Control of Jump Di ff usions.Berlin Heidelberg: Springer-Verlag,2005.

tion:60H15,49J20,93E20

A

1674-5647(2014)03-0245-12

10.13447/j.1674-5647.2014.03.06

Received date:Jan.17,2012.

Foundation item:The China Scholarship Council,the National Basic Research Program(2009CB219301) of China(973)in part,the National Public Bene fi t Scienti fi c Research Foundation(201011078)of China,the National Innovation Research Project for Exploration and Development of Oil Shale(OSP-02 and OSR-02),the NSF(41304087,11071026,61133011,61170092,60973088,61202308,11001100,11171131 and 11026043)of China, and the Basic Research Foundation of Jilin University in 2012.

*Corresponding author.

E-mail address:sundy.xinglei@gmail.com(Xing L),zhaopf@jlu.edu.cn(Zhao P F).