Home Linear Quadratic Nash Differential Games of Stochastic Singular Systems
Article Publicly Available

Linear Quadratic Nash Differential Games of Stochastic Singular Systems

  • Haiying Zhou EMAIL logo , Huainian Zhu and Chengke Zhang
Published/Copyright: December 25, 2014
Become an author with De Gruyter Brill

Abstract

In this paper, we deal with the Nash differential games of stochastic singular systems governed by Itô-type equation in finite-time horizon and infinite-time horizon, respectively. Firstly, the Nash differential game problem of stochastic singular systems in finite time horizon is formulated. By applying the results of stochastic optimal control problem, the existence condition of the Nash strategy is presented by means of a set of cross-coupled Riccati differential equations. Similarly, under the assumption of the admissibility of the stochastic singular systems, the existence condition of the Nash strategy in infinite-time horizon is presented by means of a set of cross-coupled Riccati algebraic equations. The results show that the strategies of each players interact.

1 Introduction

Singular systems, also known as descriptor systems, generalized state-space systems and implicit systems are described by differential-algebraic equations. Singular systems have been extensively studied over the past decades due to the fact that they can describe a great many natural phenomena in physical systems such as microelectronic circuits, economics, demography and so on[14].

As is well known, environmental noise exists and cannot be neglected in many dynamical systems[59]. And there are some results about the research of LQ control for stochastic singular systems: Balasubramaniam and Kumaresan[10] discussed a solution of generalized matrix Riccati differential equation for indefinite stochastic linear quadratic singular system using neural networks. The optimal control problem for a class of stochastic LQ singular periodic neuro Takagi-Sugeno fuzzy systems by using ant colony programming was studied in [11]. Kumaresan and Kuru[12] investigated the optimal control problem for stochastic linear singular Takagi-Sugeno fuzzy delay systems with quadratic performance using genetic programming. Zhang and Xing[13] investigated the stability and optimal control for stochastic singular systems.

On the other hand, during the past decades, differential games have been playing a central role in economy, ecology and elsewhere, such as optimizing behavior and enduring consequences of decisions. For that reason, this framework has developed into a major research field in control theory, and has many applications for solving real world problems, see [1416]. The differential games of singular systems can describe many practical situations and have been widely studied by many researchers for several decades. Among these, most researchers studied the differential games of deterministic singular systems in finite-time horizon and infinite-time horizon, we refer the reader to [1721] and the references therein.

However, to the best of our knowledge, few results have been obtained of the differential games for stochastic singular systems. Motivated by this, we deal with the LQ Nash differential games of stochastic singular systems.

It is worth mentioning that many researchers discussed the Nash differential games of singular systems by disintegrating the systems into two: fast and slow[18, 19, 21]. Recently Zhang and Xing deal with the stochastic singular systems as a whole by using the square completion technique, and get the results for optimal control problems involving new stochastic generalized Riccati equation. Inspired by this, we apply the same method to the corresponding differential game problems and get a set of cross-coupled stochastic Riccati equations. Actually, the conclusion is more intuitive and calculation is more easily.

The rest of the paper is organized as follows: Some important definitions and lemmas which are useful to obtain the main results are introduced in section 2. The differential games of stochastic singular system in finite-time horizon are discussed in section 3. In this case, the problem is formulated firstly. Then, the corresponding cross-coupled Riccati differential equations are derived and the existence of their solutions is shown to be sufficient conditions for the Nash strategies. The differential games of stochastic singular system in infinite-time horizon discussed in section 4. Similarly, the corresponding cross-coupled Riccati algebraic equations are derived and the existence of their solutions is shown to be sufficient conditions for the Nash strategies. In section 5, the conclusions are given.

For convenience, we will make use of the following notations throughout this paper:

A′ : transpose of a matrix or vector A; A−1: inverse of a matrix or vector A; A > 0 (A ≥ 0): positive definite (positive semi-definite) matrix A; ε[⋅]: the expectation operator with respect to the given probability measure P. Rn: the n-dimensional Euclidean space; Sn × n : the set of all n × n matrices; Sn: the space of all n × n symmetric matrices; S+n : the space of all nonnegative definite matrices of Sn; rank(A): the rank of A; deg(det(sIA)): degree of determinant sIA.

2 Preliminaries

In this section, we introduce some important definitions and lemmas which are useful to obtain our main results later.

Throughout this paper, let (Ω, F, {Ft}t ≥ 0, P) be a given filtered probability space where there exists a standard one dimensional Wiener process {w(t)}t ≥ 0. Consider the following stochastic singular systems

Edx(t)=Ax(t)dt+Cx(t)dW(t)x(0)=x0,t0(1)

where x0Rn is the initial state of the systems, ARn × n, CRn × n are known coefficient matrices associated with x(t), respectively. E is a known singular matrix with rank(E) = rn.

Assumption 1[22]

There is a pair of nonsingular matrices PRn × n and QRn × n for the triplet (E, A, C) such that the following conditions are satisfied:

PEQ=[Ir000],PAQ=[A10B1D1],PCQ=[C100Inr],

where A1Rn × n, B1Rr × (nr), C1Rn × n, D1R(nr) × (nr).

Hence, we have the following result for the existence and uniqueness of solution to the system (1).

Lemma 1[13]

System (1) has a unique solution if the above assumption holds.

Definition 1[23]

  1. The system (1) is said to be regular if det(sEA) is not identically zero;

  2. The system (1) is said to be impulse-free if deg(det(sEA)) = rank(E);

  3. The system (1) is said to be mean-square stable iflimtεxt2=0;

  4. The system (1) is said to be mean-square admissible, if the system is regular, impulse-free and mean-square stable.

Lemma 2[13]

System (1) is mean-square admissible if there exists a nonsingular matrix G, such that the following coupled LMIs hold:

EG=GE0AG+GA+CEGC<0(2)

Next, the Nash differential games for stochastic singular systems in finite-time horizon and infinite-time horizon are discussed, respectively.

3 Stochastic Nash differential games in finite-time horizon

3.1 Problem formulation

We consider the following systems

Edx(t)=Ax(t)+B1u1(t)+B2u2(t)dt+Cx(t)dw(t)x(0)=x0,t[0,T](3)

The quadratic cost function associated with each player is

Jτ(u1,u2;x0,0)=ε120Tx(t)Qτx(t)+u1(t)Rτ1u1(t)+u2(t)Rτ2u2(t)dt+12x(T)Hτx(T),τ=1,2(4)

where x(t) ∈ Rn is the system state, the coefficient matrices A, B1, B2, C are assumed to be real constant matrices with appropriate dimensions. E is a known singular matrix with rank(E) = rn, u1(t) ∈ Rm and u2(t) ∈ Rl represent the two players control inputs respectively. The matrices Qτ, Hτ (τ = 1, 2) are assumed to be positive semi-definite symmetric matrices with appropriate dimensions, Rτ 1 and Rτ 2 (τ = 1, 2) are assumed to be positive definite symmetric matrices with appropriate dimensions.

The strategies of the two players are denoted by u1(t) and u2(t), which belong to strategy spaces Γ1 and Γ2, respectively. In this note, the restriction that Γ1 and Γ2 are composed of linear feedback strategies of the form, i.e.

u1(t)=K1(t)x(t),u2(t)=K2(t)x(t)(5)

Definition 2

A linear feedback strategy pairu1(t),u2(t)Γ1a×Γ2aΓ1×Γ2is called an admissible strategy pair if the closed-loop system obtained has no impulsive solution. Correspondingly, Γ1a×Γ2aΓ1×Γ2is called the admissible strategy space.

Definition 3

An admissible linear feedback strategy pairu1(t),u2(t)Γ1a×Γ2aconstitutes a Nash equilibrium pair if

J1(u1,u2;x0,0)J1(u1,u2;x0,0),J2(u1,u2;x0,0)J2(u1,u2;x0,0)(6)
for allu1(t),u2(t)Γ1a×Γ2a,u1(t),u2(t)Γ1a×Γ2a,u1(t),u2(t)Γ1a×Γ2a.

3.2 One-player case

First, a one-player case is discussed. The result obtained for that particular case is used as the basis for the derivation of the results for the 2-player case.

Consider linear quadratic (LQ) stochastic controlled singular systems in the following form.

minuJ(u;x0,0)=ε120Tx(t)Qx(t)+u(t)Ru(t)dt+12X(T)HX(T)s.t.Edx(t)=Ax(t)+Bu(t)dt+Cx(t)+Du(t)dw(t)x(0)=x0(7)

Theorem 1[13]

If there exists a solution P(t) ∈ S+nfor the following Riccati differential equations

EP˙(t)E+EP(t)A+AP(t)E+CEP(t)EC+K(t)(BP(t)E+DEP(t)EC)+Q=0EP(T)E=HK(t)=R+DEP(t)ED1(BP(t)E+DEP(t)EC)R+DEP(t)ED>0(8)

Then, the linear state feedback optimal control law for the LQ problem is

u(t)=K(t)x(t)=R+DEP(t)ED1BP(t)E+DEP(t)ECx(t)(9)

Moreover, the corresponding function is

J(u;x0,0)=12x0EP(0)Ex0(10)

3.3 Main result

The solution for the finite-time horizon stochastic Nash differential games is given below.

Theorem 2

For system (3) ∼ (4), assume that for any (u1(t), u2(t)) ∈ Γ1a×Γ2a, the following stochastic generalized Riccati differential equations (11) and (12) admit the solutions (P1 (t), P2 (t) ) ∈ S+n × S+n :

EP˙1(t)E+EP1(t)A+B2K2(t)+A+B2K2(t)P1(t)E+Q1+CEP1(t)EC+K2(t)R12K2(t)+K1(t)B1P1(t)E=0EP1(T)E=H1K1(t)=R111B1P1(t)E(11)
EP˙2(t)E+EP2(t)A+B1K1(t)+A+B1K1(t)P2(t)E+Q2+CEP2(t)EC+K1(t)R21K1(t)+K2(t)B2P2(t)E=0EP2(T)E=H2K2(t)=R221B2P2(t)E(12)

If the system (3) is mean-square admissible, then

  1. The problem of finite-time horizon stochastic Nash differential games admits a pair of solutions (u1(t),u2(t)) with

    u1(t)=K1(t)x(t),u2(t)=K2(t)x(t)(13)
  2. The optimal cost functions incurred by playing strategies (u1(t),u2(t)) are

    Jτ(u1,u2;x0,0)=12x0Pτ(0)x0,(τ=1,2)(14)

Proof

Now, let us consider the following LQ problem in which the cost function (15) is minimal at Ki(t)=Ki(t)

Ji(ui,uj;x0,0)=ε120Tx(t)Qi+Kj(t)RijKj(t)+Ki(t)RiiKi(t)x(t)dt+12x(T)Hix(T)(15)

where x(t) follows from

Edx(t)=A+BjKj(t)+BiKi(t)x(t)dt+Cx(t)dw(t)x(0)=x0,i,j=1,2,ij(16)

Note that the LQ problem (15) ∼ (16) coincides with the LQ problem (3) ∼ (4) in Theorem 1. Applying Theorem 1 to the LQ problem (15) ∼ (16) as

A+BjKj(t)A,BiB,CC,0D,Qi+Kj(t)RijKj(t)Q,RiiR,HiH.

Note that Rii > 0, yields the fact that the function Ji(ui,uj;x0,0) is minimal at

u(t)=R+DEP(t)ED1BP(t)E+DEP(t)ECx(t)ui(t)=Rii1BiPi(t)Ex(t)(17)

Moreover, the optimal value is 12x0EPi(0)Ex0.

So this completes the proof.

4 Stochastic Nash differential games in infinite-time horizon

4.1 Preliminaries

Firstly, let us recall the stability of the infinite-time stochastic singular system.

Consider the following system

Edx(t)=Ax(t)+Bu(t)dt+Cx(t)dw(t)x(0)=x0,t0(18)

where the coefficient matrices A, B, C are real constant matrices with appropriate dimensions, and a process u(⋅) ∈ Rm is the control input.

For the system (18), we consider the following state feedback controller

u(t)=K^x(t)(19)

where K^ is a constant matrix of appropriate dimensions, to be determined.

Furthermore, we get the corresponding closed-loop system

Edx(t)=(A+BK^)x(t)dt+Cx(t)dw(t)x(0)=x0Rn,t0(20)

Definition 4[13]

The system (18) is called mean-square stabilizable if there exists a state feedback control law (19) such that the closed-loop system (20) is mean-square stable.

4.2 Problem formulation

Consider the following stochastic singular systems

Edx(t)=Ax(t)+B1u1(t)+B2u2(t)dt+Cx(t)dw(t)x(0)=x0,t0(21)

where x(t) ∈ Rn is the system state, the coefficient matrices A, C, B1, B2 are real constant matrices with appropriate dimensions. E is a known singular matrix with rank(E) = rn.

The quadratic cost functions associated with each player are

Jτ(u1,u2;x0,0)=ε120x(t)Qτx(t)+u1(t)Rτ1u1(t)+u2(t)Rτ2u2(t)dt,τ=1,2(22)

where the weighting matrices Qτ (τ = 1, 2) are given symmetric positive semi-definite matrices, Rτ 1, Rτ 2 (τ = 1, 2) are given symmetric positive definite matrices.

In this note, the restriction that Γ1 and Γ2 are composed of linear feedback strategies of the form, i.e.

u1=K1x(t),u2=K2x(t).

Therefore, we are looking for an admissible linear feedback strategy pair u1,u2Γ1a×Γ2a that satisfies

J1(u1,u2;x0,0)J1(u1,u2;x0,0),J2(u1,u2;x0,0)J2(u1,u2;x0,0)(23)

for all u1,u2Γ1a×Γ2a,u1,u2Γ1a×Γ2a,u1,u2Γ1a×Γ2a.

The following basic assumption is imposed throughout this section.

Assumption 2

The system (21) is mean-square stabilizable.

4.3 Main result

Theorem 3

For system (21) ∼ (22), suppose the following generalized Riccati algebraic equations (24) and (25) admit solutions (P1, P2) ∈ S+n × S+n :

EP1A+B2K2+A+B2K2P1E+Q1+CEP1EC+K2R12K2+K1B1P1E=0K1=R111B1P1E(24)
EP2A+B1K1+A+B1K1P2E+Q2+CEP2EC+K1R21K1+K2B2P2E=0K2=R221B2P2E(25)

Then

  1. The problem of infinite-time horizon stochastic differential games admits a pair of(u1,u2)with

    u1=K1x(t),u2=K2x(t)(26)
  2. The optimal cost functions incurred by playing strategies(u1,u2)are

    Jτ(u1,u2;x0,0)=12x0Pτx0,(τ=1,2)(27)

    Proof is similar with the corresponding proof in Theorem 3.

5 Conclusions

In this paper, we dealt with the stochastic Nash games in finite-time horizon and infinite-time horizon, respectively. Basing on the results of the stochastic optimal control problem, the Nash strategies for stochastic singular systems are derived by the method of a set of cross-coupled stochastic Riccati equations. We note that the strategies of the two players are interacted and it has applications for solving real world problems. For example, considering the investment stock problem of a security company, both the investors and security company need to consider the strategies of each other when they choose the strategies. Hence, it would be necessary to consider the differential games for the stochastic singular systems to obtain an equibrilium. Also the results are easily obtained from those of LQ scochastic singular systems, the method and the stochastic cross-coupled Raccti equations are new for the Nash games of the scochastic singular system. Due to various reasons, we only deal with the open-loop Nash equilibrium for the stochastic singular systems, and the close-loop Nash equilibrium is the next step of research.


Supported by National Natural Science Foundation of China (71171061); Natural Science Foundation of Guangdong Province (S2011010004970); China Postdoctoral Science Foundation (2014M552177)


References

[1] Lewis F L. A survey of linear singular systems. Circuits, Systems and Signal Processing, 1986, 5(1): 3–36.10.1007/BF01600184Search in Google Scholar

[2] Brenan K E, Campbell S L V, Petzold L R. Numerical solution of initial-value problems in differential-algebraic equations. Siam, 1989.10.1137/1.9781611971224Search in Google Scholar

[3] Liu C, Zhang Q, Feng Y, et al. Complex dynamics in a harvested differential-algebraic eco-epidemiological model. International Journal of Information and Systems Sciences, 2009, 5(3–4): 311–324.Search in Google Scholar

[4] Masubuchi I, Kamitane Y, Ohara A, et al. H control for descriptor systems: A matrix inequalities approach. Automatica, 1997, 33(4): 669–673.10.1016/S0005-1098(96)00193-8Search in Google Scholar

[5] Boukas E K. Stabilization of stochastic singular nonlinear hybrid systems. Nonlinear Analysis: Theory, Methods & Applications, 2006, 64(2): 217–228.10.1016/j.na.2005.05.066Search in Google Scholar

[6] Gerdin M, Glad T, Ljung L. Well-posedness of filtering problems for stochastic linear DAE models. Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC’05. 44th IEEE Conference on. IEEE, 2005: 350–355.10.1109/CDC.2005.1582180Search in Google Scholar

[7] Gerdin M, Sjoberg J. Nonlinear stochastic differential-algebraic equations with application to particle filtering. Decision and Control, 2006 45th IEEE Conference on. IEEE, 2006: 6630–6635.10.1109/CDC.2006.377135Search in Google Scholar

[8] Mao X, Yuan C. Stochastic differential equations: With Markovian switching. Imperial College Press, 2006.10.1142/p473Search in Google Scholar

[9] Schein O, Denk G. Numerical solution of stochastic differential-algebraic equations with applications to transient noise simulation of microelectronic circuits. Journal of Computational and Applied Mathematics, 1998, 100(1): 77–92.10.1016/S0377-0427(98)00138-1Search in Google Scholar

[10] Balasubramaniam P, Kumaresan N. Solution of generalized matrix Riccati differential equation for indefinite stochastic linear quadratic singular system using neural networks. Applied Mathematics and Computation, 2008, 204(2): 671–679.10.1016/j.amc.2008.04.023Search in Google Scholar

[11] Kumaresan N. Optimal control for stochastic linear quadratic singular periodic neuro Takagi-Sugeno (TS) fuzzy system with singular cost using ant colony programming. Applied Mathematical Modelling, 2011, 35(8): 3797–3808.10.1016/j.apm.2011.02.017Search in Google Scholar

[12] Kumaresan N, Kuru R. Optimal control for stochastic linear quadratic singular Takagi-Sugeno fuzzy delay system using genetic programming. Applied Soft Computing, 2012, 12(8): 2085–2090.10.1016/j.asoc.2012.03.017Search in Google Scholar

[13] Zhang Q, Xing S. Stability analysis and optimal control of stochastic singular systems. Optimization Letters, 2013: 1–16.10.1007/s11590-013-0687-5Search in Google Scholar

[14] Basar T, Olsder G J. Dynamic noncooperative game theory[M]. 2nd ed. Philadelphia, PA: SIAM, 1999.10.1137/1.9781611971132Search in Google Scholar

[15] Dockner E, Jorgensen S, Long N V, et al. Differential games in economics and management science. Cambridge University Press, 2000.10.1017/CBO9780511805127Search in Google Scholar

[16] Friesz, Terry L. Dynamic optimization and differential games. Springer, 2009.10.1007/978-0-387-72778-3Search in Google Scholar

[17] Xu H, Mizukami K. New sufficient conditions for linear feedback closed-loop Stackelberg strategy of descriptor systems. IEEE Transactions on Automatic Control, 1994, 39(5): 1097–1102.10.1109/9.284902Search in Google Scholar

[18] Engwerda J C. The open-loop linear quadratic differential game for index one descriptor systems. Automatica, 2009, 45(2): 585–592.10.1016/j.automatica.2008.09.012Search in Google Scholar

[19] Mizukami K, Tetsushi K. On closed-loop Nash equilibrium solutions for continuous descriptor systems. Bulletin of Hiroshima Koukusai Gakuin University, 2001, 34: 51–60.Search in Google Scholar

[20] Xu H, Mizukami K. On the Isaacs equation of differential games for descriptor systems. Journal of optimization theory and applications, 1994, 83(2): 405–419.10.1007/BF02190065Search in Google Scholar

[21] Xu H, Mizukami K. Linear-quadratic zero-sum differential games for generalized state space systems. IEEE Transactions on Automatic Control, 1994, 39(1): 143–147.10.1109/9.273352Search in Google Scholar

[22] Boukas E K. Control of singular systems with random abrupt changes. Springer, Berlin, 2008.Search in Google Scholar

[23] Lu R, Dai X, Du W, et al. Robust H output feedback control for uncertain stochastic singular systems. 2008 Control and Decision Conference, CCDC 2008, Chinese. IEEE, 2008: 4344–4349.Search in Google Scholar

Received: 2014-1-21
Accepted: 2014-10-15
Published Online: 2014-12-25

© 2014 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 9.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/JSSI-2014-0553/html
Scroll to top button