Home Linear Quadratic Nash Game of Stochastic Singular Time-Delay Systems with Multiple Decision Makers
Article Publicly Available

Linear Quadratic Nash Game of Stochastic Singular Time-Delay Systems with Multiple Decision Makers

  • Huainian Zhu EMAIL logo , Guangyu Zhang , Chengke Zhang , Ying Zhu and Haiying Zhou
Published/Copyright: October 25, 2015
Become an author with De Gruyter Brill

Abstract

This paper discusses linear quadratic Nash game of stochastic singular time-delay systems governed by Itô’s differential equation. Sufficient condition for the existence of Nash strategies is given by means of linear matrix inequality for the first time. Moreover, in order to demonstrate the usefulness of the proposed theory, stochastic H2∕H control with multiple decision makers is discussed as an immediate application.

1 Introduction

Singular systems, also known as descriptor systems, generalized state-space systems or implicit systems, are described by differential-algebraic equations. Singular systems have been extensively studied over the past decades due to the fact that they can describe a great many natural phenomena in physical systems such as microelectronic circuits, economics, demography and so on (see, e.g., [13]). A great number of fundamental notions and results in control and system theory based on state-space systems have been extended successfully to singular systems (see, e.g., [48] and the references therein).

In the past few decades, stochastic systems governed by Itô’s differential equations have received a great deal of research attention [9, 10]. Although a variety of results for optimal control of linear stochastic systems have been reported, the dynamic games of such systems have received comparatively little attention. Moreover, to the best of our knowledge, stochastic Nash games for singular time-delay systems have not been fully investigated. Since delays appear in many practical plants, the design of such strategy is an important issue that remains open.

This paper is concerned with the problem of Nash game of stochastic singular time-delay systems with multiple decision makers. In terms of a set of linear matrix inequalities, we present a sufficient condition for the existence of both Nash strategies and the upper bound of the cost function for the first time. Moreover, in order to demonstrate the effectiveness of the proposed theory, stochastic H2∕H control with multiple decision makers is discussed as an immediate application.

The rest of this paper is organized as follows. In Section 2, some preliminaries are given. In Section 3, the main results are given, which generalizes the results of [11]. Section 4 discusses the stochastic H2∕H control with multiple decision makers by using the obtained results. Section 5 ends this paper with some comments.

Notations: Throughout this paper, unless otherwise specified, we will employ the following notations. ( Ω, ℱ, {t}t≥0,ℙ) is a complete probability space with Ω being a sample space, being a σ-field, {t}t≥0 being a natural filtration and ℙ being a probability measure. AT is the transpose of a matrix A; A–1is inverse of a matrix A; A > 0 means that A is positive definite; In denotes the n × n identity matrix; rank(A) denotes the rank of A; deg (det(sIA)) denotes degree of determinant sIA.Rn is the n-dimensional Euclidean space; E[·] denotes the expectation operator.

2 Preliminary Results

Consider the following time-invariant stochastic singular time-delay systems

(1){Edx(t)=[Ax(t)+A1x(tδ)+Bu(t)]dt+p=1MApx(t)dwp(t)x(t)=φ(t),t[δ,0]

where x(t) ∊ Rn is the state vector; u(t) ∊ Rm is the control vector; E is a known singular matrix with rank(E) = rn; wp(t) ∊ R is a one-dimensional standard Wiener process defined in the filtered probability space (Ω,, {t}t0, ℙ)· Without loss of generality, it is assumed that wr(t) and ws(t) are mutually independent for all r, s = 1, 2, ⋯ , M, and E[w(t)wT(t)] = IM, where w(t) := [w1(t), w2(t), ⋯ , wM(t)]t. Here, the scalar δ > 0 is the time delay of the system. φ(t) is a real-valued initial function. A, A1, B, Ap are given real matrices of suitable sizes.

Let us consider the following stochastic linear quadratic control problem subject to (1):

(2)minimizeJ(u(),x(0))=E0[xT(t)Qx(t)+uT(t)Ru(t)]dtQ=QT0,R=RT>0

The following definitions are similar with Definition 3.1 which was introduced in [12].

Definition 1

The system (1) is called mean-square stable if there exists a linear state feedback law u(t) = Kx(t), ΚRm×n, such that the resultant closed-loop system is asymptotically stable in mean-square, i.e., its trajectories satisfy limtEx(t)2=0, for any φ(0).

Definition 2

  1. The system (1) is said to be regular if det(sE – A) is not identically zero;

  2. The system (1) is said to be impulse-free if deg(det(sE A)) = rank(E);

  3. The system (1) is said to be mean-square admissible, if the system is regular, impulse-free and mean-square stable.

The following lemma plays a key technical role in this paper.

Lemma 1

Assume that for any u(t), the closed-loop system is mean-square admissible. Suppose that there exists two real symmetric matrix Ρ > 0 and W >0, such that

(3)Υ(P)=[ΞETPA1A1TPEW]0

where

Ξ=ETPA+ATPE+p=1MApTPApETPBR1BTPE+Q+W

Then the optimal feedback strategy for the stochastic linear quadratic control problem is

(4)u(t)=Kx(t)=R1BTPEx(t)

Moreover, the optimal value of cost function

(5)J(u(),x(0))E[xT(0)ETPEx(0)]+E[δ0φT(τ)Wφ(τ)dτ]

Proof Its proof can be demonstrated by using the square completion technique. First, define the following quadratic function

(6)V(t)=xT(t)ETPEx(t)+tδtxT(τ)Wx(τ)dτ

where Ρ = PT > 0, W = WT > 0.

Applying Itô’s formula to the stochastic system (1), we have

(7)d[V(t)]={xT(t)[ETPA+ATPE+p=1MApTPAp+W]x(t)+2uT(t)BTPEx(t)+2xT(t)ETPA1x(th)xT(th)Wx(th)}dt+{}dwp(t)

where {⋯ } does not affect the calculation results and can be omitted.

Integrating (7) from 0 to ∞, taking expectations E[·] on both sides, one gets

(8)E[V()]E[V(0)]=E0{xT(t)[ETPA+ATPE+p=1MApTPAp+W]x(t)+2uT(t)BTPEx(t)+2xT(t)ETPA1x(th)xT(th)Wx(th)}dt

Under the assumption that the closed-loop system is mean-square admissible, we get E[V(∞)] = 0. Thus, adding this to (2) and, using the square completion technique, we have

(9)J(u(),x(0))E[V(0)]=E0ηT(t)Υ(P)η(t)dt+E0[u(t)+R1BTPEx(t)]TR[u(t)+R1BTPEx(t)]dtE0ηT(t)Υ(P)η(t)dt=J(u(),x(0))

where ηT(t)=[xT(t) xT(tδ)}.

Thus, the feedback control (4) is the optimal control. On the other hand,

(10)J(u(),x(0))E[V(0)]=E0ηT(t)Υ(P)η(t)dt0

Thus, if (3) holds, then the desired result is obtained.

3 Main Results

In this section, we will utilize the obtained results of stochastic linear quadratic optimal control to derive the results of stochastic Nash games.

3.1 Problem Formulation

Consider the following stochastic singular time-delay systems with Ν decision makers involving state-dependent noise

(11){Edx(t)=[Ax(t)+A1x(tδ)+i=1NBiui(t)]dt+p=1MApx(t)dwp(t)x(t)=φ(t),t[δ,0]

where A, A1 and Ap are n × n real matrices, ui(t)RRmi,i=1,2,,N, is the i-th control input, which represents the player i’s control strategy of this game, Bi, i = 1, 2, ⋯, Ν, are n × mi real matrices.

The cost function for each decision maker is defined by

(12)Ji(u1(),u2(),,uN(),x(0))=E0[xT(t)Qix(t)+uiT(t)Riui(t)]dt

where i = 1, 2, ⋯ ,N, Qi=QiT0,Ri=RiT>0

It should be noted that uj(·), ji does not appear in the cost function. However, since they are included in the stochastic systems (11), they must have impacts on the cost functions (12).

Without loss of generality, the strategies in this paper are restricted as linear state feedback strategies, such as ui(t) = Fix(t), where Fi is a constant matrix of suitable sizes.

Let FN denote the set of all (F1x(t), F2x(t), ⋯ , FNx(t)) such that the following closed-loop stochastic system

Edx(t)=(A+i=1NBiFi)x(t)dt+A1x(tδ)dt+p=1MApx(t)dwp(t)

is mean-square admissible.

Our problem is to look for a strategy set (u1(),u2(),,uN()) which is called the stochastic Nash equilibrium strategy set for the game, if for each i = 1, 2, ⋯ , Ν, the following inequality holds:

(13)Ji(u1(),,uN(),x(0))Ji(u1(),,ui1(),ui(),ui+1(),,uN(),x(0))

for all x(0) and (F1x(t),F2x(t),,FNx(t)) that satisfy (F1x(t),F2x(t),,FNx(t))FFN.

3.2 Solution to Stochastic Nash Games

The following theorem generalizes the existing results of [11].

Theorem 1

Assume that for all ui(t), i = 1, 2, ⋯ , Ν, the resultant closed-loop system is mean-square admissible. Suppose that Ν real symmetric matrices Pi > 0 and Ν real symmetric matrices Wi > 0 exist such that

(14)Υi(P1,,PN)=[ΞiETPiA1A1TPiEWi]0

where i = 1, 2, ⋯ , Ν,

Ξi=ETPiAAi+AAiTPiE+p=1MApTPiApETPiBiRi1BiTPiE+Qi+WiAAi=Aj=1,jiNBjRj1BjTPjE

Define the strategy set(F1x(t),F2x(t),,FNx(t)) by

(15)ui(t)=Fix(t)=Ri1BiTPiEx(t),i=1,2,,N

Then, (F1x(t),F2x(t),,FNx(t))FFN,and this strategy set denotes the stochastic Nash equilibrium. Furthermore, the optimal value of cost function

(16)Ji(F1x(t),F2x(t),,FNx(t),x(0))E[xT(0)ETPiEx(0)]+E[δ0φT(τ)Wiφ(τ)dτ]

Proof

Now, let us consider the following problem in which the cost functional (17) is minimal at ui(t)=ui(t).

(17)ϕ(ui(),x(0))=E0[xT(t)Qix(t)+uiT(t)Riui(t)]dt

where x(t) follows from

(18){Edx(t)=[AAix(t)+A1x(tδ)+Biui(t)]dt+p=1MApx(t)dwp(t)x(t)=φ(t),t[δ,0]

Note that the function Φ coincides with function J(u(·), x(0)) in Lemma 1. Applying Lemma 1 to this optimization problem as

AAiA,BiB,QiQ,RiR

yields the fact that the function Φ is minimal at

(19)ui(t)=Fix(t)=Ri1BiTPiEx(t)

Moreover, the optimal value of cost function is less than or equal to

E[xT(0)ETPiEx(0)]+E[δ0φT(τ)Wiφ(τ)dτ]

This completes the proof.

Remark 1

Note that when rank(E) = r = n, i.e., E = I, the inequality (14) is a normal matrix inequalities. This type of matrix inequalities was proposed in [11]. In this section, it is extended to the stochastic singular time-delay system case and it has more universality than the stochastic delay system.

Remark 2

Nash strategy Fix(t) of (15) can be obtained by solving the matrix inequalities (14). It should be noted that the matrix inequalities (14) can be assessed by applying the Newton’s iterative method, which was proposed in [11].

4 Application to Stochastic H2∕H Control

Over the last decade, stochastic control problems governed by Itô’s differential equation have attracted considerable research interest. Recently, stochastic linear quadratic and H control problems with state- and control-dependent noise have been investigated (see, e.g., [13, 14]). They have received much attention and have been widely used in various fields. In particular, the stochastic H2∕H control with state dependent noise and state, control and disturbance-dependent noise have been addressed (see, e.g., [10, 15, 16]), but up to present, stochastic H2∕H control with multiple decision makers have not been reported, and the design of such strategy is an issue that remains to be considered.

Now, we apply the above proposed theory to solve some problems related to stochastic H2∕H control with multiple decision makers.

Consider the following stochastic controlled system with state-dependent noise, which involve N-decision makers

(20){Edx(t)=[Ax(t)+A1x(tδ)+Bv(t)+i=1NBiui(t)]dt+Apx(t)dw(t)zi(t)=[Cix(t)Diui(t)],z(t)=[Cx(t)D1u1(t)DNuN(t)]x(t)=φ(t),t[δ,0]

where DiTDi=Imi,C=[C1TC2TCNT]T,x(t)RRn,z(t)RRm,v(t)RRl,ui(t)RRmi stand for the system state, controlled output, exogenous disturbance signal and i-th control input, respectively. All coefficient matrices are assumed to be real constant. Given a disturbance attenuation level γ > 0, define performance functions

(21)J0(u1,u2,,uN,v)=E0[γ2vT(t)v(t)zT(t)z(t)]dt

and

(22)Ji(u1,u2,,uN,v)=E0ziT(t)zi(t)dt

The infinite-time horizon stochastic H2∕H control with multiple decision makers of system (20) can be stated as follows.

Definition 3

For any given disturbance attenuation level γ > 0, find if possible strategies ui(t)RRmi,i=1,2,,N such that:

  1. ui(t) makes system (20) mean-square admissible, i.e., when v(t) = 0 and ui(t)=ui(t), the closed-loop system is regular, impulse-free and mean-square stable.

  2. (23)Lui2=supv0,x(0)=φ(t)0,t[δ,0]E0[Cx(t)2+i=1Nui(t)2]dtE0[v(t)2]d2<γ2
  3. When the worst case disturbance v*(t) ∈ Rl, if it exists, is applied to (20), ui(t) minimizes the output energy

    (24)Ji(u1,u2,,uN,v)=E0zi(t)2dt=E0(Cix(t)2+ui(t)2)dt

If the above (u1,u2,,uN,v) exist, we say that the infinite-time horizon stochastic H2∕H control with multiple decision makers is solvable. Obviously, (u1,u2,,uN,v) are the Nash equilibria of the two functions (21) and (22), which satisfy

(25)J0(u1,u2,,uN,v)J0(u1,u2,,uN,v),vRRl
(26)Ji(u1,u2,,uN,v)Ji(u1,u2,,ui1,ui,ui+1,,uN,v),uiRRmi

According to Theorem 1 discussed in Section 3, a solution to the stochastic H2∕H control can be obtained straightly.

Theorem 2

Assume that for all ui(t),i = 1, 2, ⋯ , N, the resultant closed-loop system is mean-square admissible. Suppose that Ν + 1 real symmetric matrices (P0, P1, · · · , PN) with Pi > 0, and Ν + 1 real symmetric matrices (W0, W1, ⋯ , WN) with Wi > 0 exist such that

(27)Υ0(P0,P1,PN)=[Ξ0ETP0A1A1TP0EW0]0
(28)Υi(P0,P1,PN)=[ΞiETPiA1A1TPiEWi]0

where i = 1, 2, ⋯ , Ν,

Ξ0=ETP0AAF+AAFTP0E+ApTP0Ap+W0CTCj=1NKjTKjγ2ETP0BBTP0EΞi=ETPiAAi+AAiTPiE+ApTPiAp+CiTCi+ETWiEETPiBiBiTPiEAAF=A+i=1NBiKi,AAi=A+BF+j=1,jiNBjKjF=γ2BTP0E,Ki=BiTPiE

If system(20)is mean-square admissible, then the set(u1,u2,,uN)with

(29)ui(t)=Kix(t)=BiTPiEx(t),i=1,2,,N

denotes the infinite-time horizon stochastic H2∕Hcontrol. Moreover, the worst case disturbance

(30)v(t)=Fx(t)=γ2BTP0Ex(t)

5 Conclusions

In the present paper, we have dealt with the Nash game for stochastic singular time-delay systems with multiple decision makers in infinite-time horizon. In terms of a set of linear matrix inequalities, sufficient condition for the existence of Nash strategies is given for the first time. Moreover, the infinite-time horizon stochastic H2/H control with multiple decision makers are treated by using these obtained results.


Supported by the National Natural Science Foundation of China (71171061), China Postdoctoral Science Foundation (2014M552177), and Guangdong Natural Science Foundation (2014A030310366, 2015A030310218)


References

[1] Lewis F L. A survey of linear singular systems. Circuits, Systems and Signal Processing, 1986, 5(1): 3–36.10.1007/BF01600184Search in Google Scholar

[2] Brenan K E, Campbell S L, Petzold L R. Numerical solution of initial-value problems in differential-algebraic equations. SIAM, Philadelphia, 1996.10.1137/1.9781611971224Search in Google Scholar

[3] Dai L. Singular control systems. Springer, New York, 1989.10.1007/BFb0002475Search in Google Scholar

[4] Fridman E. Stability of linear descriptor systems with delay: A Lyapunov-based approach. Journal of Mathematical Analysis and Applications, 2002, 273(1): 24-44.10.1016/S0022-247X(02)00202-0Search in Google Scholar

[5] Lan W, Huang J. Semiglobal stabilization and output regulation of singular linear systems with input saturation. IEEE Transactions on Automatic Control, 2003, 48(7): 1274-1280.10.1109/TAC.2003.814276Search in Google Scholar

[6] Xu S, Van Dooren P, Stefan R, et al. Robust stability and stabilization for singular systems with state delay and parameter uncertainty. IEEE Transactions on Automatic Control, 2002, 47(7): 1122–1128.10.1109/TAC.2002.800651Search in Google Scholar

[7] Xu S, Lam J. Robust stability and stabilization of discrete singular systems: An equivalent characterization. IEEE Transactions on Automatic Control, 2004, 49(4): 568-574.10.1109/TAC.2003.822854Search in Google Scholar

[8] Boukas E K, Liu Z K. Deterministic and stochastic time delay systems. Birkhauser, Boston, 2002.10.1007/978-1-4612-0077-2Search in Google Scholar

[9] Øksendal B. Stochastic differential equations. Springer, New York, 2003.10.1007/978-3-642-14394-6Search in Google Scholar

[10] Chen B S, Zhang W. Stochastic -H2/H control with state-dependent noise. IEEE Transactions on Automatic Control, 2004, 49(1): 45-57.10.1109/TAC.2003.821400Search in Google Scholar

[11] Mukaidani H. Dynamic games for stochastic systems with delay. Asian Journal of Control, 2013, 15(5): 1251-1260.10.1002/asjc.686Search in Google Scholar

[12] Zhang Q, Xing S. Stability analysis and optimal control of stochastic singular systems. Optimization Letters, 2014, 8(6): 1905-1920.10.1007/s11590-013-0687-5Search in Google Scholar

[13] Rami M A, Zhou X Y. Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Transactions on Automatic Control, 2000, 45(6): 1131–1143.10.1109/9.863597Search in Google Scholar

[14] Zhang W, Chen B S. State feedback H control for a class of nonlinear stochastic systems. SIAM Journal on Control and Optimization, 2006, 44(6): 1973-1991.10.1137/S0363012903423727Search in Google Scholar

[15] Zhang W, Zhang H, Chen B S. Stochastic -H2∕H control with (x,u, u)-dependent noise: Finite horizon case. Automatica, 2006, 42(11): 1891-1898.10.1109/ICARCV.2006.345327Search in Google Scholar

[16] Zhang W, Huang Y, Xie L. Infinite horizon stochastic H2/H control for discrete-time systems with state and disturbance dependent noise. Automatica, 2008, 44(9): 2306–2316.10.1016/j.automatica.2008.01.028Search in Google Scholar

Received: 2015-4-12
Accepted: 2015-6-29
Published Online: 2015-10-25

© 2015 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 17.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/JSSI-2015-0472/html
Scroll to top button