Systems & Control Letters 62 (2013) 37–42

Contents lists available at SciVerse ScienceDirect

Systems & Control Letters

journal homepage: www.elsevier.com/locate/sysconle

Observability and detectability of discrete-time stochastic systems withMarkovian jumpLijuan Shen a,b, Jitao Sun b,∗, Qidi Wu a

a Department of Control Science and Engineering, Tongji University, Shanghai 201804, Chinab Department of Mathematics, Tongji University, Shanghai 200092, China

a r t i c l e i n f o

Article history:Received 19 July 2011Received in revised form6 June 2012Accepted 19 October 2012Available online 6 December 2012

Keywords:DetectabilityObservabilityStochastic systemsMarkov jump

a b s t r a c t

This paper studies detectability and observability of Markov jump discrete-time linear stochastic systemswith multiplicative noise (MJDLS for short). The relations between some concepts of detectability andobservability are established. These relations, together with equivalent expressions of observabilityGramian allow us to obtain some other sufficient and necessary conditions of detectability andobservability. As applications, discrete-time stochastic Lyapunov equations and stochastic discretealgebraic Riccati equations (SDARE) are discussed.

© 2012 Elsevier B.V. All rights reserved.

1. Introduction

Over the last half century, stochastic systems have attractedconsiderable interest (see for instance the monograph [1] andpapers [2,3]) motivated by their widespread use. On the otherhand, Markov jump linear stochastic systems, comprising a classof hybrid systems with random switching structure, have beenextensively investigated in the last two decades. Among others,[4–8] can be cited as important contributions on this subject.

Observability and detectability play an essential role inlinear system theory (see e.g., [7,9–18]). In recent years, manyresearchers have extended these two fundamental concepts tostochastic systems, and found many new phenomena differentfrom deterministic system theory, see [4,10–14]. Among theseresults, it has been observed that many problems can begeneralized analogously from deterministic cases to stochasticsystems. For example, one usually defines a system to bedetectableif all its unstable modes produce non-zero output. As in thedeterministic framework, [11,13] chose the stabilizability of a dualsystem as a defining property to introduce stochastic detectability.However, the property that all unstable modes produce some non-zero output is only a necessary but not a sufficient condition forstabilization of the dual system. Furthermore, this definition ofdetectability has no simple equivalent algebraic criterion like theHautus-test for deterministic systems. To deal with this problem,

∗ Corresponding author. Fax: +86 21 65981985.E-mail address: sunjt@sh163.net (J. Sun).

0167-6911/$ – see front matter© 2012 Elsevier B.V. All rights reserved.doi:10.1016/j.sysconle.2012.10.005

a weaker detectability, was proposed for discrete-time Markovjump linear systems in [7] based on the standard concepts ofdetectability in [10] for time-varying systems. Another naturalconcept of detectability, based on the spectral technique, wasrecently given in [15] accompanied with the Hautus-test likethat for the deterministic systems, and these results were laterextended by [17], which studied exact detectability and exactobservability of systems with Markov jumps and multiplicativenoise.

The concept of observability is always related to that ofdetectability, see, e.g., [7,14]. [18] first introduced the exactobservability for stochastic Itô systems, which, in [14] is shownequivalent to observability discussed in [7]. Also, [16] introducedthe concept of stochastic observability which is weaker than thosediscussed in [8,19].

Unlike in the deterministic setting, for stochastic systems, thesedifferent concepts of detectability and observability are definedin different frameworks. A natural question arises: what is therelationship between these different definitions? That is one of ourmotivations to investigate the relationship between these differentdefinitions in this paper.

On the other hand, to the best of our knowledge, there existscarce reports on the observability and detectability of MJDLSexcept for [4]. Therefore, to carry out the study on detectability andobservability of MJDLS is meaningful and challenging.

This paper will investigate detectability and observability ofMJDLS. The techniques in this paper involve mainly the equivalentexpressions of observability Gramian E(m, X). We will establishthe relations between different concepts of detectability andobservability of MJDLS. These relations, together with equivalent

38 L. Shen et al. / Systems & Control Letters 62 (2013) 37–42

expressions of observability Gramian not only allow us to obtainthe Hautus-test, but also lead to some results about detectabilityand observability of MJDLS. By utilizing a different but simplermethod, we propose a characterization of unobservable space,which generalizes the results in [20] to discrete-time linearstochastic systems with multiplicative noise and Markov jump.

This paper is organized as follows. In Section 2, some basic factsand notations are recalled. In Section 3, sufficient and necessaryconditions of detectability and observability are obtained bymeansof equivalent representations of E(m, X). Finally, concludingremarks are given in Section 4.

2. Preliminaries

Let Rn⊆ Rn×n represent the normed linear space of n × n

symmetric matrices and Sn,N denote the linear space of H =

(H(1), . . . ,H(N)), ordered by the closed convex cone S+ of non-negative matrices H = (H(1), . . . ,H(N)), H(i) ≥ 0, i ∈ D =

1, 2, . . . ,N. HereH(i) ≥ 0meansH(i) is a positive semi-definitematrix. It is known Sn,N equipped with the inner product

⟨U, V ⟩ =

Nk=1

tr(UTk Vk) (1)

for all U = (U(1), . . . ,U(N)) and V = (V (1), . . . , V (N)) ∈ Sn,N

organizes a Hilbert space.Consider the discrete-time linear stochastic systems with

multiplicative noise and Markov jump as follows

x(k + 1) =

A0(r(k)) +

lp=1

Ap(r(k))wp(k)

x(k),

y(k) =

C0(r(k)) +

lp=1

Cp(r(k))wp(k)

x(k), (2)

with initial value x(0) = x0, r(0) ∼ µ0. Here x(k) ∈ Rn and y(k) ∈

Rm are respectively the state vector and the measured output. Thehomogeneous discrete-time Markov chain r = r(k), k ≥ 0takes a value on the set D with the transition probability matrixP = (pij), i, j = 1, . . . ,N . The initial distribution of r is determinedby µ0,i = P(r(0) = i), i = 1, 2, . . . ,N · wp(k) ∈ R, p = 1, . . . , lis a sequence of independent white noise defined on a completeprobability space Ω, F , P with E(wi(k)wj(k)) = δij, E(wi(k)) =

0, i, j = 1, . . . , l. Throughout the paper, the Markov chain r =

r(k), k ≥ 0 is assumed to be independent of w(k), k ≥ 0with w(k) = (w1(k), . . . , wl(k))T . For convenience, set A =

(A0, A1, . . . , Al), where Ap stands for the sequence Ap(i) ∈ Rn×n,i ∈ D, p = 0, 1, . . . , l.

For the sequences Ap(i) and pij define the operatorsLA (or, sim-ply, L, if no confusions occur) by LH = ((LH)(1), . . . , (LH)(N))for H = (H(1), . . . ,H(N)) ∈ Sn,N by

(LH)(i) =

lp=0

Nj=1

pijATp(i)H(j)Ap(i), i ∈ D, (3)

and its adjoint operator L∗ with respect to inner product (1) isgiven by (L∗H)(i) =

lp=0

Nj=1 pjiAp(j)H(j)AT

p(j). Noticing pij ≥

0, L and L∗≥ 0 obviously L satisfies LS+

⊂ S+ and thus is apositive operator.

To proceed, recall the following notions

X(k) = Ex(k)xT (k), Xi(k) = Ex(k)xT (k)Ir(k)=i, (4)

Y (k) = Ey(k)yT (k), Yi(k) = Ey(k)yT (k)Ir(k)=i, (5)

X(k) = (X1(k), X2(k), . . . , XN(k)) ∈ Sn,N . (6)

Similar to [7,8], for X(k) as defined in (4), define the observabilityGramian

E(m, X) =

m−1k=0

X(k),

lp=0

Cp

(7)

where X(0) = X , Cp = (CTp (1)Cp(1), . . . , CT

p (N)Cp(N)) with com-ponents Cp(i), i = 1, 2, . . . ,N . To understand it well, we providesome equivalent expressions

E(m, X) =

m−1k=0

X(k),

lp=0

Cp

(8a)

=

m−1k=0

lp=0

tr

E(CTp (r(k))x(k)xT (k)Cp(r(k)))

(8b)

=

m−1k=0

tr(Y (k)) =

m−1k=0

E∥y(k)∥2, (8c)

which implies E(m, X) represents the output energy accumulatedup tom. (8) provides various equivalent definitions of E(m, x) andthat makes it possible for us to choose the appropriate one to ob-tain useful results.

Definition 1. We say that system (2) (or equivalently (A, C)) isdetectable, if there exist integers τd, kd and scalars γ > 0, 0 ≤

δ < 1 such that E(τd, X) ≥ γ ∥X∥ whenever ∥X(kd)∥ ≥ δ∥X∥.

Definition 2. We say that system (2) (or equivalently (A, C)) isobservable, if there exist an integer τd and a scalar γ > 0, suchthat E(τd, X) ≥ γ ∥X∥ for each initial condition X .

We call x0 an unobservable state, if the corresponding outputresponse y(k) ≡ 0, a.s., k > 0.

Remark 1. By Definition 2, all unobservable states make asubspace, denoted as N0. Definition 1 adopts the standarddefinitions of detectability of linear time-varying systems, seee.g. [10,12]. It is also worth mentioning that such definitions arealso used to study the detectability and observability of discrete-time linear systems, respectively, in [7] with Markovian jumpand [14] subject to independent random perturbations.

Motivated by Damm [15], Ni et al. [17] and Zhang et al. [20],we can also introduce the definitions of exact observability, exactdetectability and stochastic detectability of system (2) as follows.

Definition 3. System (2) is said to be mean square stable, or inshort A is mean square stable, if for each x0 and µ0

limk→∞

E|x(k)|2 = 0. (9)

Definition 4. System (2) is said to be exactly observable if N0 =

0. System (2) is said to be exactly detectable if y(k) ≡ 0 for allk > 0 implies liml→∞ E|x(l)|2 = 0.

System (2) is said to be stochastically detectable if there existsa collection of matrices K(r(k)), r(k) ∈ D, such that the followingclosed-loop system of (2)

x(k + 1) = (A0(r(k)) + K(r(k))C0(r(k)))T x(k) +

lp=1

(Ap(r(k))

+ K(r(k))Cp(r(k)))T x(k)wp(k) (10)

is mean square stable.

Throughout the paper, ‘‘system (2) is detectable or observable’’always refers to the corresponding properties in the sense ofDefinition 1 or 2 unless otherwise specified.

L. Shen et al. / Systems & Control Letters 62 (2013) 37–42 39

Consider the recursive sequence Q as

Qi(k) =

lp=0

Cp(i) +

lp=0

Nj=1

pijATp(i)Qj(k − 1)Ap(i),

i ∈ D, k > 0, (11)

with Q(0) = 0.

Lemma 1. Consider system (2) and X(k) defined as in (4). Then X(k)satisfies the following deterministic dynamics

Xi(k + 1) = L∗

i X(k) with X(0) = X and (12a)

E(m, X) = ⟨X, Q(m)⟩. (12b)

Proof. Using (2) and the definition of conditional expectation, weget

Xi(k + 1) = Ex(k + 1)xT (k + 1)Ir(k+1)=i

=

Nj=1

E(x(k + 1)xT (k + 1)Ir(k+1)=iIr(k)=j)

=

Nj=1

E

A0(j) +

lp=1

Ap(j)wp(k)

x(k)

A0(j)

+

lp=1

Ap(j)wp(k)

x(k)

T

Ir(k+1)=iIr(k)=j

=

Nj=1

pjil

p=0

Ap(j)Xj(k)ATp(j) = L∗

i X(k).

To check the second relation, we adopt the induction method.Clearly, (12b) holds whenm = 1 since

E(1, X) =

X,

lp=0

Cp

= ⟨X, Q(0)⟩.

Assume (12b) still holds when m = k1, i.e., E(k1, X) = ⟨X, Q(k1)⟩.Now consider the casem = k1 + 1. In fact,

E(k1 + 1, X) =

k1k=0

X(k),

lp=0

Cp

=

X,

lp=0

Cp

+

k1k=1

X(k),

lp=0

Cp

. (13)

By the definition of E(m, X),k1k=1

X(k),

lp=0

Cp

=

k1−1k=0

X(k + 1),

lp=0

Cp

= ⟨X(1), Q(k1)⟩

= ⟨L∗X, Q(k1)⟩ = ⟨X, LQ(k1)⟩. (14)

Substituting (14) into (13), one can check that by (11)

E(k1 + 1, X) =

X,

lp=0

Cp

+ ⟨X, LQ(k1)⟩ = ⟨X, Q(k1 + 1)⟩,

so (12b) holds whenm = k1 + 1. That completes the proof.

By using the similar method in [7], we can obtain the followinglemma.

Lemma 2. System (2) is detectable if and only ifwhenever E(m, X) =

0 one has ∥X(k)∥ → 0 as k → ∞.

3. Detectability and observability

3.1. Detectability

Theorem 1. Consider system (2). The following conclusions areequivalent

(I) (A, C) is detectable.(II) (A, C) is exact detectable.(III)

Xi[CT0 (i), CT

1 (i), . . . , CTl (i)] = 0, i ∈ D

for every eigenvector 0 = X = (X1, . . . , XN) ∈ S+ of L∗

corresponding to some eigenvalue λ with |λ| ≥ 1.(IV) Let M be a nonsingular matrix, then the new evolution system of

x(k), transformed from (2) with x(k) = Mx(k), is detectable.

Proof. (I) ⇔ (II). Note that E(m, X) =m−1

k=0 E∥y(k)∥2=m−1

k=0tr(Y (k)), which implies E(yT (k)y(k)) = 0, k = 1, 2, . . . ,m −

1 ⇔ E(m, X) = 0. By this assertion and Lemma 2 the followingstatements are equivalent.

(a) E(yT (k)y(k)) = 0, k = 1, 2, . . . ,m − 1 ⇒ limk→∞ E∥x(k)∥2

= 0.(b) E(m, X) = 0 ⇒ limk→∞ E(xT (k)x(k)) = 0 ⇔ limk→∞ X(k)

= 0.(c) System (2) is detectable.

from which, and the fact that (a) is the definition of exactdetectability, we can get (I) ⇔ (II).

(II) ⇔ (III). Taking vec on both sides of (12a) gives

vec(Xi(k + 1)) =

lp=0

Nj=1

pjiAp(j) ⊗ Ap(j)vec(Xi(k)),

and we can get further,

vec(X(k + 1)) = Avec(X(k)),with Aji = pjiAp(j) ⊗ Ap(j), i, j = 1, . . . ,N. (15)

Similarly, we also have,

vec(Y(k + 1)) = C vec(Y(k)),with Cji = pjiCp(j) ⊗ Cp(j), i, j = 1, . . . ,N. (16)

Then (2) is exactly detectable iff (15) and (16) is completelydetectable. Thus, by PBH criteria for deterministic systems, it isequivalent to that for every eigenvector X of A corresponding toλ ≥ 0, CX = 0.

On the other hand, let λ1 be an eigenvalue of L∗ and X be thecorresponding eigenvector, then L∗X = λ1X . Thus vec(L∗X) =

Avec(X) = λ1X , i.e., λ1 is also the eigenvalue of A and vecX is thecorresponding eigenvector of A and thus we get (II)⇔ (III).

(I) ⇔ (IV). The dynamics of x can be described as

x(k + 1) =

M−1A0(r(k))M

+

lp=0

M−1Ap(r(k))Mwp(k)

x(k),

y(k + 1) =

C0(r(k))M +

lp=0

Cp(r(k))Mwp(k)x(k)

. (17)

We will prove (I) ⇒ (IV) by contradiction. Assume the argument(IV) is not tenable and there exists an eigenvector X ∈ S+ ofLM−1AM corresponding to eigenvalue λ with |λ| ≥ 1 such that

Xi[(C0M)T (i), (C1M)T (i), . . . , (ClM)T (i)] = 0, i ∈ D

40 L. Shen et al. / Systems & Control Letters 62 (2013) 37–42

that is,

MXiMT[CT

0 (i), CT1 (i), . . . , CT

l (i)] = 0, i ∈ D.

It should be pointed out thatMXMT is an eigenvector of eigenvalueλ with |λ| ≥ 1 of L of (2). By argument (III), (2) is not detectable.

(IV) ⇔ (I) is obvious lettingM be replaced byM−1 in the sameprocedure as discussed above.

The next assertion will show preservability of detectability fora specific system. This result will help us to investigate stochasticdiscrete algebraic Riccati equations (SDARE).

Proposition 1. Assume system (2) is detectable. Then, assigned thesame initial value of (2), the following system

x(k + 1) =

A0(r(k)) +

lp=1

Ap(r(k))wp(k)

x(k),

y(k) =

C0(r(k)) +

lp=1

Cp(r(k))wp(k)

x(k)

+ R12 (r(k))K(r(k))wl+1(k), (18)

is also detectable, where Ai(r(k)) = Ai(r(k)) + Bi(r(k))K(r(k)),R > 0. Bi and K are matrices with appropriate dimensions. wl+1 isa scalar white noise defined on Ω, F , P with E(wl+1(k)wj(k)) =

δ(l+1)j, j = 1, . . . , lE(wl+1(k)) = 0. In addition,wl+1 is independentof wj, j = 1, . . . , l and the Markov chain r.

Proof. In this proof, X(18)(k) refers to system (18). We will showthe conclusion by contradiction. Assuming there exists a K suchthat (18) is not detectable, then by Theorem 1, there exists someeigenvector X ∈ S+ of L(18) corresponding to λ with |λ| ≥ 1 suchthat

Xi

CT0 (i), . . . , CT

l (i), K T (i)R12 (i)

= 0. (19)

Post-multiplying (19) with (Xi[CT0 (i), . . . , CT

l (i), K T (i)R12 (i)])T

yields

Xi

l

p=0

Cp(i) + K T (i)R(i)K(i)

XTi = 0. (20)

So it is immediate to check that

XiK T (i)R(i)K(i)XTi = 0.

Since R(i) > 0, we get K(i)XTi = 0, i ∈ D and substitute it into

L∗

(18) and (19), one can have

(L∗

(18)X)(i) =

lp=0

Nj=1

pji(Ap(j) + Bp(j)K(j))Xi(Ap(j)

+ Bp(j)K(j))T

= (L∗

AX)(i),

Xi[CT0 (i), . . . , CT

l (i)] = 0,

which contradicts Theorem 1. Thus (18) is detectable.

Proposition 2. Assume system (2) is stochastically detectable,then (2) is detectable in the sense of Definition 1.

Proof. Suppose, to the contrary, that (2) is not detectable. ByTheorem 1, there exists X ∈ S+, X = 0 such that L∗X = λX with|λ| ≥ 1,

Xi[CT0 (i), CT

1 (i), . . . , CTl (i)] = 0, (21)

for some i ∈ D. On the other hand, since system (10) ismean squarestable, all eigenvalues ofL∗

(10) are in the inside of the open unit diskand thus β(L∗

(10)) < 1. Substituting (21) into L∗

(10), it shows,

L∗

(10)X = L∗X = λX .

That is a contradiction. The proof is done.

Theorem 2. Assume (A, C) is detectable. If the discrete-time stochas-tic Lyapunov matrix equation

lp=0

ATp(i)

Nj=1

pijP(j)Ap(i) − P(i) = −

lp=0

Cp(i)

admits a solution P ≥ 0 then(1) A is mean square stable.(2) N0 =

m>0 ker(Q(m)) = ker(P).

Proof. (1) To prove the conclusion by contradiction we assumethere exists X ∈ S+ such that

L∗X = β(L)X, β(L) ≥ 1.

By Theorem 1, it follows that

Xi[CT0 (i), CT

1 (i), . . . , CTl (i)] = 0, i ∈ D,

which implies,

E(1, X) =

X,

lp=0

Cp

= tr

X

lp=0

Cp

= 0. (22)

On the other hand, the following relation holds

E(1, X) = ⟨X, P − LP⟩ = ⟨(I − L∗)X, P⟩

= tr(XP − β(L)XP) ≤ 0. (23)

Therefore, from (22) and (23), we obtain E(1, X) < 0, which is acontradiction. Hence, A is mean square stable.

(2) First, consider the left-hand side of the equality. Assumingx0 ∈ N0, we have y(k) = 0 a.s., k > 0 by the definition of N0.Noticing xT0Q(m)x0 = E(m, x0xT0) =

m−1k=0 E∥y(k)∥2

=m−1

k=0tr(Y (k)) = 0,m > 0, which corresponds to the output of (12a)with X = x0xT0 , then it follows xT0Q

12 (m)Q

12 (m)x0 = 0 and

Q12 (m)x0 = 0,m > 0, i.e., x0 ∈

m>0 ker(Q(m)).

Next, we are in a position to prove

m>0 ker(Q(m)) ⊆ N0.Suppose x0 ∈

m>0 ker(Q(m)), thenwehave xT0Qmx0 = 0,m > 0.

Sincem−1

k=0 E∥y(k)∥2= E(m, x0xT0) = xT0Qmx0 = 0,m > 0, we

get that E∥y(k)∥2≡ 0, k > 0 and x0 ∈ N0. Therefore the left-hand

side of the equality holds.Finally, since A is mean square stable, by using a similar

discussion of Lemma 17 in [14], we can obtain ker(P) =m>0 ker(Q(m)).

Remark 2. Clearly, Theorem 2 is a discrete generalization ofTheorem 3.2 in [20] from a stochastic differential equation toa stochastic discrete-time system subject to a Markov jump bydefaulting that x0 satisfy the same deterministic condition. Itshould be pointed out that Theorem 2 adopts a different but moresimple method thanks to the equivalent definitions of E(m, X)in (8).

In fact, the proof of N0 =

m>0 ker(Q(m)) still holds truewithout the hypothesis of detectability.

Theorem 3. Consider system (2). The following statements areequivalent(I) (A, C) is observable.(II) There exists an integer M > 0 such that Q(M) > 0.

L. Shen et al. / Systems & Control Letters 62 (2013) 37–42 41

(III) (A, C) is exact observable.(IV) Xi[CT

0 (i), CT1 (i), . . . , CT

l (i)] = 0, i ∈ D for every eigenvector0 = X = (X1, . . . , XN) ∈ S+ of L∗.

Proof. By Lemma 1, for each initial value X ,

E(m, X) =

m−1k=0

X(k),

lp=0

Cp

=

m−1k=0

(L∗)kX,

lp=0

Cp

=

m−1k=0

X, Lk

lp=0

Cp

.

(I) ⇒ (II). If system (2) is observable, there exists an integerM > 1and a scalar γ > 0 such that E(M, X) ≥ γ ∥X∥ for each initialvalue X , which shows

M−1k=0 LkCp ≥ γ I > 0. We generalize the

technique used in [4] to define Q(M) =M−1

k=0 Lklp=0 Cp, then

it can be easily checked that Q(M) satisfies (11), i.e.,

Q(M + 1) =

Mk=0

Lkl

p=0

Cp = LQ(M) +

lp=0

Cp,

from which we can get the conclusion.(II) ⇒ (I). Set Q(M) =

M−1k=0 Lkl

p=0 Cp. If Q(M) > 0, thenfor each initial value X ,

E(M, X) =

M−1k=0

X(k),

lp=0

Cp

=

X,

M−1k=0

Lkl

p=0

Cp

> 0.

Therefore, we obtain the detectability of (2).(II) ⇔ (III) ⇔ (IV). Noticing N0 =

m>0 ker(Q(m)) in

Remark 2, it is immediate to deduce that N0 = 0 is equivalentto (II). Therefore, (II) ⇔ (III). Noting (IV) is a discrete version ofthe PBH criterion for exact observability in [17], that completes theproof.

Proposition 3. If system

x(k + 1) = A0(r(k))x(k),y(k) = C0(r(k))x(k), (24)

with the same initial value as (2), is observable, then system (2) isobservable.

Proof. Let X(24) and X refer to system (24) and (2), respectively.If (24) is observable, there exists τd > 1 and γ > 0 such thatE(24)(τd, X) =

τd−1k=0 ⟨X(24)(k), CT

0 C0⟩ =τd−1

k=0 ⟨X, Lk(24)C

T0 C0⟩ ≥

γ ∥X∥ for each initial value X . Note that

Lkl

p=0

Cp ≥ Lk(24)C

T0 C0,

equivalently,τd−1k=0

X, Lk

lP=0

Cp − Lk(1)C

T0 C0

≥ 0,

or states further,

E(τd, X) ≥ E(24)(τd, X) ≥ γ ∥X∥,

thus system (2) is observable. That completes the proof.In general, the converse of Proposition 3 does not hold. As an

example consider

A0(1) = a1, A0(2) = a2, C0(1) = 1, C0(2) = 0,A1(1) = b1, A1(2) = b2, C1(1) = c1, C1(2) = 1.

From (11) calculate that Q1(1) = 1 + c21 > 0 and Q1(2) = 1 > 0.System (A, C) is observable from Theorem 3. However, (A0, C0) isnot observable since Q2 = 0.

Proposition 4. Suppose system (2) is observable. Then, assigned thesame initial value of (2), system (18) is also observable.Proof. Let X(18)(k) refer to the system (18). We show theconclusion by contradiction. Assume there exists a K such that (18)is not observable, then by Theorem 3, there exists a vector η = 0such that ηTQ(18)(m)η = 0 for any integers m > 0. By Lemma 1,

0 = E(18)(m, ηηT ) =

m−1k=0

X(18)(k),

lp=0

Cp + K TRK

≥

m−1k=0

⟨X(18)(k), K TRK⟩,

from which we deduce KX(18)(k) = 0 and

m−1k=0

X(18)(k),

lp=0

Cp

= 0. (25)

Then using the same procedure as in Proposition 1, L∗

(18) = L∗.Note that (18) has the same initial value as (2), then using Lemma1,X(18)(k) = X(k). Substituting it into (25) one can get E(m, X) =

0 for arbitrary m. Therefore, system (2) is not observable. Thatcontradicts the assumption that (2) is observable.

3.2. Applications

The study of stochastic LQ control for the system governed bythe Itô equation can be traced back to the work of Wonham [21]and received much attention. As for our theoretical applications,we give here some results about the following SDARE

P(i) =

lp=0

ATp(i)EiAp(i) − MT

i R−1i Mi +

lp=0

Cp(i), i ∈ D (26)

with Ei =N

j=1 pijP(j), Mi =l

p=0 BTp(i)EiAp(i), Ri = Ri +l

p=0 BTp(i)EiBp(i) > 0, Ri > 0, i ∈ D are given matrices.

Proposition 5. Assume SDARE (26) admits a solution P ≥ 0, thenwe have the following statements.(1) If (A, C) is detectable, then P is a feedback stabilizing solution

of (26), i.e., (18) is mean square stable with K(i) = −R−1i Mi

[22].(2) If (A, C) is observable, then P is a feedback stabilizing solution

of (26).Proof. Letting K(i) = −R−1

i Mi, (26) can be written equivalentlyas

P(i) =

lp=0

(Ap(i) + Bp(i)K(i))TEi(Ap(i) + Bp(i)K(i))

+ K T (i)RiK(i) +

lp=0

Cp(i),

which is, in fact, the stochastic Lyapunov equation associated with(18) discussed in Propositions 1 and 4.(1) If (A, C) is detectable, then by Proposition 1, (18) is detectable

and mean square stable from Theorem 2, i.e., P is a stabilizingsolution to (26).

(2) Clearly, (2) holds true since observability implies thedetectability.

4. Conclusions

This paper has established relations between different conceptsof detectability and observability of MJDLS. Using the relationsand equivalent expressions of observability Gramian, we get

42 L. Shen et al. / Systems & Control Letters 62 (2013) 37–42

some necessary and sufficient conditions of detectability andobservability.

Acknowledgments

The authors would like to thank the Associate Editorand allthe reviewers for their helpful comments and suggestions whichhave helped to considerably improve the quality of the paper. Thiswork is supported by the NNSF of China under Grants 60904027,61034004, 61174039 and 11201215, China Postdoctoral ScienceFoundation (No. 2012M520928) and the Fundamental ResearchFunds for the Central Universities of China.

References

[1] B. Oksendal, Stochastic Differential Equations, sixth ed., Springer, 2003.[2] L.J. Shen, J.P. Shi, J.T. Sun, Complete controllability of impulsive stochastic

integrodifferential systems, Automatica 46 (6) (2010) 1068–1073.[3] H.I. Nurdin, On synthesis of linear quantum stochastic systems by pure

cascading, IEEE Trans. Automat. Control 55 (10) (2010) 2439–2444.[4] V. Dragan, T. Morozan, Observability and detectability of a class of discrete-

time stochastic linear systems, IMA J. Math. Control I 23 (3) (2006) 371–394.[5] X.R. Mao, J. Lam, L.R. Huang, Stabilisation of hybrid stochastic differential

equations by delay feedback control, Syst. Control Lett. 57 (2008) 927–935.[6] S. Aberkane, Stochastic stabilization of a class of nonhomogeneous Markovian

jump linear systems, Syst. Control Lett. 60 (3) (2011) 156–160.[7] E.F. Costa, J.B.R. Do Val, On the detectability and observability of discrete-time

Markovian jump linear systems, Syst. Control Lett. 44 (2001) 135–145.[8] Y. Ji, H.J. Chizeck, Controllability, stabilizability, and continuous-time Marko-

vian jump linear quadratic control, IEEE Trans. Automat. Control 35 (1990)777–788.

[9] S.L. Shu, F. Lin, Generalized detectability for discrete event systems, Syst.Control Lett. 60 (5) (2011) 310–317.

[10] B.D.O. Anderson, J.B. Moore, Detectability and stabilizability of time-varyingdiscrete-time linear systems, SIAM J. Control Optim. 19 (1) (1981) 20–32.

[11] B.S. Chen, W. Zhang, Stochastic H2/H∞ control with state-dependent noise,IEEE Trans. Automat. Control 49 (1) (2004) 45–57.

[12] E.F. Costa, J.B.R. Do Val, On the observability and detectability of continuous-time Markov jump linear systems, SIAM J. Control Optim. 41 (4) (2005)1295–1314.

[13] T. Damm, Rational Matrix Equations in Stochastic Control, in: Lecture Notes inControl and Information Sciences, vol. 287, Springer-Verlag, New York, 2004.

[14] Z.Y. Li, Y. Wang, B. Zhou, et al., Detectability and observability of discrete-time stochastic systems and their applications, Automatica 45 (5) (2009)1340–1346.

[15] T. Damm, On detectability of stochastic systems, Automatica 43 (2007)928–933.

[16] V. Dragan, T. Morozan, Stochastic observability and applications, IMA J. Math.Control I 21 (2004) 323–344.

[17] Y.H. Ni, W.H. Zhang, H.T. Fang, On the observability and detectability of linearstochastic systems with Markov jumps and multiplicative noise, J. Syst. Sci.Complex. 23 (1) (2010) 102–115.

[18] W. Zhang, B.S. Chen, On stabilizability and exact observability of stochasticsystems with their applications, Automatica 40 (2004) 87–94.

[19] T. Morozan, Optimal stationary control for dynamic systems with Markovperturbations, Stoch. Anal. Appl. 1 (1983) 299–325.

[20] W.H. Zhang, H.S. Zhang, B.S. Chen, Generalized Lyapunov equation approachto state-dependent stochastic stabilization/detectability criterion, IEEE Trans.Automat. Control 53 (7) (2008) 1630–1642.

[21] W.M. Wonham, On a matrix Riccati equation of stochastic control, SIAM J.Control 6 (1968) 312–326.

[22] M. Ait Rami, X.Y. Zhou, Linear matrix inequalities, Riccati equations, andindefinite stochastic linear quadratic control, IEEE Trans. Automat. Control 45(2000) 1131–1142.