SDRE-1

State-Dependent Riccati Equation (SDRE) Design for Nonlinear Control Systems

A systematic design technique for nonlinear control

systems. The SDRE strategy is nothing but pointwisely apply the LQR scheme at every nonzero state.

SDRE-2

Outline

Linear Quadratic Regulator (LQR)

Algebraic Riccati Equation (ARE)

State-Dependent Riccati Equation (SDRE)

SDRE-3

The LQR Design Consider the following linear control system and performance index: and where

Objective: To organize a control u such that the origin of the closed-loop system is stable and the cost J is minimum.

xç = Ax +Bu

x 2 Rn; u 2 Rp; Q0 õ 0 & R0 > 0:

J =R0

1(xQ0x + uR0u)dt

SDRE-4

LQR solution: If (A,B) is stabilizable and is detectable, then the optimal control is known to be

where is the solution of the following ARE:

Question: Does the solution of the ARE stated above always exist?

u = àRà10BTPx

ATP + PA à PBRà10BTP +Q0 = 0:

(A; Q0

p)

P = PT õ 0

SDRE-5

Algebraic Riccati Equation (ARE)

Lyapunov equations are useful in system analysis, while AREs are useful in control system synthesis.

Recall the ARE has the form: Let (ARE)

where

ATP + PA à PBRà10BTP +Q0 = 0:

X = P; Q = Q0 & R = BRà10BT

ATX +XA àXRX +Q = 0:

R = RT õ 0 & Q = QT õ 0:

SDRE-6

If n = 1, the ARE becomes

Note that, at most one x such that Question: Under what conditions does the solution X of

the ARE exist and satisfy (i) real and symmetric. (ii) (iii)

à rx2 + 2ax + q = 0

õ(A àRX) = õ(A àBRà10BTP) ò Cà:

) x =r

aæ a2+rqp

) a à rx = æ a2 + rqp

õ(a à rx) ò Cà

ATX +XA àXRX +Q = 0:

SDRE-7

Associated with the ARE where we introduce the following Hamiltonian matrix

Question: What properties does the Hamiltonian matrix have?

ATX +XA àXRX +Q = 0;

H :=A àRàQ àAT

ò ó2 R2nâ2n

A; Q; R 2 Rnân; R = RT õ 0 & Q = QT õ 0;

SDRE-8

Lemma 1: The spectra of H, are symmetric about the real and the imaginary axes. That is,

Proof: Let

are similar. Thus, Moreover, because H is a real

matrix. This completes the proof.

õ(H);

õ 2 õ(H) ) à õ;æ õö 2 õ(H):

Ð :=0 à InIn 0

ò ó) Ð2 = à I2n ) Ðà1 = à Ð:

) Ðà1HÐ = à ÐHÐ = àHT

) H & àHT

õ 2 õ(H) , à õ 2 õ(H):

õ 2 õ(H) , õö 2 õ(H);

ä

SDRE-9

Corollary 2: Suppose that H has no imaginary eigenvalues. Then there are exactly n eigenvalues on and n eigenvalues on

Question: When does H have no imaginary eigenvalues?

To be discussed later.

Cà

C+

SDRE-10

Question: How to solve and what are the solutions of the ARE?

Theorem 3: Let be an invariant subspace of H and

, where and is invertible.

Then

(i) is a solution of the ARE.

(ii)

(iii) The solution X is independent of the choice of the basis of That is, X is unique whenever the invariant subspace is given.

V ò C2n

V = ImX1

X2

ò óX1; X2 2 Cnân X1

X := X2Xà11

V:

õ(A àRX) = õ(HjV):

V

SDRE-11

Proof: To prove (i): is an invariant subspace of H Pre-multiply Eq. (1) by we have X is a solution of the ARE.

Xà11

)

A àRàQ àAT

ò óX1

X2

ò ó=

X1

X2

ò óË

V

) 9Ë 2 Cnân s:t:

) A àRàQ àAT

ò óInX

ò ó= In

X

ò óX1ËX

à11

(1) (post-multiply )

(àX; In)A àRàQ àAT

ò óInX

ò ó= àATX àXA +XRX àQ = 0:

(àX; In)

SDRE-12

To prove (ii): By the first n equations of (1) we have

To prove (iii): Any basis spanning can be represented as for some nonsingular matrix M. which is independent of the matrix M. ä

) X = (X2M)(X1M)à1 = X2Xà11;

X1

X2

ò óM =

X1MX2M

ò óV

A àRX = X1ËXà11

) õ(A àRX) = õ(Ë) = õ(HjV):

(2)

SDRE-13

Corollary 4: If is an invariant subspace of H,

where Then (i)

(ii)

Proof: (i) follows from Theorem 3 by setting while (ii) follows from Eqs. (1) and (2).

Im InX

ò ó

X 2 Cnân:

ATX +XA àXRX +Q = 0:

H InX

ò ó= In

X

ò ó(A àRX):

X1 = In & X2 = X;

ä

SDRE-14

Example 1: Let

Eigenvalues and eigenvectors: − − The eigenvector and generalized eigenvector associated

with are − The eigenvector and generalized eigenvector associated

with are All solutions of the ARE:

− span is H-invariant. Let

is a solution

X1

X2

ò ó= fv1; v2g

õ = 1

A = à 3 2à 2 1

ò ó; R = 0 0

0 1

ò ó& Q = 0 0

0 0

ò ó:

õ(H) = f1; 1;à 1;à 1g:

v1 = (1; 2; 2;à 2)T & v2 = (à 1;à 1:5; 1; 0)T

v3 = (1; 1; 0; 0)T & v4 = (1; 1:5; 0; 0)Tõ = à 1

fv1; v2g

) X = X2Xà11

= à 10 66 à 4

ò ó& õ(A àRX) = f1; 1g

SDRE-15

− span is H-invariant. Let

is a solution

− span is H-invariant. Let

is a solution and

− span span and span are not H-invariant. One can show that the matrices constructed from those vectors are not solutions of the ARE.

Note that, there is only one solution such that

X1

X2

ò ó= fv3; v4g

ä

fv3; v4g

) X = X2Xà11

= 0 & õ(A àRX) = fà 1;à 1g

) X = X2Xà11

= à 2 22 à 2

ò ófv1; v3g X1

X2

ò ó= fv1; v3g

õ(A àRX) = f1;à 1g:

fv1; v4g; fv2; v3g; fv2; v4g

õ(A àRX) ò Cà:

SDRE-16

Remark 1: When H has no imaginary eigenvalues, then there always exist two real matrices such that

and Moreover, can be

chosen to be nonsingular if there exists a , stated in Theorem 3, which is nonsingular.

Hint: If is nonsingular, then

(i) Columns of are either real or appear in complex conjugate. (ii) Construct a new real matrix from by taking its real and imaginary parts. (iii) Use the fact and the last matrix is nonsingular.

X1; X2 2 Rnân

V = ImX1

X2

ò óõ(HjV) ò Cà:

X1

X2

ò ó2 C2nân & X1

X1

X2

ò ó

(x + iy; x à iy) = (x; y) 1 1i à i

ò ó

X1 2 Cnân

X1

X1

X2

ò ó

2 â 2

SDRE-17

Definition: Let be the set containing the Hamiltonian matrices with the following properties:

(i) Each has no imaginary eigenvalues. (ii) The real matrix of the n-dimensional stable H-invariant subspace basis is nonsingular. We define the mapping where X is the associated ARE solution.

Note that, from (iii) of Theorem 3, the mapping Ric is well-defined. That is,

Ric : H ! Rnân; Ric(H) = X;

H ò R2nâ2n

H 2 H

X1

8H 2 H; 9!X s:t: Ric(H) = X:

X1

X2

ò ó

SDRE-18

Theorem 5: Suppose that Then (i) X is real and symmetric; (ii) (iii)

Proof: From Remark 1, X can be chosen to be real. It remains to show that X is symmetric. From (ii) of Corollary 4

Pre-multiply on both sides

H 2 dom(Ric) & X = Ric(H):

õ(A àRX) ò Cà:

ATX +XA àXRX +Q = 0

H InX

ò ó= In

X

ò óË; where õ(Ë) 2 Cà:

InX

ò óT

Ð; Ð =0 à InIn 0

ò ó;

) InX

ò óT

ÐH InX

ò ó= In

X

ò óT

Ð InX

ò óË

SDRE-19

Note that, the LHS is symmetric because is symmetric, so is the RHS. Thus, or This is a Lyapunov function with , the result then follows.

ËT(X àXT) + (X àXT)Ë = 0

) X àXT = 0

InX

ò óT

Ð InX

ò óË = ËT In

X

ò óT

ÐT InX

ò ó

õ(Ë) ò Cà

ä

ÐH

SDRE-20

Question: Suppose that H has no imaginary eigenvalues. Under what conditions is stated in Remark 1 nonsingular?

Theorem 6: Suppose that H has no imaginary eigenvalues

and Then is stabilizable.

Proof: This is trivial, because by Theorem 5 This implies that (A,R) is stabilizable.

H 2 dom(Ric) , (A;R)

R õ 0 or R ô 0:

X1

") "

H 2 dom(Ric) ) õ(A àRX) ò Cà:

SDRE-21

To show that , by definition, we have to show

that is nonsingular, i.e., − First, we note that H has n eigenvalues in

"( "

H 2 dom(Ric)

CàX1 N(X1) = f0g:

) 9 X1

X2

ò ó& Ë with rank

X1

X2

ò ó= n & õ(Ë) ò Cà s:t:

HX1

X2

ò ó=

X1

X2

ò óË (3)

SDRE-22

− Next, we show that is a symmetric matrix:

Pre-multiply (3) by

where

is symmetric LHS of (4) is symmetric RHS of (4) is symmetric. Expand RHS and its transpose This is a Lyapunov function and This proves the claim.

XT

2X1

) X1

X2

ò óT

ÐHX1

X2

ò ó=

X1

X2

ò óT

ÐX1

X2

ò óË (4)

* ÐH ))

Ð =0 à InIn 0

ò ó

X1

X2

ò óT

Ð

) (XT

2X1 àXT

1X2)Ë = ËT(XT

1X2 àXT

2X1)

) (XT

1X2 àXT

2X1) = 0

õ(Ë) ò Cà

SDRE-23

− Third, we show that is -invariant: Suppose that and pre-multiply (3) by Pre-multiply (5) by and post-multiply by is symmetric Post-multiply (5) by This proves the claim.

N(X1) Ë

x 2 N(X1) [In; 0]

AX1 àRX2 = X1Ë (5) xTXT

2x

) xTXT

2RX2x = xTXT

2X1Ëx (6)

) xTXT

2X1Ëx = xTËTXT

2X1x = 0* XT

2X1

(6) ) xTXT

2RX2x = 0 ) RX2x = 0:

x ) X1Ëx = 0 ) Ëx 2 N(X1)

(7) ( Since R is semidefinite)

SDRE-24

− Finally, we show that is nonsingular: Suppose, on the contrary, that is singular This together with -invariant implies that has an eigenvalue and associated eigenvectors. Pre-multiply (3) by Post-multiply (8) by Stabilizability of (A,R) implies that loses rank, a contradiction. This completes the proof.

X1

[0; In]

i:e:; 9õ 2 Cà & x 2 N(X1)nf0g s:t: Ëx = õx

) àQX1 àATX2 = X2Ë

x ) (AT+ õI)X2x = 0

) N(X1)6=f0g

(8)

(7)(8) ) xTXT

2(A + õI

...R) = 0:

X1

N(X1) is Ë

ËjN(X1)

X2x = 0

) X1

X2

ò óx = 0 ) X1

X2

ò ó

ä

(* x 2 N(X1))

(* õ 2 Cà)

SDRE-25

Homework 1: Let and the matrix Show that (A,R) is stabilizable if and only if (A,B) is stabilizable.

Hint: By PBH test. Corollary 7: Suppose that H has no imaginary eigenvalues

and Then is stabilizable. is stabilizable. Question: When does H has no imaginary eigenvalues?

H 2 dom(Ric) , (A;R)

R = BRà10BT R0 > 0:

R = BRà10BT

, (A;B)

SDRE-26

Theorem 8: Suppose that H has the form (i) If (A,B) is stabilizable, then H has no imaginary eigenvalues if and only if (A,C) has no unobservable modes on the imaginary axis. (ii) (A,B) is stabilizable and (A,C) has no unobservable mode on the imaginary axis. (iii) (iv) Suppose that Then N(X)={0} (i.e., X is nonsingular or X > 0) if and only if (A,C) has no stable unobservable mode.

H 2 dom(Ric) ,

H = A àBBT

à CTC àAT

ò ó; B 2 Rnâp & C 2 Rqân

H 2 dom(Ric) ) X = Ric(H) õ 0:

(9)

H 2 dom(Ric) & X = Ric(H):

SDRE-27

Proof: To prove (i): Suppose (A,C) has unobservable mode on imaginary axis

H has imaginary eigenvalues.

Let be an eigen-pair of H

) 9 j! & x 6=0 s:t: A à j!InC

ò óx = 0:

"( "

") "

) (H à j!I2n) x0

ò ó= (A à j!In)x

à CTCx

ò ó= 0:

)

j!; xz

ò óò ó

) H xz

ò ó= j! x

z

ò óor

(A à j!In)x = BBTzà (A à j!In)Hz = CTCx

ú(10)

SDRE-28

(Since (A,B) is stabilizable)

is an unobservable mode of (A,C).

) zH(A à j!In)x = zHBBTz = jjBTzjj2à xH(A à j!In)Hz = xHCTCx = jjCxjj2

ú

(12)

) zH(A à j!In)x 2 R

) jjBTzjj2 = zH(A à j!In)x = xH(A à j!In)Hz = à jjCxjj2

) BTz = 0 & Cx = 0

(10)(11) ) (A à j!In)x = 0(A à j!In)Hz = 0

ú

(11)(12) ) zH(A à j!In...B) = 0 & A à j!In

C

ò óx = 0

) z = 0

) x 6=0

& j!

(* xz

ò óis an eigenvector)

SDRE-29

To prove (ii): Follows directly from Theorem 6, Corollary 7 and (i) of

this theorem. To prove (iii): Note that ATX +XA àXBBTX + CTC = 0

) (A àBBTX)TX +X(A àBBTX) +XBBTX + CTC = 0

* õ(A àBBTX) ò Cà

) X =R0

1 e(AàBBTX)Tt(XBBTX + CTC)e(AàBB

TX)tdt õ 0;

because XBBTX + CTC õ 0:

(13)

SDRE-30

To prove (iv): Suppose that (A,C) has a stable unobservable mode Pre-multiply the ARE by and post-multiply by x X is singular. )

) 2Re(õ)xHXx à xH(XBBTX)x = 0

õ

) 9x 6=0 s:t: Ax = õx & Cx = 0:

") "

xH

) xHXx = 0 (*Re(õ) < 0)

(*xHXx = jjX1=2xjj2 = 0)

SDRE-31

− To show that Suppose that Since X is a solution of the ARE

− To show that N(X)={0}: Suppose Pre-multiply (13) by and post- multiply by x

Post-multiply Eq. (13) again by x N(X) is A-invariant. Since N(X) and nontrivial and A-invariant (by (14)) is a stable unobservable mode, because

N(X)6=f0g

)

) õ

) 90 6=y 2 N(X) s:t: õy = Ay = (A àBBTX)y & Cy = 0

ä

) 9x 6=0 s:t: Xx = 0:

"( "

xT ) Cx = 0

) XAx = 0

õ 2 õ(A àBBTX) ú Cà:

) xT(ATX +XA àXBBTX + CTC)x = 0 ) Cx = 0

(14)

) x 2 N(C)

N(X) ò N(C) :

x 2 N(X):

SDRE-32

− Suppose that

Since X is a solution of the ARE Pre-multiply (13) by and post-multiply by x Post-multiply Eq. (13) again by x N(X) is A-invariant. Since N(X) and nontrivial and A-invariant (by (14)) is a stable unobservable mode, because

N(X)6=f0g

)

) õ

) 90 6=x 2 N(X) s:t: õx = Ax = (A àBBTX)x & Cx = 0

ä

) 9x 6=0 s:t: Xx = 0:

"( "

xT ) Cx = 0

) XAx = 0

õ(A àBBTX) ú Cà & Re(õ) < 0:

) xT(ATX +XA àXBBTX + CTC)x = 0 ) Cx = 0

(14) ) N(X) ò N(C)

SDRE-33

From Theorem 8, we have the next result:

Corollary 9: Suppose that (A,B) is stabilizable and (A,C) is detectable (resp., observable). Then the Riccati equation

has a unique positive semidefinite (resp., positive definite)

solution. Moreover, the solution is stabilizing.

ATX +XA àXBBTX + CTC = 0:

SDRE-34

Remark 2: Observability of (A,C) is not necessary for the existence of a positive definite stabilizing solution. For

instance, let

Then (A,B) is stabilizable, but (A,C) is NOT observable.

However, is the stabilizing solution.

A = 1 00 2

ò ó; B = 1

1

ò ó& C = (0; 0)

X = 18 à 24à 24 36

ò ó> 0

SDRE-35

MATLAB code for ARE: Step 1: Construct the associated Hamiltonian matrix H. Step 2: Do Schur factorization for H:

where

Step 3:

H = UTHU =Ë11 Ë12

0 Ë22

ò ó

X = U21Uà111

U =U11 U12

U21 U22

ò ó; õ(Ë11) ò Cà & õ(Ë22) ò C+

SDRE-36

The SDRE Design The SDRE strategy is nothing but pointwisely apply the

LQR scheme at every nonzero state.

A excellent survey paper: T. Cimen, ``State-dependent Riccati equation (SRDE)

control: a survey,’’ Proceedings of the 17th IFAC World Congress, pp. 3671-3775, 2008.

SDRE-37

The capabilities of the SDRE design:

− The capability to directly specify and affect performance through the selection of the state-dependent state and control weighting matrices Q(x) and R(x), respectively.

− The capability to impose the hard bounds on the control of the control rate.

− The capability to satisfy state constraints and combined state and control constraints.

− The capability to directly handle unstable, non-minimum phase systems.

− The capability to preserve beneficial nonlinearities.

SDRE-38

− The capability to utilize the extra design degrees of

freedom that are available in the non-uniqueness of the state-dependent coefficient matrix to enhance the performance of the systems.

(Please refer to J.R. Cloutier and D.T. Stansbery, ``The

capabilities and art of state-dependent Riccati equation-based design,’’ Proceedings of the American Control Conference, Anchorage, USA, pp. 86-90, 2002.)

SDRE-39 Consider a class of nonlinear control system and a quadratic performance index: and where denote the system state and the control input, respectively, Note that, the weighting matrices Q(x) and R(x) are in general state-dependent.

xç = f(x) +B(x)u

x 2 Rn & u 2 Rp

J =R0

1[xQ(x)x + uR(x)u]dt

f(x) 2 Rn; B(x) 2 Rnâp; f(0) = 0;

QT(x) = Q(x) õ 0 & RT(x) = R(x) > 0:

SDRE-40 Procedure of the SDRE scheme:

Factorize f(x) into the SDC matrix representation form as f(x)=A(x)x, where

Symbolically check stabilizability of (A(x),B(x)) and observability (resp., detectability) of (A(x),C(x)) to ensure the existence of a unique positive definite (resp., positive semi-definite) solution of the following SDRE:

has full rank and satisfies

Solve for P(x) to produce the SDRE controller u = àRà1(x)BT(x)P(x)x:

AT(x)P(x) + P(x)A(x)à P(x)B(x)Rà1(x)BT(x)P(x) +Q(x) = 0;

A(x) 2 Rnân:

C(x) 2 Rqân Q(x) = CT(x)C(x):