MODERN CONTROL THEORYLecturer:Qilian Bao鲍其莲
1
Chapter 8 Fundamentals of Optimal
Control
Objectives:
• Optimal control from Hamilton-Pontryagin Equation
• Optimal Control Law for Linear System with Quadratic
Performance Index
• Design procedure and examples
2Chapter 8
8.1 Optimal Control based on Hamilton-Pontryagin
Equation1.The optimal control problem for feedback control systems.
To minimize T
dtUXLJ0
),(
subject to ),()( UXfX t
state vectornt RX )(rt RU )(
),( UXf
),( UXL
the control vector
the function vector(Rn+r→Rn)
Rn+r→Rn, J is a function
2.Target:
Euler-Lagrange Equation Hamilton-Pontryagin Equation
Chapter 8
3.The design approach
Step 1. Rewrite the state equation
0))(),(()( ttt UXfX
Step 2. Set up the augment functional
T TT
T T
dtL
dtLJ
0
0
)]),([
)]),((),([
XΛfΛUX
XUXfΛUX
Define:XΛUXfΛUX TTLF ),(),(
Step 3. Define a scalar function —Hamiltonian
),(),(),,( UXfΛUXΛUXTLH
then
XΛΛUX THF ),,(
Chapter 8
Step 4. Applying Euler-Lagrange Equation to this problem
0
0
0
ΛΛ
UU
XX
F
dt
dF
F
dt
dF
F
dt
dF
Step 5. By substituting into the equation above
0),,,(
0),,(
0),,(
Λ
XUΛXU
UΛX
ΛX
UΛX
F
Hdt
dH Furthermore
f(X,U)XU
HX
H(t)Λ
0
Hamilton-Pontryagin equation------The necessary condition
for optimizing control systems.
Chapter 8
Example Consider the optimal control problem
Min T
dttutxJ0
22 ))()((2
1
s.t )()()( tutxtx
boundary condition : x(0)=1, x(T)=0
Our task: To find the optimal control law u*(t).
Step 1. Rewrite the equation as
0)()()( tutxtx
Step 2. Set up the augment functional
T
dtxuxuxJ0
22 )](2
1
2
1[
Step 3. Write Hamiltonian
uxuxH 22
2
1
2
1
Chapter 8
Step 4. From H-P equation, we have
)()()(
0)()(
)()()(
tutxtx
ttu
ttxt
Above equation set can be rewritten as
)(
)(
11
11
)(
)(
t
tx
t
tx
Step 5. Solve the equation
011
11
IA
The characteristic is 22,1
Chapter 8
Then the solution is
)ee)((ee)t(x tttt 2222 01212 22
1
tttt eeeet 2222 1212)0(22
1)(
The optimal control u*(t) is
tttt eeeettu 2222 1212)0(22
1)(*)(*
where
TT
TT
ee
ee
22
22 1212)0(
Chapter 8
8.2 Optimal Control Law for Linear System with
Quadratic Performance Index
1.Description of the problem
dt(t)RU(t))U(t)QX(t)(XJTT
02
1
s.t B(t)U(t)A(t)X(t)(t)X
where J is The cost functional (or say performance functional).
Q is a symmetric constant positive semi-definite weighting matrix
R is a symmetric constant positive definite weighting matrix.
Find optimal control law to minimize the cost functional J.
minU
Chapter 8
2 Design approach------Riccati Differential Equation
Construct an augment functional
(t)]dtX(t)ΛB(t)U(t))t)(t)(A(t)X(ΛRU)UQX[(XJTTTT
0 2
1
The Hamiltonian 1 1
2 2
T T T TH(X,Λ,U) X QX U RU Λ AX Λ BU
According to Hamilton-Pontryagin Equation, get the optimizing
condition
)()(Λ t(t)ΛAQX(t)H(X,Λ(X,X
tT
0)()()()(
)(
ttt
t
H TΛBRU
U
U,X,
)()()()()( ttttt UBXAX Since
so )()()()()()( 1 tttttt TΛBRBXAX
)(t(t)ΛBRU(t)T1
Chapter 8
Suppose )()()( ttt XPΛ ),()( nnt P
From H-P equation, we have
)()()()()()()()( tttttttt TXPAQXXPXP
)()()()( 1 tttt TXPBRU
)()()()()()()( ttttttt TXPBRBXAX
1
Through substituting
)())()(()()()()()()[()( tttttttttt TTXPAQXPBRBAPP
1
QtPtBRtBtPtPtAtAtPtP TT )()()()()()()()()( 1
The Riccati Differential Equation
Chapter 8
The optimal control law
)()(*)()(* tttt TXPBRU
1
where )(tP is the solution of Riccati Equation.
Let )()(*)( tttTΚPBR
1
The optimal control law can be written as
)()()( ttt XKU
)()()( ttt T PBRK1
Chapter 8
3. Riccati Matrix Equation (R.E. Kalman, 1960)
For LTI system i.e., matrices A and B are constant,
0)( tP
Riccati Differential Equation deduces to Riccati Matrix Equation
0 QPBPBRPAPA
1 TT
A very famous equation----Riccati Equation
The optimal control
)(
)(**
t
tT
KX
XPBRU1
4.The symmetric solution of Riccati Equation
By transposing Riccati equation:
0 QPBBRPAPPA
1 TTTTTT
Through comparing TPP
Chapter 8
The Flow-chart for Design
Set up the mathematical model for the linear or
linearized system under consideration
)()()( ttt BUAXX
Input the data of the system to form matrices A and B
Choose the quadratic performance index
and the weighting matrices Q and R.
0)(
2
1dtJ TT
RUUQXX
Chapter 8
Check if the system controllable:
Matrix D=[B AB A2B …An-1B] possesses full rank .
If yes
Solve the Riccati equation
P*0QPBPBRPAPA1 TT
Calculate the feedback gain matrix
K*=R-1BTP*
Design the optimal controller according to
U*=-K*X(t)
Test the stability & dynamic performance of the
controlled system
Chapter 8
Example1
uxx
dtuxJu
)(2
1min 2
0
2
How to solve the optimal control law?
Solution:A=(1) B=(1) Q=(1) R=(1)
Riccati equation:01 QPBPBRPAPA TT
can be written as:012 2 PP
rankD=rank[B|AB|…|An-1B]=rank[1]=1
(A,B) is controllable, so Riccati Equation has a positive-definite solution
In fact :
12
12
*1*
*
PBRK
P
T
The optimal control law:
xxKU )12(**
Chapter 8
Example2
Controller
dt dtu 2x
1x
The model
ux
xx
2
21
The matrix formBUAXX
uUBA
1
0
00
10
The optimal control problem
BUAXXts
dtUUQXXJ TT
u
.
)(2
1min
0
Where 1)0(
0
01
RQ
Chapter 8
Riccati Equation
01 QPBPBRPAPA TT
Judgment
201
10
rank
ABBrankrankD
So the Riccati equation has a positive definite solution
21
12
02001
00
01]10][1[
1
0
00
10
01
00
2212
1211
2
2212221211
2
12
2212
1211
2212
1211
2212
1211
2212
1211
pp
ppP
pppppp
pp
pp
pp
pp
pp
pp
pp
pp
The optimal control law
21
1* 2xxPXBRKXU T
Chapter 8
• Exercise: Find the optimal control u*(t) to minimize J:
19
u-
1
0
10
10xx
0
1)0(x
tuxxJ T d0
01
0
2
≥0
• Exercise: Find the optimal control u*(t) to minimize J:
• Solution:
• Let:
•
20
u-
1
0
10
10xx
0
1)0(x
tuxxJ T d0
01
0
2
≥0
2]rank[ ABBQc
2221
1211
PP
PPP
131
13
P
0 QPBBRPPAAP
1 TT
3
2
1
0
1
131
1301
2
1)0()0(
2
1xPx
T*J
02
3* J 1* J1
现代控制理论 21
3
2
1
0
1
131
1301
2
1)0()0(
2
1xPx
T*J
1当 时, ;当 时, 。02
3* J 1* J
Design process
Establish control
goals
Identify the
variable
to control and the
manipulating
variables
Construct the
system model
Determine the
performance
specifications
Select sensors
and actuators
Simulation study
and validation
Select a
controller and
adjust the
controller
parameters
Chapter 8
MATLA B
Homework:
• Learn and practice functions related to :
1. LQR controller
2. Solution to Ricatti equation
23Chapter 8
Summary
• Principle of optimal control
• Hamilton-Pontryagin Equation
• Optimal Control Law for Linear System with Quadratic
Performance Index
• Design procedure and examples
24Chapter 8
),( tux,fx
)()( 00
tttt
xx
0
L(x,u, )dft
t
J t t MIN
)(
)(
)(
)(2
1
t
t
t
t
n
λ
0
{L(x,u, ) λ ( )[f (x,u, ) x]}dft
T
t
J t t t t
),,()(),,(),,,( ttttH TuxfλuxLλux
0
[ (x,u,λ, ) λ ( )x]dft
T
t
J H t t t
0 0
(x,u,λ, )d λ ( )x df ft t
T
t t
H t t t t
00 0
(x,u,λ, )d λ ( )x λ ( )x df ff
t ttT T
tt t
J H t t t t t
0δ Juuu δ)()( tt
xxx δ)()( tt
)(δ)()( fff ttt xxx
0
δ δx δu λ δx d 0x u
f
T Tt
T
t
H HJ t
0
δ λ δx δu d 0x u
f
T Tt
t
H HJ t
xλ
H λx
f
x
L
xλ
H
0
u
H 0
λ
u
f
u
L
Chapter 8