Optimal control modelsof the goal-oriented human locomotion
(with Y. Chitour, F. Chittaro, and P. Mason)
Frederic Jean(ENSTA ParisTech, Paris)
Nonlinear Control and Singularities – October 24-28, 2010
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 1 / 30
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 2 / 30
Inverse optimal control
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 3 / 30
Inverse optimal control
Inverse optimal control
Analysis/modelling of human motor control→ looking for optimality principles
Subjects under study:
Arm pointing motions (with J-P. Gauthier, B. Berret)Saccadic motion of the eyesGoal oriented human locomotion
Mathematical formulation: inverse optimal control
Given X = φ(X,u) and a set Γ of trajectories, find a cost C(Xu) suchthat every γ ∈ Γ is solution of
inf{C(Xu) : Xu traj. s.t. Xu(0) = γ(0), Xu(T ) = γ(T )}.
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 4 / 30
Inverse optimal control
Difficulties:
Dynamical model not always known (← hierarchical optimal control)Limited precision of both dynamical models and costs
⇒ necessity of stability (genericity) of the criterion
Non-unicity of the cost that is solution of the problem.No general method
Validation method: a program in three steps
1 Modelling step: propose a class of optimal control problems
2 Analysis step: enhance qualitative properties of the optimal synthesis(e.g. inactivations)
→ reduce the class of problems
3 Comparison step: numerical methods→ choice of the best fitting L
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 5 / 30
Inverse optimal control
Difficulties:
Dynamical model not always known (← hierarchical optimal control)Limited precision of both dynamical models and costs
⇒ necessity of stability (genericity) of the criterion
Non-unicity of the cost that is solution of the problem.No general method
Validation method: a program in three steps
1 Modelling step: propose a class of optimal control problems
2 Analysis step: enhance qualitative properties of the optimal synthesis(e.g. inactivations)
→ reduce the class of problems
3 Comparison step: numerical methods→ choice of the best fitting L
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 5 / 30
Modelling goal oriented human locomotion
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 6 / 30
Modelling goal oriented human locomotion
Trajectories of the Human Locomotion
0 θ0(x , y , )
1 1 θ1(x , y , )
0
Goal-oriented human locomotion (Berthoz, Laumond et al.)
Initial point (x0, y0, θ0) → Final point (x1, y1, θ1)(x, y position, θ orientation of the body)
QUESTIONS :
Which trajectory is experimentally the most likely?
What criterion is used to choose this trajectory?
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 7 / 30
Modelling goal oriented human locomotion
Trajectories of the Human Locomotion
HYPOTHESIS:the chosen trajectory is solution of a minimization problem
min
∫L(x, y, θ, x, y, θ, . . . )dt
among all “possible” trajectories
joining the initial point to the final one.
→ TWO QUESTIONS:
What are the possible trajectories ? dynamical constraints?
How to choose the criterion ? (inverse optimal control problem)
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 8 / 30
Modelling goal oriented human locomotion
Dynamical Constraints
[Arechavaleta-Laumond-Hicheur-Berthoz, 2006] (=[ALHB])↓
1st experimental observation: if target far enough,
the velocity is perpendicular to the body
v
→ x sin θ − y cos θ = 0 nonholonomic constraint!
(Dubins)−→{x = v cos θy = v sin θ
v = tangential velocity.
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 9 / 30
Modelling goal oriented human locomotion
Dynamical Model
2nd experimental observation in [ALHB]↓
the velocity has a positive lower bound, v ≥ a > 0,
and is a function (almost constant) of the curvature
⇒ The trajectories may be parameterized by arc-length
(we are only interested by the geometric curves)
→ v ≡ 1.
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 10 / 30
Modelling goal oriented human locomotion
Dynamical Model
Previous observations: L = L(x, y, θ, θ, . . . )
trajectories obtained through a minimization procedure
⇒trajectories in a complete functional space, θ ∈W k,p,
and
L = L(x, y, θ, θ, . . . , θ(k)).
→ the trajectories are solutions of the optimal control problem
minCL(u) =
∫ T
0L(x, y, θ, θ, . . . , θ(k))dt
among all trajectories of
x = cos θy = sin θ
θ(k) = u
u ∈ Lp,
s.t. (x, y, θ)(0) = (x0, y0, θ0) and (x, y, θ)(T ) = (x1, y1, θ1).
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 11 / 30
Modelling goal oriented human locomotion
Admissible costs
ELEMENTARY REMARKS :
k ≥ 3 not reasonable ⇒ k = 1 or k = 2.
The whole problem is invariant by rototranslations→ L = L(θ, . . . , θ(k)) independent of (x, y, θ)
0 is the unique minimum of LNormalization: L(0) = 1 ⇒ cost of a straight line = its length
L is convex w.r.t. u
TECHNICAL ASSUMPTIONS :
L is smooth (at least C2)
L is strictly convex w.r.t. u and∂2L
∂u2> 0
L(θ, . . . , θ(k−1), u) ≥ C|u|p for |u| > R.
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 12 / 30
Modelling goal oriented human locomotion
To summarize
Inverse Optimal Control Problem
Given recorded experimental data (e.g. [ALHB])), infer a cost function Lsuch that the recorded trajectories are optimal solutions of
Pk(L)
minCL(u) =
∫ T
0L(θ, . . . , θ(k))dt (k = 1 or 2)
subject to
x = cos θy = sin θ
θ(k) = uwith (x, y, θ)(0) = (x0, y0, θ0) and (x, y, θ)(T ) = (x1, y1, θ1).
REMARKS :
The time T is not fixed
The initial point can be chosen as X0 = (0, 0, π/2)
The target X1 = (x1, y1, θ1) is far from X0: |(x1, y1)| “large”
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 13 / 30
Analysis of Pk(L)
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 14 / 30
Analysis of Pk(L) General results
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 15 / 30
Analysis of Pk(L) General results
Analysis of Pk(L) – General Results
Proposition
For every target X1, there exists an optimal trajectory of Pk(L).
Every optimal trajectory satisfies the Pontryagin Maximum Principle.
REMARKS :
The control system is controllable
The proof of existence uses standard arguments (cf. Lee & Markus)
The optimal control does not belong a priori to L∞([0, T ]). Howeverit is possible to prove that, for (x1, y1) far away from 0, the optimalcontrol is uniformly bounded.→ not necessary to put an a priori bound on the control
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 16 / 30
Analysis of Pk(L) General results
CASE k = 2:
H = p1 cos θ + p2 sin θ + p3θ + p4θ − νL(θ, θ) ≡ 0
No strictly abnormal extremals → optimal traj. are C∞ (ν = 1)
Adjoint Equation: (p1, p2) are constant and
using ∂H∂u = 0, the adjoint equation writes as an ODE:
θ(4) = FL(θ, θ, θ, θ(3); (p1, p2)
),
with initial data: (θ, θ(3))(0) = (π2 , 0) [transversality condition]
CASE k = 1:
H = p1 cos θ + p2 sin θ + p3θ − νL(θ) ≡ 0
No strictly abnormal extremals → optimal traj. are C∞ (ν = 1)
Adjoint Equation: (p1, p2) are constant and
θ = GL(θ, θ; (p1, p2)
), θ(0) =
π
2
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 17 / 30
Analysis of Pk(L) Asymptotic analysis
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 18 / 30
Analysis of Pk(L) Asymptotic analysis
Analysis of P2(L)
We present only the analysis for k = 2; same thing for k = 1.
Simplifying assumption: L(θ, θ) = 1 + ψ(θ) + φ(θ)
REMIND:
Adjoint equation ⇒ fourth-order ODE on θ parameterized by (p1, p2):
θ(4) = FL(θ, θ, θ, θ(3); (p1, p2)
),
with initial data: (θ, θ(3))(0) = (π2 , 0)
H = p1 cos θ + p2 sin θ + p3θ + p4θ − L(θ, θ) ≡ 0
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 19 / 30
Analysis of Pk(L) Asymptotic analysis
Notations: (x1, y1) = ρ(cosα, sinα),
Θ = (θ, θ, θ, θ(3)).
( )1 y1,y
x
x
α
Proposition
Given ν > 0, |Θ(t)− (α, 0, 0, 0)| < ν for t ∈ [τν , T − τν ].
Proposition
As |(x1, y1)| → ∞, (p1, p2) ∼(x1, y1)
|(x1, y1)|= (cosα, sinα).
(α, 0, 0, 0) equilibrium of the “limit” equation θ(4) = FL(Θ; (x1,y1)|(x1,y1)|
)Consequence: an optimal trajectory has a stable behaviour near the
equilibrium Yeq = (α, 0, 0, 0).
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 20 / 30
Analysis of Pk(L) Asymptotic analysis
Asymptotic Trajectories
The equilibrium Yeq = (α, 0, 0, 0) is not stable:its stable manifold Ws is 2-dimensional.
Z(0)
eq
Yeq
u
s
u
s
Y → the limit optimal trajectorymust be “contained” in Ws.
→ Close to Yeq, the trajectoryis tangent to the stable spaceof the linearized ODE
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 21 / 30
Analysis of Pk(L) Asymptotic analysis
Asymptotic Trajectories
Ws contains a 1-parameter family of trajectories: we compute themnumerically and select the one starting with θ(0) = π/2 and θ(3)(0) = 0.
-8 -6 -4 -2 2 4 6 8
-8
-6
-4
-2
2
4
6
8
We obtain also the limit value of p(0)→ initialization of a shooting algorithm.
We can recover dL from the phase portraitof Θ
(Figure: L = 1 + θ2 + θ2)
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 22 / 30
Analysis of Pk(L) Asymptotic analysis
5 10 15 20
5
10
15
20
25
30
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 23 / 30
Analysis of Pk(L) Stability results
Outline
1 Inverse optimal control
2 Modelling goal oriented human locomotion
3 Analysis of Pk(L)General resultsAsymptotic analysisStability results
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 24 / 30
Analysis of Pk(L) Stability results
Stability results
Let L0 be a cost admissible for k = 1 or k = 2.
Let Lε be a family of costs admissible for k = 2 s.t. Lε → L0 in thefollowing sense:
|Lε(θ, θ)− L0(θ, θ)| or |Lε(θ, θ)− L0(θ)| ≤ C(ε)|θ|p
Proposition
The optimal trajectories (xε, yε, θε, θε) of P2(Lε) converge uniformlyto the optimal trajectories of Pk(L0), i.e.,
dunif((xε, yε, θε, θε), T0
)→ 0,
T0 = {(x, y, θ, θ) s.t. (x, y, θ, θ) or (x, y, θ) is optimal for Pk(L0)}.
With additional technical hypothesis, the adjoint vectors (and so θand θ(3)) also converge uniformly.
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 25 / 30
Analysis of Pk(L) Stability results
Stability results
CONSEQUENCES
The optimal synthesis of a problem Pk(L0) is stable underperturbations of the cost.
Our modelling is ”reasonable” (physiological costs)
A solution of the inverse problem = a cost and his perturbations→ we will look for the simplest cost in this class
Question: k = 1 for the simplest one?
Adjoint equation of a perturbation = perturbation of the adjointequation
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 26 / 30
Analysis of Pk(L) Stability results
Analysis of the case k = 1
H = p1 cos θ + p2 sin θ + θL′(θ)− L(θ) ≡ 0
Remark: If θ(t0) = 0, then p1 cos θ(t0) + p2 sin θ(t0) = 1,
→ depends on one parameter
Proposition
Let T be the set of trajectories s.t. θ = 0 at some time.To any trajectory in T , with θ(t0) = 0, we apply the transformation:
t = t− t0θ(t) = θ(t)− θ(t0)(x(t), y(t)) = Rot(−θ(t0))
[(x(t), y(t))− (x(t0), y(t0))
]Then, for every fixed t, the set of (x(t), y(t), θ(t)), for all trajectories in T ,is a curve in R3.
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 27 / 30
Analysis of Pk(L) Stability results
Numerical test
Numerical test: apply the transformation to the recorded curves.Does it give a curve?
Transformation applied at three different times t1, t2, t3 to ∼700recorded trajectories
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 28 / 30
Analysis of Pk(L) Stability results
Validity of the test (work in progress)
Numerically: same test applied to the solutions of some P2(L)
Transversality argument: for k = 2, a generic cost L should not givesuch a curve through the transformation
Analysis of the optimal synthesis using the asymptotic trajectories
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 29 / 30
Analysis of Pk(L) Stability results
Conclusion
Models with k = 1 should be sufficient.
A deep explanation:
recorded trajectories theoretical solutions
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 30 / 30
Analysis of Pk(L) Stability results
Conclusion
Models with k = 1 should be sufficient.
A deep explanation:
recorded trajectories theoretical solutions
F. Jean (ENSTA ParisTech) Optimal control and locomotion Porquerolles, October 2010 30 / 30