Upload
vuthuan
View
223
Download
1
Embed Size (px)
Citation preview
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Part III – Optimal Control
Timm Faulwasser <[email protected]>
Laboratoire d’Automatique
Course Overview (Optimal Control Part)
Introduction
Direct Solution Approaches to Optimal Control
• Direct Collocation
• Direct Single Shooting
• Direct Multiple Shooting
Variational Approach to Optimal Control
• Euler-Lagrange Equations
• Sufficient Optimality Conditions
Pontryargin’s Maximum Principle • Input and State Constraints • Time-optimal Control • Singular Problems Indirect Solution Approaches to Optimal Control
Summary & Research Outlook
Overview of Numerical Methods for Optimal Control
Continuous Time Optimal Control
Hamilton-Jacobi-Bellman-Equation
• tabulation in state space • closed-loop opt. control
Indirect Methods
• Pontryargin’s Maximum
Principle (NCO of OCP)
Direct Solution Methods
• convert OCP into NLP
Direct Single Shooting
• only discretized controls in NLP
• sequential approach
Direct Collocation
• discretized states and controls
in NLP (collocation) • simultaneous approach
Direct Multiple Shooting
• discretized controls and node
start values in NLP • simultaneous approach
M. Diehl, H.G. Bock, H. Diedam, and P.B. Wieber. Fast Direct Multiple Shooting Algorithms for Optimal Robot Control. Lecture Notes in Control and Information Sciences, 340, 2006.
Optimal Control: A Primer on Direct Solutions Methods to Optimal Control Problems
Timm Faulwasser <[email protected]>
Laboratoire d’Automatique
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
General OCP
Optimal Control Problem
subject to
Numerical solution procedures for OCPs?
content of this lecture
Overview of Numerical Methods for Optimal Control
Continuous Time Optimal Control
Hamilton-Jacobi-Bellman-Equation
• tabulation in state space • closed-loop opt. control
Indirect Methods
• Pontryargin’s Maximum
Principle (NCO of OCP)
Direct Solution Methods
• convert OCP into NLP
Direct Single Shooting
• only discretized controls in NLP
• sequential approach
Direct Collocation
• discretized states and controls
in NLP (collocation) • simultaneous approach
Direct Multiple Shooting
• discretized controls and node
start values in NLP • simultaneous approach
Direct Solution Methods
Main Idea:
First discretize then optimize!
• studied extensively over the last decades (30 years)
• successfully applied to many applications
• take advantage of powerful & fast NLP solvers
• applicable to ODEs, DAEs, and PDEs
Direct Single Shooting
• only discretized controls in NLP
• sequential approach
Direct Collocation
• discretized states and controls
in NLP (collocation) • simultaneous approach
Direct Multiple Shooting
• discretized controls and node
start values in NLP • simultaneous approach
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
General OCP
Optimal Control Formulation
subject to subject to
Numerical solution procedures for OCPs?
Essential Tricks for Numerical Optimal Control
Free end time problems
use time transformation
end time is decision variable
Time-varying systems
extended state variable
Try to exploit problem structure!
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
Parametrization of Controls
Main Idea: describe control trajectory by finitely many parameters
Lagrange Interpolating Polynomials
• split time horizon
• in each interval approximate by polynomial function
• number of optimization variables on
• collocation points
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
• Control constraints via bounds on
• Continuity of control can be enforced via inequality constraints
Parametrization of Controls Direct Collocation (Direct Simultaneous Method)
Main Idea:
OCP finite dimensional NLP via discretization of control and state variables
State Collocation
• via Lagrange Interpolation Polynomials
• time derivative
• discretized ODE
Direct Collocation (Direct Simultaneous Method)
Continuous Time OCP
Fully-discretized Problem
subject to
subject to
Direct Collocation (Direct Simultaneous Method)
Exercise Discretize the following OCP into an NLP via collocation:
Use affine functions for state and inputs and 1 stage.
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Direct Collocation (Direct Simultaneous Method)
Advantages and Disadvantages
• very large scale NLPs
• number of stages and collocation points has to be chosen apriori
• infeasible path method: – ODEs satisfied at convergence only computationally efficient, unstable systems doable
• stage times can be optimized too
• path constraints via inequality constraints at collocation points
Pseudospectral Methods:
• variant of orthogonal collocation with 1 stage and high-order polynomials
• used in software tools DIDO, PROPT-TOMLAB
Direct Collocation (Direct Simultaneous Method)
Advantages and Disadvantages
• very large scale NLPs
• number of stages and collocation points has to be chosen apriori
• infeasible path method: – ODEs satisfied at convergence only computationally efficient, unstable systems doable
• stage times can be optimized too
• path constraints via inequality constraints at collocation points
Exercise: Sufficient conditions for the existence of optimal solution?
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
Direct Single Shooting (Direct Sequential Approach)
Idea:
discretize only the control variables
Original OCP
subject to
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Direct Single Shooting (Direct Sequential Approach)
OCP finite dimensional NLP with embedded ODEs
Idea:
discretize only the control variables
Original OCP
subject to
Partially Discretized Problem
subject to
Direct Single Shooting (Direct Sequential Approach)
Exercise Discretize the following OCP into an NLP via the direct sequential approach:
Use a piecewise-constant approximation for the controls.
Direct Single Shooting (Direct Sequential Approach)
Sketch of Numerical Solution Approach
Issues:
1. How to compute cost and constraint derivatives? ( Part III of this lecture)
2. How to handle constraints?
IN: OUT:
0.
i. Integrate ODEs
(system & sensitivities).
ii. Compute gradients.
iii. Solve NLP
iv. IF BREAK_COND = FALSE
GOTO i.
ELSE
How to handle path constraints?
reformulation as integral constraints
discretization as interior point constraints
Typically reformulation and discretization are combined/recommended.
Relaxation needed:
at a given set of points
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Direct Single Shooting (Direct Sequential Method)
Advantages and Disadvantages
• small scale NLPs (compared to collocation)
• the accuracy of state variable enforced by error-control of state-of-the art integrators
• feasible path method: – ODEs satisfied at each NLP iteration computationally more demanding, unstable systems can be
difficult
• path constraints via inequality constraints at collocation points
Exercise: Sufficient conditions for the existence of optimal solution?
Example: Dynamic Optimization
A Scalar Example • direct single shooting • piecewise constant controls • NLP solved with SQP methods • path constraint reformulated as
Example: Dynamic Optimization
A Scalar Example • direct single shooting • piecewise constant controls • NLP solved with SQP methods • path constraint reformulated as
Example: Dynamic Optimization
A Scalar Example • direct single shooting • piecewise constant controls • NLP solved with SQP methods • path constraint reformulated as
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Example: Dynamic Optimization
A Scalar Example • direct single shooting • piecewise constant controls • NLP solved with SQP methods • path constraint reformulated as
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
Direct Multiple Shooting
Original OCP
subject to
Idea: • discretize the control variables • split integration into multiple stages with discontinuous initial conditions
Direct Multiple Shooting
OCP finite dimensional NLP with (multiple) embedded ODEs
Original OCP
subject to
Partially Discretized Problem
subject to
Idea: • discretize the control variables • split integration into multiple stages with discontinuous initial conditions
H.G. Bock and K.J. Plitt. A multiple shooting algorithm for direct solution of optimal control problems. In Proceedings 9th IFAC World Congress Budapest,
pages 243–247. Pergamon Press, 1984.
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Direct Multiple Shooting
Sketch of Numerical Solution Approach
Same procedure as single shooting but with multiple independent ODEs.
IN: OUT:
0.
i. Integrate ODEs
(system & sensitivities).
ii. Compute gradients.
iii. Solve NLP
iv. IF BREAK_COND = FALSE
GOTO i.
ELSE
Direct Multiple Shooting
Advantages and Disadvantages
• between sequential and simultaneous methods: sequential infeasible path methods
• the accuracy of state variable enforced by error-control of state-of-the art integrators
• can deal with unstable systems
• structure of the NLP should be exploited (special SQP solvers)
• can be regarded as a generalization of single shooting
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
Computation of Function Derivatives
Main Possibilities
• numeric differentiation
• symbolic differentiation
• algorithmic differentiation (automatic differentiation)
Motivation
• derivatives of objectives and constraints (gradients)
• sensitivities of ODE or DAEs
• Jacobians to solve implicit equations (e.g. implicit ODE solvers, DAEs, …)
x … states u … inputs
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Numerical Differentiation
Finite Differences • scalar function • finite difference approximation
computationally cheap How to choose increment h? • typically accuracy
Complex Step Derivative Approximation • scalar analytic function • finite difference approximation using complex increment
computationally cheap + precise (increments can be extremely small) • easy to implement in MATLAB (“imaginary trick”) • use transp instead of ‘ for transposed matrices and vectors J.R.R.A. Martins, P.Sturdza and J.J. Alonso. The Complex-Step Derivative Approximation. ACM Transactions on Mathematical Software (TOMS),
pages 245—262, 2003.
Symbolic Differentiation
Calculate derivatives symbolically • via computer algebra tools (Maple, Mathematica, Matlab). usually high precision easy to use often complex formulas for derivatives computationally expensive analytic formula needed
Example • Speelpennings function • gradient
needs n(n-1) operations
Algorithmic Differentiation (Automatic Differentiation)
Example (Forward mode) •
Main Idea • many functions = compositions of elementary functions (sin, cos, …) • derivatives of elementary functions are known
• chain rule, product rule, , … differentiation of computer code; efficient and accurate computation of derivatives
Details: • A. Griewank and A. Walther. Evaluating Derivatives. SIAM 2008.
• AutoDiff homepage www.autodiff.org
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Derivatives of Functionals
Functional in Mayer form subject to ODE
• How to compute value of functional?
use state-of-the-art ODE solvers
• How to compute the functional derivatives?
possible solutions
• finite differences
• forward sensitivity analysis
• adjoint sensitivity analysis
• algorithmic differentiation, …. (not covered here)
Derivatives of Functionals
Functional in Mayer form subject to ODE
Finite Differences
1. Solve ODEs for nominal parameter value 2. Loop for solve ODEs for perturbed parameter
Trade-off: Choose small but not too small.
Derivatives of Functionals: Forward Sensitivities
Forward Sensitivities •formal derivation state sensitivities needed
• 1 set of ODEs per parameter ODEs to be solved • independent of number of functionals
Functional in Mayer form subject to ODE
Derivatives of Functionals: Adjoint Sensitivities
Adjoint (Reverse) Sensitivities
• adjoint variables needed (backward integration )
• interpolation of states between mesh points • ODEs
Optimal Control: A Primer on Direct Solutions Methods Dr. Timm Faulwasser, Laboratoire d‘Automatique, EPFL
22.04.2013
Derivatives of Functionals: Adjoint Sensitivities
Adjoint (Reverse) Sensitivities
• adjoint variables needed (backward integration )
• interpolation of states between mesh points • ODEs
Overview
Computation of Function Derivatives • finite differences
• symbolic derivatives
• algorithmic differentiation
Computation of Functional Derivatives • finite differences
• sensitivity equations
• adjoint sensitivities
Direct Solution Approaches to Optimal Control • direct collocation
• direct single shooting
• direct multiple shooting
Summary
Summary
Approaches to numerical optimal control:
Direct Methods
• direct collocation
• direct single shooting
• direct multiple shooting
Indirect Methods
• based on necessary conditions of optimality
later in the course
Hamilton-Jacobi-Bellman-Theory
• not covered
Software Tools for Numerical Optimal Control
Final question: How to solve your problems? • large variety of powerful software (free and commercial) to solve NLPs and optimal control
problems • useful web resource on optimization software http://plato.asu.edu/guide.html • web interface to NLP solvers http://www.neos-server.org/neos/ • understanding of underlying concepts is necessary to use these tools; especially if fast
solutions are required (predictive control) • indirect solutions methods are covered later in the course • further reading:
• script of this course (B. Chachuat) & script on numerical optimal control http://homes.esat.kuleuven.be/~mdiehl/TRENTO/numopticon.pdf
Toolboxes for Dynamic Optimization (Direct Solution Methods) • ACADO: direct multiple and single shooting, www.acadotoolkit.org/ • BOCOP : www.bocop.org • DIDO: pseudo-spectral methods http://www.elissarglobal.com/academic/products/ • HQP: direct multiple shooting http://hqp.sourceforge.net/ • MUSCOD-II : direct multiple shooting http://www.iwr.uni-heidelberg.de/~agbock/RESEARCH/muscod.php • PROPT TOMLAB: pseudo-spectral, commercial package • ….