Math 692A:Geometric Numerical Integration

Melvin Leok

Mathematics, Purdue University.

mleok@math.purdue.eduhttp://www.math.purdue.edu/˜mleok/

Mathematics, Purdue UniversityNSF DMS-0504747, 0714223 and DMS-0726263

2

The Mathematics of Falling Cats

3

Connections, Curvature, and Geometric Phase

Connections

•Connections provide a means of comparing elements of a fiberbased at different points on the manifold.

Holonomy and Curvature

•Geometric Phase is an exampleof holonomy.

•Curvature can be thought of asinfinitesimal holonomy.

area = A

finish

start

4

Example of Holonomy

Vatican Museum double helical staircase designed by Giuseppe Momo in 1932.

5

Geometric Control of Spacecraft

Geometric Phase based controllers

• Shape controlled using internal momentum wheels and gyroscopes.

• Changes in shape result in corresponding changes in orientation.

• More precise than chemical propulsion based orientation control.

rigid carrier

spinning rotors

6

Geometric Control of Spacecraft

7

Shape Dynamics of Formations of Satellites

NASA Terrestrial Planet Finder (TPF)

• The NASA Terrestrial Planet Finder (TPF) and the ESA Darwinmissions are examples of a potential application of the geometricformation control of satellite clusters.

Small satellites in arranged in a formation to yield a large

effective aperture telescope. Courtesy NASA/JPL-Caltech.

Artist’s conception of the ESA Darwin

flotilla. Courtesy ESA.

• It is quite natural to think of controlling the shape of the cluster,and its orientation (group) separately.

8

Numerically implementing geometric control algorithms

Demand for long-time stability in control algorithms

• Trend towards autonomous space and underwater vehicle missionswith long deployment times and low energy propulsionsystems.

• Small inaccuracies in the control algorithms accumulated over longtimes will significantly diminish the operational lifespan of suchmissions.

• Need for geometric integrators to efficiently achieve good qualita-tive results for simulations over long times.

9

Geometry and Numerical Methods

Dynamical equations preserve structure

• Many continuous systems of interest have properties that are con-served by the flow:

Energy

Symmetries, Reversibility, Monotonicity

Momentum - Angular, Linear, Kelvin Circulation Theorem.

Symplectic Form

Integrability

• At other times, the equations themselves are defined on a mani-fold, such as a Lie group, or more general configuration manifoldof a mechanical system, and the discrete trajectory we computeshould remain on this manifold, since the equations may not bewell-defined off the surface.

10

Motivation: Geometric Integration

Main Goal of Geometric Integration:

Structure preservation in order to reproduce long time behavior.

Role of Discrete Structure-Preservation:Discrete conservation laws impart long time numerical stabilityto computations, since the structure-preserving algorithm exactlyconserves a discrete quantity that is always close to the continuousquantity we are interested in.

11

Geometric Integration: Energy Stability

Energy stability for symplectic integrators

Continuous energyIsosurface

Discrete energyIsosurface

Control on global error

12

Geometric Integration: Energy Stability

Energy behavior for conservative and dissipative systems

0 200 400 600 800 1000 1200 1400 16000

0.05

0.1

0.15

0.2

0.25

0.3

Time

En

erg

y

Variational

Runge-Kutta

Benchmark

0 100 200 300 400 500 600 700 800 900 10000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Time

En

erg

y

Midpoint Newmark

Explicit NewmarkVariational

non-variational Runge-Kutta

Benchmark

(a) Conservative mechanics (b) Dissipative mechanics

13

Geometric Integration: Energy Stability

Solar System Simulation

• Forward Euler

qk+1 = qk + hq(qk,pk),

pk+1 = pk + hp(qk,pk).

• Inverse Euler

qk+1 = qk + hq(qk+1,pk+1),

pk+1 = pk + hp(qk+1,pk+1).

• Symplectic Euler

qk+1 = qk + hq(qk,pk+1),

pk+1 = pk + hp(qk,pk+1).

14

Geometric Integration: Energy Stability

Forward Euler

−30−20

−100

1020

30

−30

−20

−10

0

10

20

30−30

−20

−10

0

10

20

30

2144

1980 2000 2020 2040 2060 2080 2100 2120 2140 21600

0.5

1

Energy error

15

Geometric Integration: Energy Stability

Inverse Euler

−30−20

−100

1020

30

−30

−20

−10

0

10

20

30−30

−20

−10

0

10

20

30

2077

1990 2000 2010 2020 2030 2040 2050 2060 2070 2080−50

0

50

Energy error

16

Geometric Integration: Energy Stability

Symplectic Euler

−30−20

−100

1020

30

−30

−20

−10

0

10

20

30−30

−20

−10

0

10

20

30

2268

1950 2000 2050 2100 2150 2200 2250 2300−2

0

2x 10

−3

Energy error

17

Introduction to Computational Geometric Mechanics

Geometric Mechanics

• Differential geometric and symmetry techniques applied to thestudy of Lagrangian and Hamiltonian mechanics.

Computational Geometric Mechanics

• Constructing computational algorithms using ideas from geometricmechanics.

• Variational integrators based on discretizing Hamilton’s principle,automatically symplectic and momentum preserving.

18

Introduction to Computational Geometric Mechanics

Why adopt this approach?

• Provides a systematic framework to construct structure-preservingnumerical schemes that are consistent with the underlying geome-try of the problem.

• Structure-preservation can be important from the point of view ofstability, accuracy, and efficiency.

• Provides insight into the discrete analogues of differential geometricstructures that are preserved by numerical schemes.

19

Discrete Mechanics

Traditional Approach

• Discretization is performed at the level of the equation, for exam-ple, finite-differences and finite-elements applied to the differentialequations of motion.

Alternative Approach

• Discretize underlying variational principles.

20

Continuous Mechanics

Lagrangian Formulation of Mechanics

• Given a Lagrangian L : TQ → R, the solution curve satisfiesHamilton’s variational principle,

δ

∫ t1

t0

L (q (t) , q (t)) dt = 0.

• This is equivalent to the Euler-Lagrange equation,

d

dt

∂L

∂q− ∂L

∂q= 0.

• When the Lagrangian has the form L = 12q

TMq − V (q), thisreduces to Newton’s Second Law,

d

dt(Mq) = −∂V

∂q.

21

Discrete Mechanics

Discretization of Mechanics

• Approximate TQ by Q×Q. Diagonal group action of G on Q×Q

• The discrete Lagrangian Ld : Q × Q → R is a generatingfunction of the discrete flow. A particularly simple choice of discreteLagrangian is given by,

Ld (qk, qk+1) = h · L(

qk+1 + qk

2,qk+1 − qk

h

)• Consider the discrete action sum S : QN+1 → R defined by

S =

N−1∑k=0

Ld (qk, qk+1) .

which is a discrete analogue of the action integral.

22

Discrete Mechanics

Discrete Variational Principle

• Hamilton’s variational principle states that the solution curve hasan action integral that is stationary.

• Analogously, the discrete variational principle states that,

δ

N−1∑k=0

Ld (qk, qk+1) = 0.

23

Discrete Mechanics

Discrete Variational Principle

q(a)

q(b)

δq(t)

Q

q(t)

q0

qN

δqi

Q

qi

varied curve

varied point

24

Discrete Mechanics

Discrete Euler-Lagrange equation

• To extremize S over q0, · · · , qN , we set

∂S∂qk

= 0, k = 0, . . . , N.

This implies,

D1Ld (qk, qk+1) δqk+D2Ld (qk−1, qk) δqk = 0, k = 1, . . . , N−1.

Since δq0 = δqN = 0 and the rest of the variations are arbitrary,we obtain the discrete Euler-Lagrange equation,

D1Ld Φ + D2Ld = 0.

25

Correspondance between discrete and continuous mechanics

• Consider the exact discrete Lagrangian given by,

Ld (qk, qk+1) =

∫ tk+1

tk

L (q (t) , q (t)) dt

where q : [tk, tk+1] → Q is a solution of the Euler-Lagrange equa-tions for L which satisfies the boundary conditions q (tk) = qk andq (tk+1) = qk+1.

• This choice is motived by the Jacobi solution of Hamilton-Jacobi Theory, where the action integral is a generating functionof the Hamiltonian flow.

26

Correspondance between discrete and continuous mechanics.

• Can show that with the exact discrete Lagrangian, the discretecurve qkn

k=0 satisfying the Discrete Euler-Lagange equations dis-cretely sample the continuous solution curve q(t) of the Euler-Lagrange equations for the corresponding Lagrangian.

qk = q (tk)

• In practice, the exact discrete Lagrangian, like Jacobi’s solution,cannot be computed exactly. As such, numerical quadrature meth-ods are used to approximate the exact discrete Lagrangian.

• Can be shown that a discrete Lagrangian given by an n-th orderaccurate quadrature method yields an n-th order accurate sym-plectic integrator. This recovers a class of previously discoveredSymplectic Partitioned Runger-Kutta methods.

27

Canonical Symplectic Structure

Hamilton’s Canonical Equations

qi =∂H

∂pi

pi = −∂H

∂qi

• If we adopt the coordinate system, z = (q, p), the CanonicalSymplectic Form, Ω = dpi∧ dqi has the matrix representationgiven by:

J =

(0 1

−1 0

)• Hamilton’s equations, z = XH = J∇H can be written as:

Ω (XH(z), v) = dH.z

28

Discrete Symplectic Structure

Fiber Derivative

FLd : Q×Q → T ∗Q

(q0, q1) 7→ (q0, D1Ld (q0, q1))

Discrete Symplectic Form

ωL = (FLd)∗ (ΩL)

= d((FLd)

∗(−pidqi

))= d

(−∂Ld

∂qik

(qk, qk+1)

)dqi

k

=∂2Ld

∂qik∂q

jk+1

(qk, qk+1) dqik ∧ dq

jk+1

29

Momentum Maps

Continuous Momentum Map

• A momentum map J : T ∗Q → g∗ is a generalization of the familarconjugate momentum in Hamiltonian mechanics.

• Given a mechanical system that is invariant under the action ofthe Lie group G on the configuration manifold Q, we define amomentum map by,⟨

J(αq), ξ⟩≡⟨αq, ξQ(q)

⟩ Discrete Momentum Map

•Discrete Momentum Map Jd : Q×Q → g∗,

〈Jd (qk, qk+1) , ξ〉 ≡⟨D1Ld (qk, qk+1) , ξQ (qk)

⟩

30

Momentum Maps

Example: Angular MomentumLet SO(3) act on R3 by matrix multiplication.

A · q = Aq

The infinitesimal generator is given by ωR3(q) = ωq = ω × q, where

ω ∈ R3. The momentum map J : T ∗R3 → so(3)∗ ∼= R3 is given by,

〈J(q,p), ω〉 = p · ωq = ω · (q× p),

that is,J(q,p) = q× p,

which is the familiar expression for angular momentum.

31

Discrete Mechanics

What does this way of thinking buy you?

• Systematic method of constructing symplectic-momentum integra-tors of arbitrarily high-order.

• In contrast to non-constructive condition on the coefficients of thepartitioned Runge–Kutta scheme:

biaij + bjaji = bibj, i, j = 1, . . . , s,

bi = bi, i = 1, . . . s.

• Naturally leads to generalized schemes such as Lie group, multi-scale, and pseudospectral variational integrators.

32

Comparing representations of the rotation group

Euler Angles

• Local coordinate chart, exhibits singularities.

• Requires change of charts to simulate large attitude maneuvers.

Unit Quaternions

• Reprojection used to stay on unit 3-sphere.

• The 3-sphere is a double-cover of SO(3) which causes topologicalproblems for optimization.

Rotation Matrices

• 9 dimensional space (3×3 matrices) with a 6 dimensional constraint(orthogonality), but the exponential map saves the day.

33

Variational Lie Group Techniques

Basic Idea

• To stay on the Lie group, we parametrize the curve by the initialpoint g0, and elements of the Lie algebra ξi, such that,

gd(t) = exp(∑

ξslκ,s(t))

g0

• This involves standard interpolatory methods on the Lie algebrathat are lifted to the group using the exponential map.

• Automatically stays on SO(n) without the need for reprojection,constraints, or local coordinates.

• Order of accuracy of method is independent of the retraction, asthe variational principle is at the level of the Lie group, and theretraction is simply used to locally parametrize the group.

34

Model Problem

3D Pendulum• A rigid body, with a pivot point

and a center of mass that arenot collocated, under gravita-tional forces.

• Three degrees of freedom.

• Exhibits surprisingly rich andcomplex dynamics.

35

Physical Realization of a 3D Pendulum

Triaxial Attitude Control Testbed

Attitude Dynamics and Control Laboratory, University of Michigan, Ann Arbor.

36

Example of a Lie Group Variational Integrator

3D Pendulum

• Lagrangian

L(R,ω) =1

2

∫Body

‖(ρ)ω‖2dm− V (R),

where · : R3→R3×3 is a skew mapping such that xy = x× y.

• Equations of motion

Jω + ω × Jω = M,

where M = ∂V∂R

TR −RT ∂V

∂R.

R = Rω.

37

Example of a Lie Group Variational Integrator

3D Pendulum

•Discrete Lagrangian

Ld(Rk, Fk) =1

htr [(I3×3 − Fk)Jd]−

h

2V (Rk)− h

2V (Rk+1).

•Discrete Equations of Motion

Jωk+1 = FTk Jωk + hMk+1,

S(Jωk) =1

h

(FkJd − JdF

Tk

),

Rk+1 = RkFk.

38

Example of a Lie Group Variational Integrator

Automatically staying on the rotation group

• The magic begins with the ansatz,

Fk = exp(fk),

and the Rodrigues’ formula, which converts the equation,

Jωk =1

h

(FkJd − JdF

Tk

),

into

hJωk =sin ‖fk‖‖fk‖

Jfk +1− cos ‖fk‖

‖fk‖2fk × Jfk.

• Since Fk is the exponential of a skew matrix, it is a rotation matrix,and by matrix multiplication Rk+1 = RkFk is a rotation matrix.

39

Numerical Simulation

Chaotic Motion of a 3D Pendulum

40

Numerical Simulations

Flyby of two dumbbells

41

Numerical Simulations

Effect of representations (Runge-Kutta)

0 5 10 150

1

2

3

4

T

T ro t

T tran

0 5 10 15−4

−2

0

2

4

t

E

T U E

Runge-Kutta with quaternions

0 5 10 150

1

2

3

4

T

T ro t

T tran

0 5 10 15−4

−2

0

2

4

t

E

T U E

Runge-Kutta with Euler angles

42

Numerical Simulations

Effect of representations (Runge-Kutta)

0 5 10 150

1

2

3

4

T

T ro t

T tran

0 5 10 15−4

−2

0

2

4

t

E

T U E

Runge-Kutta with SO(3)

0 5 10 15−1

0

1

2

3x 10

−3

ΔE

0 5 10 150

2

4

6x 10

−3

SO(3

) er

ror

t

‖I−RT R‖ ‖I−RT2 R2‖

Runge-Kutta with SO(3) (Integration error)

43

Numerical Simulations

Lie group variational integrator on SO(3)

Trajectory in inertial frame

0 5 10 150

1

2

3

4

T

0 5 10 15−4

−2

0

2

4

t

E

T ro t

T tran

T U E

Transfer of energy

44

Numerical Simulations

Conservation Properties: Lie Group Integrator

0 20 40 60 80 100−2

0

2

4

6

8

10

12

14x 10

−4

ΔE

t

deviation in total energy

0 20 40 60 80 1000

0.05

0.1

0.15

0.2

norm

(I−

RT R

)0 20 40 60 80 100

0

1

2

3x 10

−5

norm

(I−

R2T R

2)t

error in the rotation matrix

45

Numerical Simulations

Conservation Properties: Lie Group Integrator

0 20 40 60 80 100−5

0

5x 10

−14

0 20 40 60 80 100−5

0

5x 10

−14

ΔγT

0 20 40 60 80 100−5

0

5x 10

−14

t

deviation in total linear momentum

0 20 40 60 80 100−2

−1

0

1x 10

−6

0 20 40 60 80 100−5

0

5x 10

−6

ΔπT

0 20 40 60 80 100−5

0

5x 10

−6

t

deviation in total angular momentum

46

Numerical Simulations

Comparison with other methods

• Our Lie group variational integrator (LGVI) is a Lie Stormer–Verlet method, so it is a second-order symplectic Lie group method.

• We compare it to other second-order accurate methods:

Explicit Midpoint Rule (RK):Preserves neither symplectic nor Lie group properties.

Implicit Midpoint Rule (SRK):Symplectic but does not preserve Lie group properties.

Crouch-Grossman (LGM):Lie group method but not symplectic.

47

Numerical Simulations

Comparison with other methods

0 10 20 30

−0.1593

−0.159

time

E

RKSRKLGMLGVI

Computed total energy for 30 seconds

10−4

10−3

10−2

10−15

10−10

10−5

100

Step size

mea

n |I−

RTR

|

Mean orthogonality error ‖I −RTR‖ vs. step size

10−4

10−3

10−2

10−8

10−6

10−4

10−2

Step size

mea

n |Δ

E|

RKSRKLGMLGVI

Mean total energy error |E − E0| vs. step size

10−4

10−3

10−2

102

103

104

105

Step sizeC

PU ti

me

(sec

)

CPU time vs. step size

48

Applications to Asteroid Simulations

Computational Considerations

• Asteroids are approximated by rubble piles (hard sphere models)or simplicial complexes.

• Force evaluations using fast multipole methods or polyhedral po-tential techniques.

• Computational cost is dominated by cost of force evaluation.

• Lie Stormer–Verlet variational integrators use only one force eval-uation per timestep, and are only implicit in the attitude, andconverge in 2-3 Newton steps.

• Used by Scheeres, et al. to accurately simulate binary near-EarthAsteroid 66391 (1999 KW4).

49

Applications to Asteroid Simulations

50

Numerically implementing geometric control algorithms

Traditional approach

• Local analysis of the connection near the desired shape position.

• Gives a closed form expression for the geometric phase associatedwith infinitesimally small loops in shape space.

• Resulting shape trajectories are often suboptimal and slow.

Alternative approach

• Homotopy-based optimal control algorithm using geometrically ex-act numerical schemes.

• Allows for large-amplitude trajectories that are global in nature,and more efficient than infinitesimal loops.

51

Discrete Optimal Control

Essential Ideas

• Use the discrete Lagrange–d’Alembert principle,

δ∑

Ld (qk, qk+1) +∑

Fd (qk, qk+1) · (δqk, δqk+1) = 0,

to derive the discrete forced Euler–Lagrange equations, andimpose these as constraints at every time-step.

• This yields greater fidelity to the equations of motion that im-posing the dynamical constraints using the method of collocation.

• The resulting numerical solutions are group equivariant, whichimplies that the numerical solutions are independent of the choiceof coordinate frame.

52

Shooting based optimization using variational integrators

General approach

• Relax terminal boundary conditions, and guess initial control torque.

• Evolve attitude and control torques using equations of motion andoptimality conditions.

• Compute sensitivity of terminal boundary conditions on initial con-trol torques, and iterate to convergence.

Robust computation of sensitivities

• Structure preserving properties of the variational integrator allowthe robust computation of sensitivities.

• Problems considered do not appear to require multiple shooting toachieve convergence.

53

Under-actuated control of a 3D pendulum

Attitude Maneuver

0 0.2 0.4 0.6 0.8 1−10

0

10

20

u (N

m)

0 0.2 0.4 0.6 0.8 1−10

−5

0

5

10

t (sec)

Control input u

0 0.2 0.4 0.6 0.8 1−20

0

20

0 0.2 0.4 0.6 0.8 1−20

−10

0

Ω (

rad/

sec)

0 0.2 0.4 0.6 0.8 1−1

0

1

t (sec)

Angular velocity Ω

10 20 30 40 50 6010

−15

10−10

10−5

100

105

IterationE

rror

Convergence rate

54

Under-actuated control of a 3D pendulum

55

Under-actuated control with symmetry of a 3D pendulum

−2

−1.5

−1

−0.5

−2

−1.5

−1

−0.5

56

Under-actuated control with symmetry of a 3D pendulum

−12

−10

−8

−6

−4

−2

−12

−10

−8

−6

−4

−2

57

Variational Integrators on S2

Constrained variations

• The condition that q ∈ S2 = q ∈ R3|q · q = 1 yields a con-strained variation of q in the variational principle.

δq = ξ × q,

where ξ ∈ R3 is constrained to be orthogonal to q, i.e., ξ · q = 0.

Lagrangian

• The configuration space is a cartesian product of two-spheres, Q =(S2)n, and the Lagrangian has the form,

L(q1, . . . , qn, q1, . . . , qn) =1

2

n∑i,j=1

qTj Mij qi − V (q1, . . . , qn),

58

Variational Integrators on S2

Spherical Pendulum

rk+1 =(hωk +

h2g

2lrk × e3

)× rk + rk

√1−

∥∥∥hωk +h2g

2lrk × r3

∥∥∥2

ωk+1 =ωk +hg

2lrk × e3 +

hg

2lrk+1 × e3

Double Spherical Pendulum[2

1+f1·f1I3×3 − 2α

1+f2·f2r1r2

− 2β1+f1·f1

r2r12

1+f2·f2I3×3

] [f1

f2

]=

[hω1k

− hα(r1k× (r2k

× ω2k)) + h2g

2l1(r1k

× e3) + 2αf2·f21+f2·f2

r1r2

hω2k− hβ(r2k

× (r1k× ω1k

)) + h2g2l2

(r2k× e3) + 2βf1·f1

1+f1·f1r2r1

]r1k+1 = (I3×3 + f1)(I3×3 − f1)

−1r1k

r2k+1 = (I3×3 + f2)(I3×3 − f2)−1r2k

where α = m2m1+m2

l2l1

and β = l1l2.

• Implicit system of equations can be solved using fixed-point itera-tion, and requires about 5-6 iterations to reach machine precision.

59

Variational Integrators for S2

Double Spherical Pendulum Simulation

60

Variational Integrators for S2

Elastic Rod

61

Variational Integrators for S2

Coupled Spherical Pendula

62

Variational Integrators for S2

3-body problem on the sphere

63

Variational Integrators for S2

Magnetic Arrays

64

Towards Larger-Scale Scientific Computing Problems

Hierarchical Multi-Resolution Lie Group VIs

• General framework motivated by multiscale multiphysics methods.

• Based on the use of empirical shape functions obtained fromlocalized simulations.

• Decompose a strand of DNA into its constituent nucleotides, andsimulate the free nucleotide dynamics to obtain shape functions.

• Incorporate empirical shape functions into larger scale simulationusing rigid body deformations of lower level solutions.

• Rigid-body motions form a closed category under recursion.

• Allows for computational compression by reusing redundantcomputations.

65

Towards Larger-Scale Scientific Computing Problems

Hierarchical Multi-Resolution Lie Group VIs

Schematic of hierarchical method Strand of DNA composed of nucleotides