Lecture 13: RLSC - Prof. Sethu Vijayakumar 1

Lecture 13: Dynamical Systems based Representation

Contents: • Differential Equation

• Force Fields, Velocity Fields

• Dynamical systems for Trajectory Plans

• Generating plans dynamically

• Fitting (or modifying) plans

• Imitation based learning

Thanks to my collaborator Auke Ijspeert (EPFL) for many of the contents on the slides for this lecture.

Lecture 13: RLSC - Prof. Sethu Vijayakumar 2

Movement policies as Dynamical Systems

• Represent complex movements in globally stable attractor landscapes of nonlinear autonomous differential equations

• Choose kinematic representation for easy re-use in different workspace location

• Ensure easy temporal and spatial scaling (topological equivalence)

• Use local learning to modify the attractors according to demonstration of teacher and self-learning

Discreet & Rhythmic Movement Primitives

),,( goalf desdes

Lecture 13: RLSC - Prof. Sethu Vijayakumar 3

Discreet & Rhythmic Movement superposition

Open loop with oscillators Closed loop control in horizontal plane

Open (vertical) + Closed (horizontal) loop control

Lecture 13: RLSC - Prof. Sethu Vijayakumar 4

• Differential equation: an equation that describes how state variables evolve over time, for instance:

)( ycy

What is a Differential Equation?

Lecture 13: RLSC - Prof. Sethu Vijayakumar 5

• Ordinary differential equation: differential equation that involves only ordinary derivatives (as opposed to partial derivatives)

• Autonomous equation: differential equation that does not

(explicitly) depend on time

• Linear differential equation: differential equation in which the state variables only appear in linear combinations

• Nonlinear differential equation: differential equation in which

some state variables appear in nonlinear combinations (e.g. products, cosine,…)

• Fixed point: point at which all derivatives are zero (can be an attractor, a repeller, or a saddle point, cf later)

• Limit cycle: periodic isolated closed trajectory (can only occur in nonlinear systems)

Some Definitions

Lecture 13: RLSC - Prof. Sethu Vijayakumar 6

Limit cycles Chaos Attractors

Attractor

saddles

Unstable node

From Strogatz 1994

Interesting Regimes of Differential Eqs.

Lecture 13: RLSC - Prof. Sethu Vijayakumar 7

• First order linear system:

• How to solve this equation, for a given y(t=0), c, and ?

• Two methods: analytical solution or numerical integration

• Analytical solution:

• Numerical integration: Euler method, Runge-Kutta,…

)( ycy

ctcyty )exp()()( 0

First Order Linear Systems

Lecture 13: RLSC - Prof. Sethu Vijayakumar 8

)( ycy

ctcyty )exp()()( 0

First Order Linear Systems

Lecture 13: RLSC - Prof. Sethu Vijayakumar 9

yx

yxcy

))((

Second Order Linear Systems

Lecture 13: RLSC - Prof. Sethu Vijayakumar 10

yx

yxcy

))((

8

8

1

8

2

8

Second Order Linear Systems

Lecture 13: RLSC - Prof. Sethu Vijayakumar 11

yxx

xyy

coscos2

coscos2

Attractor

saddles

Repeller

Second Order Non-linear System

Lecture 13: RLSC - Prof. Sethu Vijayakumar 12

Task of the trajectory formation system: • To encode demonstrated trajectories with high accuracy, • To be able to modulate the learned trajectory when:

• Perceptual variables are varied (e.g. timing, amplitude) • Perturbations occur

Movement recognition

Movement execution (Inv. Dyn.)

Trajectory formation

system

Joint angles

Learning a movement by demonstration

Lecture 13: RLSC - Prof. Sethu Vijayakumar 13

Traditionally, the problem of replaying a trajectory has been decomposed into two different issues: • One of encoding the trajectory, and • One of modifying the trajectory, for instance, in case the

movement is perturbed, or when it requires to be modulated.

Our approach: combine both abilities in a nonlinear

dynamical system Aim: to encode the trajectory in a nonlinear dynamical system

with well defined attractor landscape

Encoding a trajectory

Lecture 13: RLSC - Prof. Sethu Vijayakumar 14

y

dy/dt

Single point attractor

Discrete movements

y

dy/dt

Limit cycle attractor

Rhythmic movements

Nonlinear Dynamical Systems Approach

Lecture 13: RLSC - Prof. Sethu Vijayakumar 15

Sensuit

Two types of movement recordings

« Kinesthetic » demonstration

Lecture 13: RLSC - Prof. Sethu Vijayakumar 16

position velocity

Demo for

One DOF

Pointing Demonstration

Lecture 13: RLSC - Prof. Sethu Vijayakumar 17

Shaping Attractor Landscapes

Goal: g

?

z zz g y z

y f z

z zz g y z

y z

Can one create more

complex dynamics by non-

linearly modifying the

dynamic system:

Lecture 13: RLSC - Prof. Sethu Vijayakumar 18

Goal: g

Amplitude and phase system

Discreet Control Policy

Output System

Lecture 13: RLSC - Prof. Sethu Vijayakumar 19

Shaping Attractor Landscapes

1

1

2 0

0

,

where

,

1exp and

2

z z

y

v v

x

k

i i

i

k

i

i

i i i

z g y z

y f x v z

v g x v

x v

w b v

f x v

w

x xw d x c x

g x

A globally stable learnable nonlinear point attractor:

Local Linear

Model Approx.

Canonical Dynamics

Trajectory Plan Dynamics

Lecture 13: RLSC - Prof. Sethu Vijayakumar 20

Learning the Attractor

Given a demonstrated trajectory y(t)demo and a goal g

Extract movement duration

Adjust time constants of canonical dynamics to movement duration

Use LWL to learn supervised problem

target ,demo

y

yy z f x v

[ Stefan Schaal, Sethu Vijayakumar et al, Proc. of Intl. Symp. Rob. Res.(ISRR) (2001) ]

Also extended to rhythmic primitives :

Lecture 13: RLSC - Prof. Sethu Vijayakumar 21

Trajectory following & Generalization

Backhand Demonstration Backhand Reproduction

Lecture 13: RLSC - Prof. Sethu Vijayakumar 22

Basic oscillator:

Output signal:

1

)( 0

rrr

N

i i

N

i

T

ii

m

Iwzy

zyyz

1

1

))((

],,[ vx

TvxI ],[

Rhythmic Control Policies

Lecture 13: RLSC - Prof. Sethu Vijayakumar 23

Modulation of Goal: Anchor

Lecture 13: RLSC - Prof. Sethu Vijayakumar 24

Drumming: Modulating Frequency

Drumming: Kinesthetic Demo

Imitation and Modulation

Biped Locomotion using DMPs

Approach

5-link biped robot

- phase resetting and frequency adaptation - phase regulation between coupled oscillators

[Nakanishi, Morimoto, Endo, Cheng, Schaal, and Kawato, 2004]

•Movement primitives as a central pattern generator (CPG) • Synchronization between CPG and robot

Height 40cm Mass hip 2.0kg knee 0.64kg foot 0.06kg

Lecture 13: RLSC - Prof. Sethu Vijayakumar 25

Control Architecture

Rhythmic Movement primitives

Phase resetting

periodic pattern generation learning from demonstrated trajectories

reset the phase of the oscillator at heel contact

Dynamical primitives CPG

oscil lator

output with local models

Robot and Environment

(index: i=1~# of oscillators)

phase resetting frequency

update

[Nakanishi, Morimoto, Endo, Cheng, Schaal, and Kawato, 2004]

Frequency adaptation

Phase regulation among oscillators coupling among oscillators

Lecture 13: RLSC - Prof. Sethu Vijayakumar 26

Experiments: Bipedal Locomotion

Lecture 13: RLSC - Prof. Sethu Vijayakumar 27