Upload
joanna-willis
View
259
Download
2
Tags:
Embed Size (px)
Citation preview
Motion CaptureMotion Capture
Motion Capture
Motion capture Motion Tracking or Mocap is a technique of digitally recording movements for entertainment sports medical applicationshellip
Way of using a physical device to control animation
Motion Capture Technologyminusminusmagnetic
bull Electromechanical transducers
-Polhemus Fastrakbull Limited rangeresolutionbull Tetheredbull Sensitive to metalbull No identification problembull 6 dof informationbull Realtimebull Low frequency 30 to 120 Hzbull Few markers 10minus20
Motion Capture Technologyminusminus Mechanical Capture Systems
bull Any environmentbull Measures joint anglesbull Restricts the motion
Motion Capture Technologyminusminus Optical Capture Systems
bull Place markers on the object or on the actor
Passive reflection Active light sources
bull Cameras determine marker positions (Raw Point Data)
bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions
Motion Capture
Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time
bull Occlusioncorrespondence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture
Motion capture Motion Tracking or Mocap is a technique of digitally recording movements for entertainment sports medical applicationshellip
Way of using a physical device to control animation
Motion Capture Technologyminusminusmagnetic
bull Electromechanical transducers
-Polhemus Fastrakbull Limited rangeresolutionbull Tetheredbull Sensitive to metalbull No identification problembull 6 dof informationbull Realtimebull Low frequency 30 to 120 Hzbull Few markers 10minus20
Motion Capture Technologyminusminus Mechanical Capture Systems
bull Any environmentbull Measures joint anglesbull Restricts the motion
Motion Capture Technologyminusminus Optical Capture Systems
bull Place markers on the object or on the actor
Passive reflection Active light sources
bull Cameras determine marker positions (Raw Point Data)
bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions
Motion Capture
Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time
bull Occlusioncorrespondence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture Technologyminusminusmagnetic
bull Electromechanical transducers
-Polhemus Fastrakbull Limited rangeresolutionbull Tetheredbull Sensitive to metalbull No identification problembull 6 dof informationbull Realtimebull Low frequency 30 to 120 Hzbull Few markers 10minus20
Motion Capture Technologyminusminus Mechanical Capture Systems
bull Any environmentbull Measures joint anglesbull Restricts the motion
Motion Capture Technologyminusminus Optical Capture Systems
bull Place markers on the object or on the actor
Passive reflection Active light sources
bull Cameras determine marker positions (Raw Point Data)
bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions
Motion Capture
Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time
bull Occlusioncorrespondence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture Technologyminusminus Mechanical Capture Systems
bull Any environmentbull Measures joint anglesbull Restricts the motion
Motion Capture Technologyminusminus Optical Capture Systems
bull Place markers on the object or on the actor
Passive reflection Active light sources
bull Cameras determine marker positions (Raw Point Data)
bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions
Motion Capture
Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time
bull Occlusioncorrespondence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture Technologyminusminus Optical Capture Systems
bull Place markers on the object or on the actor
Passive reflection Active light sources
bull Cameras determine marker positions (Raw Point Data)
bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions
Motion Capture
Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time
bull Occlusioncorrespondence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture
Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time
bull Occlusioncorrespondence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
2 Perform a motion trial3 Compute marker positions from
camera images4 Identify and uniquely label markers5 Calculate joint angles from markers
paths
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
1 Find the skeleton dimensions and exact marker positions on the body
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
Automatic Calibration
bull Design Goalsbull 1048707 Fully
automaticbull 1048707 Any skeletonbull 1048707 Accurate
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
Inputbull Generic
Skeletonbull Actorrsquos
kinematics
structure and
rough handle
positions
Input
bull Calibration Data
bull Initial path data
that exercises
all of the subjectrsquos
DOFs
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
Independent Variables
bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
Marker Identification
At each frame motion capture gives us a set of points
We would like something more intuitive
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
Marker Identification
Making sense of raw datahellip
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture -- Optical motion capture process
Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture
Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Motion Capture
The minimized fitness function can be defined as
F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +
O1 (O1j ndash O1j)2 + c Cj2
where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature
Jj
Jj
Jj
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
BIOMECHANICAL MOVEMENTS
There is a huge amount literature devoted
simulating human movements See for instance the seminal paper by A Witkin
and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]
and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
BIOMECHANICAL MOVEMENTS
The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization
The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques
We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function
We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the
constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function
S
CJ
2
2
S
R
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
BIOMECHANICAL MOVEMENTS
bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations
bull To minimize the objective function the deriviate R(S) = 0 is used in [2]
bull A second-order Taylor series expansion is used to approximate R at S
bull Letting Sα = X ndash S we can solve
bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated
)(0)(2
2
SS
R
S
RSR
SS
R
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
BIOMECHANICAL MOVEMENTS
We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian
J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are
evaluated at S + Sα Solving for Sβ in
- C = J (Sα + Sβ)
drives C to zero The new value of S is incremented by (Sα + Sβ)
The algorithm computes new state vectors until the reduction
in the objective requires violating constraints
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Biomechanics To better understand human
movements biomechanical models must be developed that accurately describe human physiology and control strategies
The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Biomechanics
Experiment To implement a simulation that
determines the movement trajectory simultaneously with the optimum activation torques
The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate
The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Biomechanics
Experiment
bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH
bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal
During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored
A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Biomechanics
Experiment The masses and limb lengths of the spacetime-constraints
walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly
The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Biomechanics
Experiment The duration of a walk cycle can vary and resides under
the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988
Early Computer Animations (Pixarrsquos Luxo Jr 1986)
Can we hope to generate such motions automatically
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Some aspects of animation are best left to the animator (personality appeal etc)
But perhaps we can relieve the animator from the burden of making the motion look physically plausible
Forward Simulation Solves an initial value problembull Given initial state forces along the way and
equations describing the systemhellip
bull Integrate the equations
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
But What Abouthellipbull What if the animator wants to specify
both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some
conditions describing what happens during the motion
bull And what if some of the forces are unknown
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
An example We have a physical simulation that can be driven
as an initial-value problembull Given a sequence of torques that are applied by a
bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the
torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
An example Given a sequence of landmarks
compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo
bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy
bull One could propose low-energy trajectories that donrsquot go through landmarks
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Constrained optimization problem 1048707 Physical systems tend to minimize
energy 1048707 Spatial and physical constraints
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Example of Boundary Value Problem- moving particle
bull Particle position x(t)bull Force on particle f(t)
bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b
bull Objective function
x
dttfRt
t1
0
2)(
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Boundary Value Problembull Find f(t) such that the resulting x(t)
satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is
minimalbull Possible approach solve for f(t) and x(t)
simultaneously using the equation of motion as a constraint between them
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h
bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn
bull Solve a constrained optimization problem in these unknowns
bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n
bull Boundary constraints
ca =x1minusa = 0 cb =xn minusb= 0
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Constrained Optimizationbull Minimize an objective function
R(s1hellip s1)
in n variables s1hellip sn subject to m constraints
Ci(s1hellip sn ) = 0
Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the
constraints are satisfiedbull Idea iterate alternating between zeroing the gradient
of R and zeroing C
Spacetime Constraints
Iterative Convergence
Spacetime Constraints
Iterative Convergence