View
4
Download
0
Category
Preview:
Citation preview
Institute of Visual Computing
Inverse Kinematics II and Motion Capture
Luca Ballan
Mathematical Foundations of Computer Graphics and Vision
Comparison 0 1
2 3
A B C
is equal to the length of the bone computing the derivative on the
angle is easy
the meaning of is not intuitive However this incorporates the real concept of geodesic interpolation/averaging has to be done in this
space
Real exponential map Fake exponential map
Special Euclidean group SE(3)
Angle/axis representation of the rotation
Twist
exp
Screw motion: rotation along an axis + a translation along the same axis.
Proposition: Any rigid transformation in SE(3) can be expressed as a rotation about an axis combined with a translation parallel to that axis.
Special Euclidean group SE(3)
Twist exp
Tangent space of SE(3) at the identity I
hat
Properties
Identity
Inverse (basic property of exp map)
in general not “Linear” (like in so(3))
Derivative
Metric on SE(3)
Riemannian/Geodesic/Angle metric ( = the length of the geodesic connecting and )
Interpolation and averaging can be performed in the space of rigid transformations also
SLERP (spherical linear interpolation)
I
Metric on SE(3)
Riemannian/Geodesic/Angle metric ( = the length of the geodesic connecting and )
Interpolation and averaging can be performed in the space of rigid transformations also
SLERP (spherical linear interpolation)
Fréchet mean
I
A note on Interpolation
Inverse Kinematics
0 1 2 3
A B C
Target q
Newton’s method let be the current estimate for the solution compute the Taylor expansion of around
Inverse Kinematics
0 1 2 3
A B C
Target q
Newton’s method let be the current estimate for the solution compute the Taylor expansion of around
Find such that
Inverses
Find such that
is not injective (in general) (multiple inverse exists for each point of the image)
If it is injective, then the left-Inverse exists
Inverses
Find such that
might not contain the solution
Better to find the such that is minimized
Inverses
Find such that is minimized,
To force the unicity of the solution we force to be the one with minimum norm
Pseudo inverse works well in many cases but not near a singularity (singular values close to 0)
Make the constraint on the norm of soft, not hard as before
Singularities
this is always non singular if the damping factor is correctly chosen
the solution of this is the same as solving for
The Newton’s method with this damping step is known as the Damped Least Square method, or as the Levenberg-Marquardt algorithm.
Heuristic approaches
Cyclic Coordinate Descent is an alternating optimization approach where only one coordinate at a time is optimized.
Let be the estimate of the solution at iteration t
Heuristic approaches
Cyclic Coordinate Descent is an alternating optimization approach where only one coordinate at a time is optimized.
Let be the estimate of the solution at iteration t
Heuristic approaches
Cyclic Coordinate Descent is an alternating optimization approach where only one coordinate at a time is optimized.
Let be the estimate of the solution at iteration t
solving for a single twist is a very easy problem
Heuristic approaches
*
Heuristic approaches
Content
Inverse Kinematics
Kinematic Trees/Graphs
Pose Estimation/Motion Capture
Kinematic Tree
e2 e1
e3
Root
Root
A
B
C
D
E
F
G
I
H L
M
N
A
B
C
D E
F
G I H
L
M N
e1 e2
e3
a kinematic tree is a tree of rigid transformations
Kinematic Tree
e2 e1
e3
Root
a kinematic tree is a tree of rigid transformations
Root
A
B
C
D
E
F
G
I
H L
M
N
A
B
C
D E
F
G I H
L
M N
e1 e2
e3
each path in a kinematic tree from the root to any other node is a kinematic chain
Kinematic chain
Kinematic Tree
e2 e1
e3
Root
a kinematic tree is a tree of rigid transformations
Root
A
B
C
D
E
F
G
I
H L
M
N
A
B
C
D E
F
G I H
L
M N
e1 e2
e3
each path in a kinematic tree from the root to any other node is a kinematic chain
Kinematic Tree: Human Skeleton a kinematic tree is a tree of rigid transformations
The pelvis is typically denoted as the root of the tree (but this is only a convention)
each path in a kinematic tree from the root to any other node is a kinematic chain
An Inverse Kinematics Problem on a tree consists in finding the
configuration of the kinematic chain for which the distance between the end effectors and some pre-defined targets points are minimized it is an Inverse Kinematics Problem with Multiple Targets
Inverse Kinematics for Trees Given a kinematic tree
e2 e1
e3
Root
A
B C
D E
F
G I H
L
M N
Target q2
Target q3
Target q1
Inverse Kinematics for Trees Given a kinematic tree
Multi-objective optimization
e2 e1
e3
Root
A
B C
D E
F
G I H
L
M N
Target q2
Target q3
Target q1
Find x such that it simultaneously minimizes all the distances with the targets
It is difficult to find x satisfying all the minima
Inverse Kinematics for Trees Given a kinematic tree
e2 e1
e3
Root
A
B C
D E
F
G I H
L
M N
Target q2
Target q3
Target q1
Find the x which minimizes the sum of the squared errors
i.e. the mean solution
equivalent to
norm
Inverse Kinematics for Trees Given a kinematic tree
e2 e1
e3
Root
A
B C
D E
F
G I H
L
M N
Target q2
Target q3
Target q1
Find the x which minimizes the sum of the errors
i.e. the median solution
equivalent to
norm
Inverse Kinematics for Trees: Solution
is still a non-linear least square optimization problem
The norm case
Newton’s method or Levenberg-Marquardt
…
Kinematic Graph
e1
Root
Root
A
B
C
D
E
F
e1
a kinematic graph is a graph of rigid transformations (it contains cycles)
(parallel manipulators)
G
H
I
L
M
N
Inverse Kinematics for Graphs
e1 e2
e2 e1
Root
Root
A
B
C
D
E
F
G
H
I
L
M
N
e1
subject
Target q
Constrained optimization
Lagrange Multipliers or simply
Content
Inverse Kinematics
Kinematic Trees/Graphs
Pose Estimation/Motion Capture
Pose Estimation
Given an image depicting an articulated object, estimate the pose of that object
[Eichner 10]
determine all the DOF of that pose
Pose Estimation
Marker-based pose estimation Marker-less pose estimation
[Faceshift]
Pose estimation for multiple frames is called motion capture
Marker-based Pose Estimation
Infrared cameras with IR illuminator
IR reflective markers
Vicon system
Marker-based Pose Estimation
Images captured from the IR cameras
…
Kinematic tree with a set of end effectors
Set of 3D points (computed using triangulation) Correspondences
between 3D points and end effectors
Triangulate 2D points -> 3D points Compute correspondences between 3D points and end effectors
Marker-based Pose Estimation
Kinematic tree with a set of end effectors
Targets
…
Multi-objective optimization
…
Correspondences If correspondences between 3D points and targets are not known (or the heuristic used to
compute them is not accurate), these need to be estimated together with the pose
One of the possible solution to this problem is the ICP registration (ICP=Iterative Closest Points) [Besl & McKay 92]
But how can we formulate mathematically this problem?
Correspondences
Complete bi-partite graph (with weights ) (representing all the possible matching)
All the edges on this graph are possible matching candidates We need to find the actual matches (unknown to the problem)
Targets End effectors
Correspondences Target i in X is matched with end effector j in Y
cost of matching target i in X to end effector j in Y
• Binary Integer problem + Continuous problem • Alternating optimization approach
• Integer Programming (Hungarian algorithm) + Gradient Descent
Correspondences It might happen that some targets are missing or that some targets are wrong
Kinematic tree with a set of end effectors
Targets
Occluded, not visible Wrong estimate
One end effector does not have a corresponding target
One target does not have a corresponding end effector
Correspondences
virtual nodes
additional unknowns
Marker-Less Pose Estimation
Output: Input:
Scene Motion (angles and positions)
Marker-Less Pose Estimation
Comparison
Input video frames at time t
Pose Rendered images
Generative model
Comparison Error
Correct Pose
Minimum
Maximum likelihood
Generative Model
the motion of a vertex
the linear combination of all the motions that the vertex would undergo if rigidly attached to
every bone, one at a time Kinematic tree + 3D model
At a rigging pose
Marker-Less Pose Estimation
Comparison
Input video frames at time t
Pose Rendered images
Generative model
Comparison Error
Correct Pose
Minimum
Maximum likelihood
Comparison
Input video frame at time t
Pose
Generative model
= is a distance map in 3D between a point and S
Solution
= is a distance map in 3D
Final Examination
Homework: 10% each Oral Exam: 60%
(60%-40%) about 40min per person (20 Luca, 20 J.C.) (100%) about 1h per person (30 Luca, 30 J.C.)
each person has to answer different questions/exercises spanning the entire
program ETH will schedule the exam sometime in August
For the ones who didn’t do the homework who did not do all the homework who is not satisfied with the homework grades
and want to improve it
he/she can do the 100% exam to recover the missing points (inform in advance)
Recommended