of 17/17
Real Time Motion Capture Using a Single Time-Of-Flight Camera Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123 邱邱邱

Real Time Motion Capture Using a Single Time-Of-Flight Camera

  • View

  • Download

Embed Size (px)


Real Time Motion Capture Using a Single Time-Of-Flight Camera. Varun Ganapathi , Christian Plagemann , Daphne Koller , Sebastian Thrun CVPR 2010. Q36981123 邱碁森. Outline. Introduction Probabilistic Model Inference Experiments Conclusions. Introduction. - PowerPoint PPT Presentation

Text of Real Time Motion Capture Using a Single Time-Of-Flight Camera

Real Time Motion Capture Using a Single Time-Of-Flight Camera

Real Time Motion CaptureUsing a Single Time-Of-Flight CameraVarun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian ThrunCVPR 2010

Q36981123 OutlineIntroductionProbabilistic ModelInferenceExperimentsConclusionsIntroductionMotion capture is used to human-machine interaction, smart surveillance and so on.Time-of-flight sensors offers rich sensory information, not sensitive to changes in lighting, shadows, and some other problems.This paper propose an efficient filtering algorithm for tracking human pose for fast operation at video frame.

What is Probabilistic Model?A tree-shaped kinematic chain (skeleton)Human body is modeled as 15 body partsThe transformations of the body Xt at time t is a set: Xt= {Xi}, i = 1~15

X1: the root of tree the pelvis partroot(pelvis): could freely rotate and translate other parts: connected to the their parent, allow to rotate (not to translate)What is Probabilistic Model? (cont.)The absolute orientation of a body part i: Wi(X)multiplying the transformations of its ancestors in the kinematic chainWi(X) = X1X2 ... Xparent(i) XiWhy need the Probabilistic Model?Determine the most likely state at at time tthe pose set Xtthe first discrete-time derivative set Vt (velocities)zt: the recorded range measurements

The system is modeled as a dynamic Bayesian network (DBN)

Probabilistic ModelThe measured range scan is denoted by z = {zk} k=1M where zk gives the measured depth of the pixel at coordinate k.Probabilistic ModelAssumption: the accelerations in our system are drawn from a Gaussian distribution with zero mean

InferenceHow to perform efficient inference at each frame?Model Based Hill Climbing Search (HC)A component locally optimizes the likelihood function Evidence Propagation (EP)An inference procedure generate likely states which are used to initialize the HCInference n. 9Model Based Hill Climbing Searchcoarse-to-fineThe procedure can then potentially be applied to a smaller interval about the value chosen at the coarser levelhill-climbingStart from the base of kinematic chain which includes the largest body parts, and proceed toward the limbs

123optimize the X axis0.50.450.4...-0.35-0.4-0.45-0.5sample:then chose the best one10Evidence PropagationProblem: fast motion cause motion blurocclusion cause the estimate of the state of hidden parts to driftthe likelihood function has ridges (difficult to navigate)This procedure that identifies promising locations for body parts to find likely posesEvidence PropagationSteps in this procedure:Body Part Detection: identify possible body part locations from the current range imageProbabilistic Inverse Kinematics: update the body configuration X given possible correspondences between mesh vertices and part detections Data Association and Inference: determine the best subset of such correspondencesBody Part DetectionFive body parts: head, left hand, right hand, left foot and right foot are found from the current range image.Interest Point(AGEX) Detection start on the geodesic centroid of the mesh: AGEX1(M)recursively find the vertex AGEXk(M) which has max geodesic distance to AGEXk-1(M)Identification of Partspoints are classified as body part by training these data using a marker-based motion capture system( LED mark)

C. Plagemann, V. Ganapathi, D. Koller, and S. Thrun. Realtimeidentification and localization of body parts from depthimages. In IEEE Int. Conference on Robotics and Automation(ICRA), Anchorage, Alaska, USA, 2010.13Evidence Propagation

ExperimentsUsing a Swissranger SR4000 Time-of-Flight camera

Tracking results on real-world test sequences, sorted from most complex (left) to least complex (right).

ExperimentsA Tennis sequence

Only use Model-Based search Our combined trackerConclusions A novel algorithm for combining part detections with local hill-climbing for marker less tracking of human pose.With the hybrid, GPU-accelerated filtering approach