A Data-Driven Approach to Quantifying Natural Human Motion

A Data-Driven Approach to A Data-Driven Approach to Quantifying Natural Human Quantifying Natural Human

MotionMotionSIGGRAPH ’05SIGGRAPH ’05

Liu Ren, Alton Patrick, Alexei A. Efros, Jassica K. Hodgins, aLiu Ren, Alton Patrick, Alexei A. Efros, Jassica K. Hodgins, and James M. Rehgnd James M. Rehg

Carnegie Mellon University & Georgia Institute of TechnoloCarnegie Mellon University & Georgia Institute of Technologygy

Date: 8/24/2005Date: 8/24/2005Speaker: AlvinSpeaker: Alvin

Alvin\CAIG Lab\NCTAlvin\CAIG Lab\NCTUU

A Data-Driven Approach to QuantA Data-Driven Approach to Quantifying Natural Human Motionifying Natural Human Motion

22

OutlineOutline

IntroductionIntroduction Input DataInput Data ApproachesApproaches ResultsResults Conclusions & Future WorksConclusions & Future Works



33

IntroductionIntroduction

GoalGoal Quantify the naturalness of human motionQuantify the naturalness of human motion

SolutionSolution Train a classifier based on human-labeled dataTrain a classifier based on human-labeled data Train only on positive examplesTrain only on positive examples

AssumptionAssumption Motions that we have seen repeatedly are Motions that we have seen repeatedly are

judged as naturaljudged as natural



44

Introduction cont.Introduction cont.

ApplicationApplication Verify the motion editing operationsVerify the motion editing operations

ContributionContribution Pose the questionPose the question Decompose human motion into Decompose human motion into

constituent partsconstituent parts Contribute a substantial databaseContribute a substantial database



55

OutlineOutline




66

Input DataInput Data

Training DatabaseTraining Database Testing MotionsTesting Motions



77

Training DatabaseTraining Database

1289 trials (422,413 frames)1289 trials (422,413 frames) 34 subjects34 subjects Vicon motion capture system of 12 cameVicon motion capture system of 12 came

rasras Downsample from 120Hz to 30 Hz Downsample from 120Hz to 30 Hz 41 markers41 markers



88

Data FormatData Format

ASF/AMC formatASF/AMC format Root positionRoot position Root orientationRoot orientation Relative joint angles of 18 jointsRelative joint angles of 18 joints

151-dimensional feature vector151-dimensional feature vector Joint angle (4) and velocity (4)Joint angle (4) and velocity (4) Root’s linear velocity(3) and angular Root’s linear velocity(3) and angular

velocity(4)velocity(4)



99

Testing Motions - NegativeTesting Motions - Negative

170 trials, 27774 frames170 trials, 27774 frames Edited motionsEdited motions Keyframed motionsKeyframed motions NoiseNoise Motion TransitionsMotion Transitions Insufficient cleaned motion capture dataInsufficient cleaned motion capture data



1010

Testing Motions - PositiveTesting Motions - Positive

261 trials, 92377 frames261 trials, 92377 frames MoCapMoCap NoiseNoise Motion TransitionsMotion Transitions

Judge by an expert viewerJudge by an expert viewer



1111

OutlineOutline




1212

ApproachesApproaches

FrameworkFramework Mixture of GaussiansMixture of Gaussians Hidden Markov ModelsHidden Markov Models Switching Linear Dynamic SystemSwitching Linear Dynamic System Naive Bayes (baseline method)Naive Bayes (baseline method) User StudyUser Study



1313

FrameworkFramework

Select the statistical modelSelect the statistical model Fit the model parameters using Fit the model parameters using

natural human motions as training natural human motions as training datadata

Compute a score for a novel input Compute a score for a novel input motionmotion



1414

EnsembleEnsemble

72

8

4+3

31

24 24

24

24

24

8

8

88

88

88

8

8

88

8 8

8 8



1515

Advantages of ensembleAdvantages of ensemble

Avoid the problem of overfittingAvoid the problem of overfitting Detect the unnatural motions confined tDetect the unnatural motions confined t

o a small set of joint angleso a small set of joint angles Provide guidance about what elements Provide guidance about what elements

deserve the most attentiondeserve the most attention



1616

ScoringScoring



1717

Mixture of Gaussians (MoG)Mixture of Gaussians (MoG)

The combinations of a finite number of Gaussian distributions

Used to model complex multidimensional distributions

EM algorithm is used to learn the parameters of the Gaussian mixture



1818

MoG cont.MoG cont.

500 Gaussians500 Gaussians Weak at modeling the dynamicsWeak at modeling the dynamics



1919

Hidden Markov Models Hidden Markov Models (HMM)(HMM)



2020

HMM cont.HMM cont.



2121

HMM cont.HMM cont.

The distribution of poses is represented The distribution of poses is represented with a mixture of Gaussianswith a mixture of Gaussians

State was modeled as a single GaussianState was modeled as a single Gaussian Parameters are learned by EMParameters are learned by EM 180 hidden states for full body180 hidden states for full body 60 hidden states for other feature group60 hidden states for other feature group



2222

Switching Linear Dynamic Switching Linear Dynamic System (SLDS)System (SLDS)

State is associated with LDS instead of GaState is associated with LDS instead of Gaussian distributionussian distribution

Second-order auto-regressive (AR) modelSecond-order auto-regressive (AR) model Initial state is described by MoGInitial state is described by MoG Parameters are estimated using EMParameters are estimated using EM 50 switching states for full body50 switching states for full body 5 switching states for other feature group5 switching states for other feature group



2323

Principal Component Principal Component AnalysisAnalysis

HMM & SLDSHMM & SLDS 99% variance kept for the full-body 99% variance kept for the full-body

modelmodel 99.9% variance kept for the smaller 99.9% variance kept for the smaller

modelmodel



2424

Naive Bayes (NB)Naive Bayes (NB)

Compute 1D marginal histogram for eacCompute 1D marginal histogram for each feature over the entire training databah feature over the entire training databasese

Each histogram has 300 bucketsEach histogram has 300 buckets Summing over the log likelihoods of eacSumming over the log likelihoods of eac

h of the 151 features for each frameh of the 151 features for each frame Nomalizing the sum by the lengthNomalizing the sum by the length



2525

User StudyUser Study

29 ♂ & 25 ♀29 ♂ & 25 ♀ 118 motion sequences118 motion sequences 2 segments with a 10 minute break2 segments with a 10 minute break The order of sequences is randomThe order of sequences is random



2626

OutlineOutline




2727

ResultsResults



2828

Receiver Operating Receiver Operating Characteristic Curve (ROC Characteristic Curve (ROC

curve)curve) False positiveFalse positive

Classifier predicts natural when the motion is unnaClassifier predicts natural when the motion is unnaturaltural

True positive rateTrue positive rate tp / (tp + fn)tp / (tp + fn)

False positive rateFalse positive rate fp / (fp + tn)fp / (fp + tn)

Without need to choose a thresholdWithout need to choose a threshold The more area under the ROC curve, the more The more area under the ROC curve, the more

accurate the testaccurate the test



2929



3030



3131



3232



3333

Demo



3434

OutlineOutline




3535

ConclusionsConclusions

Unusual motions are sometimes Unusual motions are sometimes labeled unnatural (like falling)labeled unnatural (like falling)

Short errors and slow motions may Short errors and slow motions may not be detectednot be detected

Used to improve the performance of Used to improve the performance of motion synthesis and motion editing motion synthesis and motion editing toolstools



3636

Future WorksFuture Works

Explore dimensionality reduction Explore dimensionality reduction approaches for SLDS modelapproaches for SLDS model

More sophisticated methods for More sophisticated methods for normalizing or computing the scorenormalizing or computing the score

Screening for the style of a particular Screening for the style of a particular cartoon charactercartoon character



3737

Thank you for your attention

Documents

A Data-Driven Approach to Quantifying Natural Human Motion