Motion Capture in Motion Capture in 3D Animation3D Animation Animation : MoviesAnimation : Movies Video GamesVideo Games Robot ControlRobot Control

Motion Capture as a Motion Capture as a ToolTool Motion capture (MOCAP) is an Motion capture (MOCAP) is an

effective 3D animation tool for effective 3D animation tool for realistically capturing human motion realistically capturing human motion

OutlineOutline RotoscopingRotoscoping The MOCAP PipelineThe MOCAP Pipeline Limitations of MOCAPLimitations of MOCAP The Future of MOCAPThe Future of MOCAP

RotoscopingRotoscoping Captured VideoCaptured Video

HistoryHistory

Johnston, et al., 1995Johnston, et al., 1995

RotoscopingRotoscopingHistoryHistory

TracingTracing

Johnston, et al., 1995Johnston, et al., 1995

RotoscopingRotoscoping Post ProcessingPost Processing

Johnston, et al., 1995Johnston, et al., 1995

The MOCAP PipelineThe MOCAP Pipeline CalibrationCalibration CaptureCapture 3D Position Reconstruction3D Position Reconstruction Fitting to the SkeletonFitting to the Skeleton Post Processing: IK Post Processing: IK Inverse KinematicsInverse Kinematics

CaptureCapture

Optical Magnetic Mechanical

Costa-Sousa, M., 2004Costa-Sousa, M., 2004

Infra Red LEDs

Magnetic Transmitter

Potentiometers

MOCAP issuesMOCAP issues Issues: Issues: data inaccuracy, Expensivedata inaccuracy, Expensive OpticalOptical: : $90’000 – $210’000$90’000 – $210’000 high res cameras, occlusion, no outdoors, high res cameras, occlusion, no outdoors, more and more markers, camerasmore and more markers, cameras MagneticMagnetic: : $100’000 – $120’000$100’000 – $120’000 heavier sensors, body wires, 10x accurate < heavier sensors, body wires, 10x accurate <

optical, smaller workspaces, sensitive to metal, optical, smaller workspaces, sensitive to metal, EMI, flooringEMI, flooring

MechanicalMechanical: $20’000 – $30’000: $20’000 – $30’000 US US limited movement, heavy and cumbersome,limited movement, heavy and cumbersome, must know rigid body at design time must know rigid body at design time

MOCAP issuesMOCAP issues

• Resolution/range of motion• Calibration• Accuracy

– Marker movement– Capture rate

• Occlusion/correspondence

MOCAP issues: MOCAP issues: Marker Placement

• Location should move rigidly with limb

• Stay away from bulging muscles• Shoulders: skeletal motion not

closely tied to skin• Shoulders: not a rigid body chain

MOCAP issues: MOCAP issues: Marker Set

3D points -> joint anglesIt matters!So do assumptions about human kinematics

CalibrationCalibration

Image Image SpaceSpace

World World SpaceSpace

(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)

3D Position 3D Position Reconstruction Reconstruction (Utopia)(Utopia)

3D Position 3D Position Reconstruction Reconstruction (Reality)(Reality)

Multiple MarkersMultiple Markers For small number of markers: SizeFor small number of markers: Size Occlusions are a problemOcclusions are a problem Multiple Hypothesis TrackingMultiple Hypothesis Tracking

Ringer, et al., 2002

C1

C2

Multiple MarkersMultiple Markers Mechanical Optical HybridsMechanical Optical Hybrids

http://www.animazoo.com/services/gypsyHybrid.htmhttp://www.animazoo.com/services/gypsyHybrid.htm

Fitting to the SkeletonFitting to the Skeleton Utopian approachUtopian approach

– 10 – 20% length 10 – 20% length changeschanges

Markers on both Markers on both sidessides

Joint Joint DisplacementDisplacement

Use Rotation Use Rotation Angles OnlyAngles Only

Post ProcessingPost Processing Motion EditingMotion Editing

– Cut, Copy, PasteCut, Copy, Paste Motion WarpingMotion Warping

– Speed up or Slow DownSpeed up or Slow Down– Rotate, Scale or TranslateRotate, Scale or Translate

Motion Signal ProcessingMotion Signal Processing– Smoother MotionsSmoother Motions

Limitations of MOCAPLimitations of MOCAP Only realistic motion capturedOnly realistic motion captured

– Cartoony Animations not possibleCartoony Animations not possible WYSIWYG (what you see is what you get)WYSIWYG (what you see is what you get)

– Can’t add more expressionCan’t add more expression– Continally need to recapture motionContinally need to recapture motion

Robertson, B., 2001

Eric Darnell, co-director of Antz

The main problem with motion capture associated with characters has to do with mass distribution, weight, and exaggeration. He says that it is impossible for a performer to produce the kind of motion exaggeration that a cartoon character needs, and the mass and weight of the performer almost never looks good when applied to a character of different proportions.

Richard Chuang, VP at PDI

The mapping of human motion to a character with non-human proportions doesn’t work, because the most important things you get out motion capture are the weight shifts and the subtleties and that balancing act of the human body. If the proportions change, you throw all that out the door, so you might as well animate it.

MOCAP issues: MOCAP issues: Marker Placement

• Location should move rigidly with limb

• Stay away from bulging muscles• Shoulders: skeletal motion not

closely tied to skin• Shoulders: not a rigid body chain

The Future of MOCAPThe Future of MOCAP Markerless MOCAPMarkerless MOCAP Cheaper MOCAPCheaper MOCAP Retargeting and reuse of mocap dataRetargeting and reuse of mocap data

hollywoodjesus.com

Ngan, S., 2004Bregler et al., 2002