Motion Capture in Motion Capture in 3D Animation3D Animation Animation : MoviesAnimation : Movies Video GamesVideo Games Robot ControlRobot Control
Motion Capture as a Motion Capture as a ToolTool Motion capture (MOCAP) is an Motion capture (MOCAP) is an
effective 3D animation tool for effective 3D animation tool for realistically capturing human motion realistically capturing human motion
OutlineOutline RotoscopingRotoscoping The MOCAP PipelineThe MOCAP Pipeline Limitations of MOCAPLimitations of MOCAP The Future of MOCAPThe Future of MOCAP
RotoscopingRotoscoping Captured VideoCaptured Video
HistoryHistory
Johnston, et al., 1995Johnston, et al., 1995
RotoscopingRotoscopingHistoryHistory
TracingTracing
Johnston, et al., 1995Johnston, et al., 1995
RotoscopingRotoscoping Post ProcessingPost Processing
Johnston, et al., 1995Johnston, et al., 1995
The MOCAP PipelineThe MOCAP Pipeline CalibrationCalibration CaptureCapture 3D Position Reconstruction3D Position Reconstruction Fitting to the SkeletonFitting to the Skeleton Post Processing: IK Post Processing: IK Inverse KinematicsInverse Kinematics
CaptureCapture
Optical Magnetic Mechanical
Costa-Sousa, M., 2004Costa-Sousa, M., 2004
Infra Red LEDs
Magnetic Transmitter
Potentiometers
MOCAP issuesMOCAP issues Issues: Issues: data inaccuracy, Expensivedata inaccuracy, Expensive OpticalOptical: : $90’000 – $210’000$90’000 – $210’000 high res cameras, occlusion, no outdoors, high res cameras, occlusion, no outdoors, more and more markers, camerasmore and more markers, cameras MagneticMagnetic: : $100’000 – $120’000$100’000 – $120’000 heavier sensors, body wires, 10x accurate < heavier sensors, body wires, 10x accurate <
optical, smaller workspaces, sensitive to metal, optical, smaller workspaces, sensitive to metal, EMI, flooringEMI, flooring
MechanicalMechanical: $20’000 – $30’000: $20’000 – $30’000 US US limited movement, heavy and cumbersome,limited movement, heavy and cumbersome, must know rigid body at design time must know rigid body at design time
MOCAP issuesMOCAP issues
• Resolution/range of motion• Calibration• Accuracy
– Marker movement– Capture rate
• Occlusion/correspondence
MOCAP issues: MOCAP issues: Marker Placement
• Location should move rigidly with limb
• Stay away from bulging muscles• Shoulders: skeletal motion not
closely tied to skin• Shoulders: not a rigid body chain
MOCAP issues: MOCAP issues: Marker Set
3D points -> joint anglesIt matters!So do assumptions about human kinematics
CalibrationCalibration
Image Image SpaceSpace
World World SpaceSpace
(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)(X,Y)(X,Y) (X,Y,Z)(X,Y,Z)
3D Position 3D Position Reconstruction Reconstruction (Utopia)(Utopia)
3D Position 3D Position Reconstruction Reconstruction (Reality)(Reality)
Multiple MarkersMultiple Markers For small number of markers: SizeFor small number of markers: Size Occlusions are a problemOcclusions are a problem Multiple Hypothesis TrackingMultiple Hypothesis Tracking
Ringer, et al., 2002
C1
C2
Multiple MarkersMultiple Markers Mechanical Optical HybridsMechanical Optical Hybrids
http://www.animazoo.com/services/gypsyHybrid.htmhttp://www.animazoo.com/services/gypsyHybrid.htm
Fitting to the SkeletonFitting to the Skeleton Utopian approachUtopian approach
– 10 – 20% length 10 – 20% length changeschanges
Markers on both Markers on both sidessides
Joint Joint DisplacementDisplacement
Use Rotation Use Rotation Angles OnlyAngles Only
Post ProcessingPost Processing Motion EditingMotion Editing
– Cut, Copy, PasteCut, Copy, Paste Motion WarpingMotion Warping
– Speed up or Slow DownSpeed up or Slow Down– Rotate, Scale or TranslateRotate, Scale or Translate
Motion Signal ProcessingMotion Signal Processing– Smoother MotionsSmoother Motions
Limitations of MOCAPLimitations of MOCAP Only realistic motion capturedOnly realistic motion captured
– Cartoony Animations not possibleCartoony Animations not possible WYSIWYG (what you see is what you get)WYSIWYG (what you see is what you get)
– Can’t add more expressionCan’t add more expression– Continally need to recapture motionContinally need to recapture motion
Robertson, B., 2001
Eric Darnell, co-director of Antz
The main problem with motion capture associated with characters has to do with mass distribution, weight, and exaggeration. He says that it is impossible for a performer to produce the kind of motion exaggeration that a cartoon character needs, and the mass and weight of the performer almost never looks good when applied to a character of different proportions.
Richard Chuang, VP at PDI
The mapping of human motion to a character with non-human proportions doesn’t work, because the most important things you get out motion capture are the weight shifts and the subtleties and that balancing act of the human body. If the proportions change, you throw all that out the door, so you might as well animate it.
MOCAP issues: MOCAP issues: Marker Placement
• Location should move rigidly with limb
• Stay away from bulging muscles• Shoulders: skeletal motion not
closely tied to skin• Shoulders: not a rigid body chain
The Future of MOCAPThe Future of MOCAP Markerless MOCAPMarkerless MOCAP Cheaper MOCAPCheaper MOCAP Retargeting and reuse of mocap dataRetargeting and reuse of mocap data
hollywoodjesus.com
Ngan, S., 2004Bregler et al., 2002