Motion Capture

Animation – A broad Brush

Traditional Methods Cartoons, stop motion

Keyframing Digital inbetweens

Motion Capture What you record is what you get

Simulation Animate what you can model (with equations)

What is Motion Capture ?

Capture of motion of (human) actor Motion Capture of an object involves sensing, digitizing,

and recording that object in motion• Whole body

• Hands

• Face

One way of acting out an animation

Why Motion Capture ?

All the fine details of human motion will be reproduced Naturalness of human motion is in its subtle details

Style Mood Weight shift, and so on

Applications Animation / Interactive characters Feature films / video games Medicine Sports Robotics

What is captured ?

What we need is Position and orientation of the root segment Joint angles Length of each link (is it possible ?) Skeleton connectivity Skin deformation

The mocap system actually provides Marker positions on the skin, or The positions and orientations of markers

Motion Capture Pipeline

Record motion from physical object Use motion to animate virtual object

Set up

Equipment

Set up

EquipmentRecord

Motion

Record

MotionProcess

Data

Process

DataGenerate

Animation

Generate

Animation

What Types of Objects?

Human whole body Portions of body Facial animation Animals Puppets Other objects

How to use the data?

Off-line Processing by filtering, inverse kinematics Produce libraries of motion trajectories

• Choose among them

• Blend between them

• Modify on the fly

On-line (performance animation) Driving character directly based on what actor does in

real time

Motion Capture Technologies

Optical passive Optical active Electromagnetic Electromechanical Acoustic Optical fiber

Optical Passive

High resolution, high speed cameras 120-1000 Hz, 1000x1000 pixels Infrared or visible light strobe Retro-reflective markers

Pros High quality Flexible marker placement No cables Not seriously constrained by markers

Cons Extensive post-processing Controlled environments (Indoor only, no sunlight) Correspondence problem Occlusion

Optical Active

Phoenix Tech, ReActor, Optitrak, Visualeyez Markers emit electric signal (LEDs) No correspondence problem

Electromagnetic

Electromagnetic field transmitter Sensors on the body

Both position and orientation information Pros

Realtime No occlusion/correspondence

Cons Limited accuracy Smaller workspace Heavier sensors and wires on body Sensitive to electromagnetic interference

Company: Ascension, Polhemus

Electromechanical

Exoskeleton Mechanical skeleton attached on body Pros

Truly realtime (500 Hz) No range limit No occlusion/correspondence problem

Cons Restriction of movement Fixed sensor positions

Company: Sacros, Gypsy

Optical Fiber

Bending the fiber attenuates the transmitted light Recently used for full-body capture Fiber Optic Sensors

Flexible FO sensors strapped to various parts of the performers body.

Sensors can directly measure joint rotations Used in conjunction with electromagentic

sensor for head and torso.

Shapewrap II Measurand

Acoustic

The multiple transmitters trigger “click” The receivers on the body measures the time taken

for the sound travel Pros

No occlusion

Cons Limited range Limited number of sensors Cables on body Acoustic interference

Motion Capture Systems

Challenges: Signal is not perfect

• Noisy

• missing data

• not perfectly aligned with joints

Retargeting• Data is only valid for virtual character who possesses

same scale as real character.

Motion Capture Systems

Challenges: Even if motion capture data was perfect, we still have th

e following challenges:• Re-use – use the motion for a slightly different purpose

• Creating impossible motion – Motion capture won’t do it, but may be desired in animation

• Change of intent – we can’t always predict what motion we will need

Take Home Message: Motion Capture captures a particular, single motion.

Motion Capture Data

So what CAN we do with motion capture data? We can

• speed up

• slow down

• time warp

• Motion warp

However, one must remember that Captured data is Sampled Data.

Retargeting Motion Capture Data

In general, moCap data is useful for a single articulated figure. Adapt motion to another character

Motion capture data formats

No “standard” moCap data format Defacto standards from motion capture system manufac

turers Must specify both structure of skeleton as well as sampl

ed data for each joint.

Demo

motion capture technologies Vicon (passive optical)

• http://www.vicon.com/spots/vicongallery.html

PTI (active optical) Ascension technolgy (magnetic) MetaMotion (mechanical) OrganicMotion (markerless) Image Metrics (markerless -facial) Moven (inertial) LumiNetra (light aware SIGGRAPH 2007 paper) Mova (make-up)

Motion capture data formats

Popular formats Acclaim File Format

• .asf (Acclaim skeleton format)

• .amc (Acclaim motion capture)

Biovision• .bva (BioVision animation)

• .bvh (BioVision Hierarchical)

C3D• Independent Binary format with programmer support.

• http://www.c3d.org

Acclaim

The Acclaim format is made up of two files a skeleton file

• the ASF file (Acclaim Skeleton File).

A motion file• the AMC file (Acclaim Motion Capture data).

Acclaim Skeleton File

Parsing the ASF File “:version”

• the version of the skeleton definition ":name"

• the skeleton to be named ":units"

• defines the units to be used for various types of data • Ex) mass, length, angle

":documentation" ":root"

• the root segment of the skeleton hierarchy • "axis" : defines the rotation order of the root object • "order" : order they will appear in the AMC file

– TX TY TZ RZ RX RY Translation & Rotation Order• "position" and "orientation"

– the starting position and orientation of the root – These are typically, but not always, zero

Acclaim Skeleton File

Parsing the ASF File “:bonedata”

• a description of each of the segments

• "begin" and "end" pair

• “id” : a unique id for the segment

• "name" : the name of the segment

• "direction" – defines the direction from the parent to the child segment

• "length" : The length of the segment

• "axis" : an axis of rotation for the segment

• "dof" : specifies the number of motion channels

• "limits" : limits on each of the channels in the dof specification

Acclaim Skeleton File

Parsing the ASF File “:hierarchy”

• Describes the hierarchy of the bones

declared in the “:bonedata” section

:hierarchybegin root hips hips hips1 hips2 hips3 hips1 chest chest chest1 chest2 chest3 chest1 neck neck head chest2 leftcollar leftcollar leftuparm leftuparm leftlowarm leftlowarm lefthand chest3 rightcollar rightcollar rightuparm rightuparm rightlowarm rightlowarm righthand hips2 leftupleg leftupleg leftlowleg leftlowleg leftfoot hips3 rightupleg rightupleg rightlowleg rightlowleg rightfootend

Acclaim Motion Capture Data

Parsing the AMC file defines the actual channel animation a line declaring the frame number the bone animation data

• bone name and data for each channel

defined for that bone.

Tool ASF/AMC Viewer

Biovision BVH

BVH(BioVision Hierarchical data ) developed by Biovision, a motion capture services company A BVH file has two parts

• a header section – describes the hierarchy and initial pose of the skeleton

• a data section – contains the motion data

Biovision BVH

Header section "HIERARCHY“

• "ROOT" – followed by the name of the root – " { " and " } " pair– "OFFSET"

» X,Y and Z offset of the segment from its parent

– "CHANNELS" » the number of channels » the type of each channel

• "JOINT" – identical to the root definition except for the number of channels – "OFFSET " , "CHANNELS"

• "End Site" – indicates that the current segment is an end effector (no children)– "OFFSET "

- 6 channels for the root (Tx Ty Tz Rz Rx Ry)

- 3 channels for every other object (Rz Rx Ry)

Biovision BVH

Motion Section "MOTION"

• followed by a line indicating the number of frames

• "Frames:" – the number of frames

• "Frame Time:" – the sampling rate of the data – Ex) 0.033333 30 frames a second

• The rest of the file contains the actual motion data – The numbers appear in the order of the channel specifications as

the skeleton hierarchy was parsed

Biovision BVH

Interpreting the data To calculate the position of a segment

• Translation information– For any joint segment

» the translation information will simply be the offset as defined in the hierarchy section

– For the root object » The translation data will be the sum of the offset data and the

translation data from the motion section

• Rotation information– comes from the motion section

Motion Tools

Tools Bvhacker: http://davedub.co.uk/bvhacker/ Avimator: http://www.avimator.com/ Qavimator: http://www.qavimator.org/ Motion Capture Display and Editing: ASF/AMC viewer

Examples

Avimator1. The File Menu

File management, Program options, and Settings.

2. Figure: The Posing / Animation dummy (or Actor), standing on the stage: This is our main click and drag edit element.

3. Frame Slider Bar: the control to move between frames.

4. Edit Part Menu: An alternative to click select. A Menu to pick a particular body part to be edited.

5. Rotational Sliders: Fine controls to edit the X,Y,Z coordinates of selected body part.

6. Play Button / Keyframe Button: Preview plays animation / Sets the “Keyframe” of an animation.

7. Frame Quantity edit box: This is the control to edit the number of frames in an animation.

Examples : Avimator

Step 1 Open “Relaxed.bvh” Change the number of frames to “90”

Step2 choosing male or female and turning off the joint limits.

Step3 Keyframe Animation

• in between frames (=Keyframe) Step4

Move to frame “2” and click the “key frame” button. Setup our Actor to the starting position of the salute.1. Shift Double-Click rShldr X=-4 Y=-4 Z=822. Shift Double-Click rForeArm X=16 Y=9 Z=03. Shift Double-Click rThigh X=0 Y=1 Z=04. Shift Double-Click rShin X=1 Y=1 Z=05. Shift Double-Click rFoot X=0 Y=-3 Z=-4

Examples : Avimator

Step 5 Move to frame “30”. 1. Select rCollar X=0 Y=-52 Z=-232. Select rShldr X=23 Y=106 Z=823. Select rForeArm X=-12 Y=99 Z=-9

Step 6 Frame “30” COPY FRAME. move to frame “80” PASTE FRAME click the “Key Frame” button

Step 7 Frame “2” COPY FRAME. move to frame “90” PASTE FRAME click the “Key Frame” button

Step 8 play our animation our actor (or actress) should be repeatedly saluting us.

Step 9 save our file as before with FILE and SAVE. “salute_anim.bvh”