Upload
fakegreendress
View
375
Download
0
Embed Size (px)
Citation preview
DIY Motion Capture
Kinect Motion Capture for Filmmaking3/23/13
Why DIY MoCap?Motion capture is now cheap and accessibleNo longer need a giant studioIts FAST.Ideal use cases: short-turnaround animation, rapid prototyping etc.1
How does the Kinect work?
Photo by y3rdua
Getting Started: HardwareYou will need:A Kinect, or generic equivalent such as the Asus XtionUSB/AC adapter cable, if you have a Kinect that was bundled with the Xbox (dont buy the Microsoft adapter, the cheap ones work fine)Enough space to perform
Getting Started: Software
OpenNI driver bundle:simple-openni.googlecode.com
KinectToPin:kinecttopin.com
Microsoft SDK vs. OpenNI4
Two competing SDKsOpenNIFree and open sourceWorks with all models of Kinect as well as other devices such as the Asus Xtion.Cross platform even supports ARM processors and Android
Microsoft (official) SDKMore tracking pointsEstimates occluded joints and hand/foot rotationSpecial case capture modes (seated figure etc.)Only works with Kinect For Windows on Windows
KinectToPinKinect Motion Capture for After Effects
6
What is KinectToPin?
A standalone motion capture applicationA UI panel for After Effects+
Rig 2D character models in 3D
Working BackwardsTraditional AnimationStart with scene already storyboarded and timed outCreate sketches of key poses for each characterAdd in-between drawings for movementColor and composite characters in final sceneMotion Capture WorkflowStart with full color, full-motion charactersRefine blocking and movement; add facial animationChoose shots and timing of edit, add camera moves(Optional) Re-record any performances that arent working
Working Backwards
Think of it like working with raw footage.
Character Design2D models in 3D space
Why multi-layered characters?Single-layer model:
Multi-layered:
Design Tips
Break your model at every joint.Build in overlap so joints have room to rotate without disconnecting.Parts that fall in front of and behind another layer (e.g. the hair of the model at left) should be two layers.The hands, feet and head can each have multiple elements.
CutoutsRaster-based characters drawn or cut out of existing images or photographs.
Work big if you want to be able to zoom in on your charactersWhen cutting out layers, use Refine Edge to smooth your edges and clean up any extra junk that got selectedClone/heal/paint in portions that will be missing or problematic when the model changes pose
Vector-Based CharactersDrawn with math, not pixels. Easier to break into layers but watch out.
Can be built in Illustrator, Photoshop or with AE Shape LayersVector-based characters are not infinitely scalable as youd expect: rigging with AEs Puppet Tool generates a mesh at a set resolution.Building puppets with vector layers adds an extra step: they need to be precomposed before rigging.
Motion CaptureRecording with KinectToPin
Setting up your space to record
6 10 feet
17
CapturingRecord 15-point 3D skeletal tracking dataCapture in sync to dialogueRemote control with a standard presentation remoteOpen source and cross-platform: runs on OpenNI, built in Processing, also works with generic sensors like the Asus Xtion
Acting for KinectThe Kinect cant record what it cant see. Keep your entire body in the cameras rangeFace the camera when capturingWatch out for occluded joints (sitting down, putting your hands behind your back etc. can cause it to lose the track)Its surprisingly good at picking up subtle posture shifts, but not smaller motions like laughter. Use a different performer for each character if possible
Data formats for exportKinectToPin can output quite a few data, but the ones well be using today are:XML (Native recording format, no export required)After Effects 3D Point Controls (CS5.5+)After Effects 2D Point Controls (CS4+)
Tip: Edit KinectToPins settings.txt file to choose your output formats.
Lunch.
Importing tracking dataNative import or copy & paste
Meet the KinectToPin UI Panel.Create character templates2D and 3D setupsNative XML import (SLOW)Automatically add expressions to smooth tracking dataAutomate rigging
The Puppet ToolTHREE OR MORE PINS ON ONE LAYER: Elbows dont bend this way. Ow.
TWO PINS EACH, MULTIPLE LAYERS:Rotation with stretch and squash!
AE Puppet best practicesWork BIG. Start high-res, scale down in the puppet comp, then scale back up in the project comp (enable Collapse Transformations).Precompose if youre using vectors or shape layersKeep multiple angles of the same character in syncEnd big.Start big.
Character RiggingUsing the KinectToPin UI panel
Retargeting for arbitrary character shapes
Can add multiple motion tracks and duplicate nullsPin names need to match control nulls
28
Its easy!Just grab a red control null and move it to the appropriate position.
Quick auto-lip syncSelect audio layer and choose Convert Audio to Keyframes.
Quick auto-lip sync2. Alt-click the stopwatch to add an expression. Use the pickwhip to connect the audio keyframes to the mouth animation.
Quick auto-lip syncSize the audio data to fit. Use this expression:
audio = [your audio keyframes];linear(audio,sourceMin,sourceMax,targetMin,targetMax)Lowest value of source dataClosed mouth (probably 0)Highest value of source dataWidest possible open mouth
Embrace the limitations of the format.One last thought:
2D vs. 3D: Avoiding the Uncanny ValleyAwesome glitches
33
Further non-K2P toolsFREEBrekel KinectOpen-source capture for 3D animation (Windows-only)www.brekel.com
DuikInverse kinematics plugin for AEwww.duduf.netCOMMERCIALiPi SoftUses two Kinects to recordwww.ipisoft.com
NI MateKinect for OSC/MIDI/Blenderwww.ni-mate.com
34
Get in TouchVictoria Necewww.victorianece.com@FakeGreenDress