Upload
penelope-june-warren
View
240
Download
0
Tags:
Embed Size (px)
Citation preview
1
SS5305 – Advanced Motion Capture
2
SS5305 – Advanced Motion Capture • 3D motion capture for entertainment
• Data capturing using a motion capture system
• Pre-processing of 3D motion capture data
• Analysis of processed motion data
• Motion regeneration of entertainment (traditional dancing)
Assessment: Examination: NONE Course Work: 100%The course work comprises of the following:-- 2 Class Tests – (20% each)- Project Assignment – (60%)
3
An Introduction to Advanced Motion Capture
()
()
()
()
DCM
4
Objectives
• History of Motion Capture• Basic Motion Capture System using Digital
Video Cameras• Optical/Video Motion Capture Systems• Introduction to Qualisys• Motion Capture Trends
5
Motion Capture System using Digital Video Cameras
6
Camera Set up
SMN Arosha Senanayake SS4301 - Software Engineering
• Focus, exposure set manually• Shutter speed set to 1/1000• Connect camera to laptop via DV cable and PCMCIA port
7
Calibration of Motion Capture System
SMN Arosha Senanayake SS4301 - Software Engineering
8
Marker Placements and Capturing
• Placed to digitized the points for later analysis• Arm and Leg are computed as angles in the s/w
Shoulder
Wrist
Knee
Toe
Elbow
Hip
Ankle
Heel
9
Stick Motion Regeneration
Hardward for Motion Capture
1. Passive optical
2. Active optical
3. Time modulated active
4. Markerless
5. Non-optical: mechnical, magnetic
Passive Optical Capture
Reflectors are placed on the body.
Advantages:
1. High resolution (sub-pixel)2. Works in ambient light3. No wires or electronics!
Disadvantages:
1. Occlusion – objects may block cameras2. Marker identity – how does the camera tell which marker is which?3. Requires rest of body to be blocked out (another color) 4. Variable lighting is problematic
In common setups 6 to 24 cameras are used to avoid occlusion.
Active Optical Capture
LED lights used as markers.
Advantages:
1. Works in the most lighting conditions – including in dark. 2. Similar resolution as passive
Disadvantages:
1. Power must be supplied to LEDs – wires on the suit2. Occlusion, Marker identity and Error are still the main issues.
Marker swap problem – Caused by lack of marker identity Identity of markers may switch when nearby.
1 1
2 2
1
12
2
Time Modulated Active Markers
Strobe each LED light, one at a time, at the same frame rate as the camera. Camera sees only one marker at a time.
Advantages:
1. Solve marker identity 2. Data recorded is much cleaner and higher resolution
Disadvantages:
1. Harder to implement – requires radio signal to transmit camera frame sync to the LEDs2. LEDs must have additional hardware to determine order of strobe.3. Frame rate is divided by the number of markers. Factual = F camera / N markers
Cost: $50,000 for 8 camera, 1 actor systems
Markerless Systems
Most markerless systems use structured light.A projector or emitter casts out light in a coherent pattern, such as vertical bars.
Advantages:
1. No markers needed on the body2. Can record depth information with one camera
Disadvantages:
1. Most systems record point clouds. The do not record marker positions. 2. Addition steps are needed to fit a skeleton inside the point cloud.3. Occlusion is still a problem4. Frame rate depends more on processing time for data fit than on camera rate.
Motion Capture Workflow
1.
2.
3.
4. 5. 6. 7.
automated by most mocap systems
2007-01-24
Qualisys Quality in Motion
2007-01-24
OQUS camera
2007-01-24
Detection of markers – 2D and 3D
Camera 1 Camera 2
• Oqus 5+ 4 MP @180fps
• Oqus 1 0.3MP @ 250fps
• Oqus 210c 2 MP @ 360fps (Color High Speed Video)
• Oqus 4 3 MP @ 480fps
• Oqus 3+ 1.3 MP @ 500fps 0.3 MP @ 1750fps
• Oqus7 12MP@120fps 3MP@300fps
Oqus models:
•Oqus 5+ series 4.0Mpixel @ 180fps
•Oqus 4 series 3.0Mpixel @ 480fps
•Oqus 3+ series 1.3Mpixel @ 500fps
•Oqus 2 series 2Mpixel @ 360fps
•Oqus 1 series 0.3Mpixel @ 250fpsFocus 5mAperture 2.8Exposure 500usThreshold 200
Integrated WiFi• Run system off a laptop with no cable between the first camera and the laptop.
Suitable for outdoors – Oqus 3+ & Oqus 4• Active filtering technology enables problem free measurements in direct sunlight.
High-speed video mode• The only camera on the market to combine motion capture with high-speed video in the same body! Capture 1.3Mpixel full-frame video @ 500fps• NEW! Stream High Speed Video data to your PC with Oqus 2 series
Oqus210c – Color High-speed Video
• High Speed Video only
• Color image
• Can be calibrated with the rest of the system
• Up to 340 fps in Buffer mode (full frame, full resolution)
• Real-time color video streaming at 100fps (0,5 Megapixel)
• Real-time color video overlay
2007-01-24
Qualisys as a mobile system
Daisy-chain connectivity: Quicker set-up, shorter cable lengths (standard – 15 meters) for larger covered volume
Bundled data/power cables: Only 8 cables and 2 power supplies for 8 cameras
On-board 2D digitizing: No hub needed, run it off a laptop
Camera and cable cases: Standard light-weight Pelican cases, 3 cases for a complete 8 camera system.
WiFi communication: Between the first camera system and PC
Colfibre tripods: Light-weight with enough height (2m).
System connectivity:
1. Direct connection between the first camera and PC No hubs needed
2. Single 120W power supply for 4 Oqus cameras Minimal number of cables from wall outlets
3. Bundled data and power cable Only 8 cables and 2 power supplies for an 8 camera system
4. Daisy-chain connection (from camera to camera)Shorter cable lengths even with larger volumesQuicker setup
Easy to relocate – Pack and go
5. Standard 100Mbit Ethernet Up to 100m between the
cameras (if ever needed)
Major components of Qualisys system:
Oqus - Motion Capture Camera• Tracking movements in 3D with infrared technology
Data processing, analysis and visualization:•Visual3D and Motion Monitor - biomechanical analysis and report generation•Motion Builder, 3DStudioMax, Maya – RT visualization and animation production•Matlab, LabView, Excel - complex mathematic calculations.
QTM – Qualisys Track Manager•Calibration and 3D tracking •Data collection and cleaning•Automatic identification of markers•Synchronization with external equipment (force plates, EMG etc.)•3D Data real-time streaming (to Visual3D, Motion Builder, Matlab, LabView)•Data export to various formats (c3d, tsv, avi etc)
iPhone & iPod touch applications:
In December 2011 Qualisys proudly introduced two new free iPhone applications
The view finder allows you to see individual camera views easing the setup of large volumes.
QTM remote allows users to start andstop measurements remotely as well as add event markers.
Avater (2010),James Cameron
Solution:
Block off the face using individual, head-mounted cameras, which record only the face. Use motion cameras and passive markers for the body.Allows for both large volumes and small details.
Trend –Performance captureis a collection of techniques that combine torecord the totalmotion of an actor.
Markers include:Body capture Green lines, white dotsFacial capture Head-mounted device, /w camera boomsHair capture Blue and red ropes
Trend – Markerless capture:Origins in 3D laser scanning
3D Lego Digitizer http://www.rchoetzlein.com/project/digitizer/
Trend – Markerless capture: Structured Light
Faster: Do all lines at once
Projector with structured light mapped onto the object. Use two cameras to determine object structure.
Structured light can be linear, binary coded, gray coded, or color coded. The encoding allows you to uniquely identify points.
Light may be infrared (Kinetic).
Q: High frequency gives details about height of point. But how do we tell if the point is on left or right side of obj?
A: Low frequency gives overall characteristics of pixels.
No markers.
Structured light creates a point cloud.
Skeleton is fit inside point cloud from root joints to extremities.Torso defines primary orientation,and also constraints placement of next joint layer in hierarchy.
Volume construction
Pointcloud
Fittorse
Fitextremities
Trend – Markerless capture: Direct-to-3D models
http://www.youtube.com/watch?v=dTisU4dibSc&playnext=1&list=PLD31C3C36D294EEDB
Christian Theobalt, Stanford University
http://www.stanford.edu/group/biomotion/Markerless.html
Performance Capture from Sparse Multi-view Video, SIGGRAPH 2010
Trend – Monocular capture
Fabio Remondino, Andreas RoditakisInstitute for Geodesy and Photogrammetry - ETH Zurich, Switzerland3D Reconstruction of Human Skeleton from Single Images or Monocular Video Sequences2003, 25th Pattern Recognition Symposium
One camera, without depth, is under-constrained.
However, the human body has fixed limb lengths and ratios.
Use the body ratios as an additional constraint.
Trend – Low Cost Systems
Cheap hardware: Microsoft Kinect, Web cameras.
Open source software: OpenKinect open kinect drivers libfreenect open kinect drivers
OpenNI skeleton fittingFaceAPI facial tracking
Main challenges: 1) Integration into existing frameworks,2) Usually requires programming experience3) Can be difficult to modify for research