Three-dimensional Three-dimensional Motion Capture, Motion Capture, Modelling and Modelling and Analysis of Analysis of Ski JumpersSki Jumpers

Atle NesAtle NesCSGSC 2005 CSGSC 2005

Trondheim, April 28thTrondheim, April 28th

OverviewOverview

1.1. Project descriptionProject description

2.2. What kind of data are we interested in?What kind of data are we interested in?

3.3. Capturing data:Capturing data: Image acquisition, Camera systemImage acquisition, Camera system

4.4. Processing data:Processing data: Feature points, Motion capture, PhotogrammetryFeature points, Motion capture, Photogrammetry

5.5. Interpreting data:Interpreting data: Visualization, Motion analysisVisualization, Motion analysis

6.6. ConclusionConclusion

Project descriptionProject description

• Task:Task: Design a computer Design a computer system that can capture system that can capture and study the motion of and study the motion of ski jumpers in 3D.ski jumpers in 3D.

• Goal:Goal: The results will be The results will be used to give feedback to used to give feedback to the ski jumpers that can the ski jumpers that can help them to increase help them to increase their jumping lengths.their jumping lengths.

Data collectionData collection

• Will be gathered and analyzed in close Will be gathered and analyzed in close cooperation with Human Movement cooperation with Human Movement Science Program at NTNU.Science Program at NTNU.

Data:Data:• Mainly from Mainly from outdoor ski jumpsoutdoor ski jumps captured at captured at

Granåsen ski jumping hill here in Granåsen ski jumping hill here in Trondheim.Trondheim.

• Also from Also from indoor ski jumpsindoor ski jumps captured at captured at Dragvoll sports facilities.Dragvoll sports facilities.

Granåsen ski jump arenaGranåsen ski jump arena

Image acquisitionImage acquisition

• Video sequences are captured Video sequences are captured simultanuously from multiple video simultanuously from multiple video cameras.cameras.

Two decisive camera factors:Two decisive camera factors:

• Spatial resolution (pixels) Spatial resolution (pixels)

• Time resolution (frame rate)Time resolution (frame rate)

Camera equipmentCamera equipment

• 3 x AVT Marlin F080b3 x AVT Marlin F080b• IEEE1394 Firewire, DCAMIEEE1394 Firewire, DCAM• 8-bit greyscale w/ max resolution 8-bit greyscale w/ max resolution

1024x768x15fps or 640x480x30fps1024x768x15fps or 640x480x30fps

• Extra trigger cable/signalExtra trigger cable/signal Video capture Video capture synchronization.synchronization.

• Different camera lensesDifferent camera lenses Capture the Capture the same area from different distances.same area from different distances.

• Optical fibreOptical fibre Extends the distance from Extends the distance from computer to cameras in the hill, keeping computer to cameras in the hill, keeping the transmission speed.the transmission speed.

Feature pointsFeature points

Robust feature points:Robust feature points:• Human body markers Human body markers

(easy detectable)(easy detectable)• Naturally robust features Naturally robust features

(more difficult).(more difficult).

• Want to have automatic Want to have automatic detection of robust detection of robust feature points using feature points using simple image processing simple image processing techniques.techniques.

Motion captureMotion capture

• Localizing, identifying and tracking Localizing, identifying and tracking identical feature points in both identical feature points in both sequences of video imagessequences of video images as well as as well as accross accross different camera viewsdifferent camera views..

• Synchronized video streams ensures Synchronized video streams ensures good 3D coordinate accuracy.good 3D coordinate accuracy.

Tracking w/ missing dataTracking w/ missing data

• Occluded featuresOccluded features Redundancy using multiple Redundancy using multiple cameras with different views.cameras with different views.

• Probability theoryProbability theory Guess the point position based on Guess the point position based on feature point velocity.feature point velocity.

• Another problem Another problem Blur effect Blur effect

??

PhotogrammetryPhotogrammetry

• Matching corresponding feature Matching corresponding feature points from two or more cameras points from two or more cameras allows us to calculate the exact allows us to calculate the exact position of that feature point in 3D.position of that feature point in 3D.

• Good camera placement is important Good camera placement is important for good triangulation capabilities for good triangulation capabilities (3D coordinate accuracy).(3D coordinate accuracy).

Camera calibrationCamera calibration

• Coordinate systemCoordinate system On site On site calibration using known coordinates in calibration using known coordinates in the ski jumping arena.the ski jumping arena.

• Direct Linear Transformation (DLT) by Direct Linear Transformation (DLT) by Abdel-Aziz and Karara in 1971.Abdel-Aziz and Karara in 1971.

• Lens distortion (unlinear)Lens distortion (unlinear)• Intelligent removal of the worst Intelligent removal of the worst

calibration points (sources of error).calibration points (sources of error).

VisualizationVisualization

• Feature point tracks are Feature point tracks are connected back onto a connected back onto a dynamic model of the ski dynamic model of the ski jumper.jumper.

• Dynamic model of ski Dynamic model of ski jumper is combined with jumper is combined with static model of ski jump static model of ski jump arena.arena.

Motion analysisMotion analysis

• Done in close cooperation with Done in close cooperation with Human Movement Science ProgramHuman Movement Science Program

• Extract movements that have greatest Extract movements that have greatest influence on the result.influence on the result.

• Using statistical tools and prior Using statistical tools and prior knowledge about movementsknowledge about movements

• Project some movements to unseen Project some movements to unseen 2D views.2D views.

Related applicationsRelated applications

Medical:Medical:

• Diagnosis of infant spontaneous Diagnosis of infant spontaneous movements for early detection of movements for early detection of possible brain damage (cerebral possible brain damage (cerebral palsy).palsy).

• Diagnosis of adult movements (walk), Diagnosis of adult movements (walk), for determination of cause of for determination of cause of problems.problems.

Related applications 2Related applications 2

Sports:Sports:

• Study top athletes for finding optimal Study top athletes for finding optimal movement patterns. movement patterns.

Surveillance:Surveillance:

• Crowd surveillance and identification Crowd surveillance and identification of possible strange behaviour in a of possible strange behaviour in a shopping mall or airport.shopping mall or airport.

ConclusionConclusion

• I have presented an overview of a I have presented an overview of a system that can capture, visualize system that can capture, visualize and analyze ski jumpers in a ski and analyze ski jumpers in a ski jumping hill.jumping hill.

• Remains to see how well such a Remains to see how well such a system can perform and if it can help system can perform and if it can help the ski jumpers improve their skills.the ski jumpers improve their skills.

Any questions?Any questions?