Three-dimensional Motion Capture, Modelling and Analysis of Ski Jumpers Atle Nes CSGSC 2005 Trondheim, April 28th

Overview 1.Project description 2.What kind of data are we interested in? 3.Capturing data: Image acquisition, Camera system 4.Processing data: Feature points, Motion capture, Photogrammetry 5.Interpreting data: Visualization, Motion analysis 6.Conclusion

Project description Task: Design a computer system that can capture and study the motion of ski jumpers in 3D.Task: Design a computer system that can capture and study the motion of ski jumpers in 3D. Goal: The results will be used to give feedback to the ski jumpers that can help them to increase their jumping lengths.Goal: The results will be used to give feedback to the ski jumpers that can help them to increase their jumping lengths.

Data collection Will be gathered and analyzed in close cooperation with Human Movement Science Program at NTNU.Will be gathered and analyzed in close cooperation with Human Movement Science Program at NTNU.Data: Mainly from outdoor ski jumps captured at Granåsen ski jumping hill here in Trondheim.Mainly from outdoor ski jumps captured at Granåsen ski jumping hill here in Trondheim. Also from indoor ski jumps captured at Dragvoll sports facilities.Also from indoor ski jumps captured at Dragvoll sports facilities.

Granåsen ski jump arena

Image acquisition Video sequences are captured simultanuously from multiple video cameras.Video sequences are captured simultanuously from multiple video cameras. Two decisive camera factors: Spatial resolution (pixels)Spatial resolution (pixels) Time resolution (frame rate)Time resolution (frame rate)

Camera equipment 3 x AVT Marlin F080b3 x AVT Marlin F080b IEEE1394 Firewire, DCAMIEEE1394 Firewire, DCAM 8-bit greyscale w/ max resolution 1024x768x15fps or 640x480x30fps8-bit greyscale w/ max resolution 1024x768x15fps or 640x480x30fps Extra trigger cable/signal  Video capture synchronization.Extra trigger cable/signal  Video capture synchronization. Different camera lenses  Capture the same area from different distances.Different camera lenses  Capture the same area from different distances. Optical fibre  Extends the distance from computer to cameras in the hill, keeping the transmission speed.Optical fibre  Extends the distance from computer to cameras in the hill, keeping the transmission speed.

Feature points Robust feature points: Human body markers (easy detectable)Human body markers (easy detectable) Naturally robust features (more difficult).Naturally robust features (more difficult). Want to have automatic detection of robust feature points using simple image processing techniques.Want to have automatic detection of robust feature points using simple image processing techniques.

Motion capture Localizing, identifying and tracking identical feature points in both sequences of video images as well as accross different camera views.Localizing, identifying and tracking identical feature points in both sequences of video images as well as accross different camera views. Synchronized video streams ensures good 3D coordinate accuracy.Synchronized video streams ensures good 3D coordinate accuracy.

Tracking w/ missing data Occluded features  Redundancy using multiple cameras with different views.Occluded features  Redundancy using multiple cameras with different views. Probability theory  Guess the point position based on feature point velocity.Probability theory  Guess the point position based on feature point velocity. Another problem  Blur effectAnother problem  Blur effect ?

Photogrammetry Matching corresponding feature points from two or more cameras allows us to calculate the exact position of that feature point in 3D.Matching corresponding feature points from two or more cameras allows us to calculate the exact position of that feature point in 3D. Good camera placement is important for good triangulation capabilities (3D coordinate accuracy).Good camera placement is important for good triangulation capabilities (3D coordinate accuracy).

Camera calibration Coordinate system  On site calibration using known coordinates in the ski jumping arena.Coordinate system  On site calibration using known coordinates in the ski jumping arena. Direct Linear Transformation (DLT) by Abdel-Aziz and Karara in 1971.Direct Linear Transformation (DLT) by Abdel-Aziz and Karara in Lens distortion (unlinear)Lens distortion (unlinear) Intelligent removal of the worst calibration points (sources of error).Intelligent removal of the worst calibration points (sources of error).

Visualization Feature point tracks are connected back onto a dynamic model of the ski jumper.Feature point tracks are connected back onto a dynamic model of the ski jumper. Dynamic model of ski jumper is combined with static model of ski jump arena.Dynamic model of ski jumper is combined with static model of ski jump arena.

Motion analysis Done in close cooperation with Human Movement Science ProgramDone in close cooperation with Human Movement Science Program Extract movements that have greatest influence on the result.Extract movements that have greatest influence on the result. Using statistical tools and prior knowledge about movementsUsing statistical tools and prior knowledge about movements Project some movements to unseen 2D views.Project some movements to unseen 2D views.

Related applications Medical: Diagnosis of infant spontaneous movements for early detection of possible brain damage (cerebral palsy).Diagnosis of infant spontaneous movements for early detection of possible brain damage (cerebral palsy). Diagnosis of adult movements (walk), for determination of cause of problems.Diagnosis of adult movements (walk), for determination of cause of problems.

Related applications 2 Sports: Study top athletes for finding optimal movement patterns.Study top athletes for finding optimal movement patterns.Surveillance: Crowd surveillance and identification of possible strange behaviour in a shopping mall or airport.Crowd surveillance and identification of possible strange behaviour in a shopping mall or airport.

Conclusion I have presented an overview of a system that can capture, visualize and analyze ski jumpers in a ski jumping hill.I have presented an overview of a system that can capture, visualize and analyze ski jumpers in a ski jumping hill. Remains to see how well such a system can perform and if it can help the ski jumpers improve their skills.Remains to see how well such a system can perform and if it can help the ski jumpers improve their skills.

Any questions?