1. SS5305 – Advanced Motion Capture
SMN Arosha Senanayake SS Software Engineering

2. SS5305 – Advanced Motion Capture
3D motion capture for entertainment
Data capturing using a motion capture system
Pre-processing of 3D motion capture data
Analysis of processed motion data
Motion regeneration of entertainment (traditional dancing)
Assessment:
Examination: NONE
Course Work: 100%
The course work comprises of the following:-
-          2 Class Tests – (20% each)
-          Project Assignment – (60%)
SMN Arosha Senanayake SS Software Engineering

3. An Introduction to Advanced Motion Capture
()
()
()
()
DCM
SMN Arosha Senanayake SS Software Engineering

4. Objectives History of Motion Capture
Basic Motion Capture System using Digital Video Cameras
Optical/Video Motion Capture Systems
Introduction to Qualisys
Motion Capture Trends

5. Motion Capture System using Digital Video Cameras

6. SMN Arosha Senanayake SS4301 - Software Engineering
Camera Set up
Focus, exposure set manually
Shutter speed set to 1/1000
Connect camera to laptop via DV cable and PCMCIA port
SMN Arosha Senanayake SS Software Engineering
SMN Arosha Senanayake SS Software Engineering

7. Calibration of Motion Capture System
SMN Arosha Senanayake SS Software Engineering
SMN Arosha Senanayake SS Software Engineering

8. Marker Placements and Capturing
Placed to digitized the points for later analysis
Arm and Leg are computed as angles in the s/w
Elbow
Hip
Ankle
Heel
Shoulder
Wrist
Knee
Toe

9. Stick Motion Regeneration

10. Hardward for Motion Capture
1. Passive optical 2. Active optical 3. Time modulated active 4. Markerless 5. Non-optical: mechnical, magnetic

11. Passive Optical Capture
Reflectors are placed on the body.
Advantages: 1. High resolution (sub-pixel) 2. Works in ambient light 3. No wires or electronics! Disadvantages: 1. Occlusion – objects may block cameras 2. Marker identity – how does the camera tell which marker is which? 3. Requires rest of body to be blocked out (another color) 4. Variable lighting is problematic
In common setups 6 to 24 cameras are used to avoid occlusion.

12. Active Optical Capture
LED lights used as markers.
Advantages: 1. Works in the most lighting conditions – including in dark. 2. Similar resolution as passive Disadvantages: 1. Power must be supplied to LEDs – wires on the suit 2. Occlusion, Marker identity and Error are still the main issues.
Marker swap problem – Caused by lack of marker identity Identity of markers may switch when nearby. 1 1 1 2 2 2 2 1

13. Time Modulated Active Markers
Strobe each LED light, one at a time, at the same frame rate as the camera. Camera sees only one marker at a time. Advantages: 1. Solve marker identity 2. Data recorded is much cleaner and higher resolution Disadvantages: 1. Harder to implement – requires radio signal to transmit camera frame sync to the LEDs 2. LEDs must have additional hardware to determine order of strobe. 3. Frame rate is divided by the number of markers Factual = F camera / N markers Cost: $50,000 for 8 camera, 1 actor systems

14. Markerless Systems
Most markerless systems use structured light. A projector or emitter casts out light in a coherent pattern, such as vertical bars. Advantages: 1. No markers needed on the body 2. Can record depth information with one camera Disadvantages: 1. Most systems record point clouds. The do not record marker positions. 2. Addition steps are needed to fit a skeleton inside the point cloud. 3. Occlusion is still a problem 4. Frame rate depends more on processing time for data fit than on camera rate.

15. Motion Capture Workflow1. 3. 2. 4. 5. 6. 7.
automated by most mocap systems

16. Qualisys
Quality in Motion

17. Obs kör video

18. OQUS camera
Each camera records 2D coordinates of visible markers and outputs them in real time to QTM software.

19. Obs kör video

20. Detection of markers – 2D and 3D
Camera Camera 2
Obs kör video

21. Oqus7 12MP@120fps 3MP@300fps Oqus 4 3 MP @ 480fps
Oqus 210c fps
(Color High Speed Video)
Oqus7

22. Oqus models: Oqus 5+ series 4.0Mpixel @ 180fps
Focus 5m
Aperture 2.8
Exposure 500us
Threshold 200
High-speed video mode
The only camera on the market to combine motion capture with high-speed video in the same body! Capture 1.3Mpixel full-frame 500fps
NEW! Stream High Speed Video data to your PC with Oqus 2 series
Suitable for outdoors – Oqus 3+ & Oqus 4
Active filtering technology enables problem free measurements in direct sunlight.
Integrated WiFi
Run system off a laptop with no cable between the first camera and the laptop.

23. Oqus210c – Color High-speed Video
High Speed Video only
Color image
Can be calibrated with the rest of the system
Up to 340 fps in Buffer mode (full frame, full resolution)
Real-time color video streaming at 100fps (0,5 Megapixel)
Real-time color video overlay

24. Obs kör video

25. Qualisys as a mobile system
Daisy-chain connectivity: Quicker set-up, shorter cable lengths (standard – 15 meters) for larger covered volume
Bundled data/power cables: Only 8 cables and 2 power supplies for 8 cameras
On-board 2D digitizing: No hub needed, run it off a laptop
Camera and cable cases: Standard light-weight Pelican cases, 3 cases for a complete 8 camera system.
WiFi communication: Between the first camera system and PC
Colfibre tripods: Light-weight with enough height (2m).

26. System connectivity:
Direct connection between the first camera and PC No hubs needed
Single 120W power supply for 4 Oqus cameras Minimal number of cables from wall outlets
Bundled data and power cable Only 8 cables and 2 power supplies for an 8 camera system
Daisy-chain connection (from camera to camera)
Shorter cable lengths even with larger volumes
Quicker setup
Easy to relocate – Pack and go
5. Standard 100Mbit Ethernet Up to 100m between the cameras (if ever needed)

27. Major components of Qualisys system:
Oqus - Motion Capture Camera
Tracking movements in 3D with infrared technology
QTM – Qualisys Track Manager
Calibration and 3D tracking
Data collection and cleaning
Automatic identification of markers
Synchronization with external equipment (force plates, EMG etc.)
3D Data real-time streaming (to Visual3D, Motion Builder, Matlab, LabView)
Data export to various formats (c3d, tsv, avi etc)
Data processing, analysis and visualization:
Visual3D and Motion Monitor - biomechanical analysis and report generation
Motion Builder, 3DStudioMax, Maya – RT visualization and animation production
Matlab, LabView, Excel - complex mathematic calculations.

28. iPhone & iPod touch applications:
In December 2011 Qualisys proudly introduced two new free iPhone applications
The view finder allows you to see individual camera views easing the setup of large volumes.
QTM remote allows users to start and
stop measurements remotely as well as add event markers.

29. Trend – Performance capture is a collection of
techniques that combine to record the total motion of an actor.
Avater (2010), James Cameron
Solution: Block off the face using individual, head-mounted cameras, which record only the face. Use motion cameras and passive markers for the body. Allows for both large volumes and small details.

30. Markers include: Body capture Green lines, white dots Facial capture Head-mounted device, /w camera booms Hair capture Blue and red ropes

31. Trend – Markerless capture: Origins in 3D laser scanning
3D Lego Digitizer

32. Trend – Markerless capture: Structured Light
Q: High frequency gives details about height of point. But how do we tell if the point is on left or right side of obj?
Faster: Do all lines at once Projector with structured light mapped onto the object. Use two cameras to determine object structure. Structured light can be linear, binary coded, gray coded, or color coded. The encoding allows you to uniquely identify points. Light may be infrared (Kinetic).
A: Low frequency gives overall characteristics of pixels.

33. Point cloud
Volume construction
Fit torse
Fit extremities
No markers. Structured light creates a point cloud. Skeleton is fit inside point cloud from root joints to extremities. Torso defines primary orientation, and also constraints placement of next joint layer in hierarchy.

34. Trend – Markerless capture: Direct-to-3D models
Performance Capture from Sparse Multi-view Video, SIGGRAPH 2010
Christian Theobalt, Stanford University

35. Trend – Monocular capture
One camera, without depth, is under-constrained. However, the human body has fixed limb lengths and ratios. Use the body ratios as an additional constraint.
Fabio Remondino, Andreas Roditakis
Institute for Geodesy and Photogrammetry - ETH Zurich, Switzerland
3D Reconstruction of Human Skeleton from Single Images or Monocular Video Sequences 2003, 25th Pattern Recognition Symposium

36. Trend – Low Cost Systems
Cheap hardware: Microsoft Kinect, Web cameras. Open source software: OpenKinect open kinect drivers libfreenect open kinect drivers OpenNI skeleton fitting FaceAPI facial tracking Main challenges: 1) Integration into existing frameworks, 2) Usually requires programming experience 3) Can be difficult to modify for research