1. Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject recognition. 3 categories of objectives

2. Rick Parent - CIS681 Motion Tracking Identification and location of subject characteristics Based on features or regions Optical flow silhouettes Applications Pedestrian flow traffic flow.

3. Rick Parent - CIS681 Pose Reconstruction 2D or 3D tracking of body parts Usually with geometric model DoFs? Silhouette Color, Textures What? Entire pose Hand & arm gestures Facial expression & lips Applications Medical - gait analysis Sports Entertainment With instrumentation – MoCap.

4. Rick Parent - CIS681 Recognition Extract discriminators Neural Nets PCA Abstract out the “what” high level semantics Can be based on tracking or pose reconstruction What? Gestures - by arm motion Person - by his/her gait American Sign Language (ASL) - by finger position.

5. Rick Parent - CIS681 Problems. Complex, varying environment Occlusion of body parts. Segmentation issues

6. Rick Parent - CIS681 Possible simplifying assumptions on environment Static and/or uniform background Restrict number of objects Restriction on movement Restrict complexity of objects.

7. Rick Parent - CIS681 Segmentation problems Image quality Low contrast images Motion blur Strong shadows, reflections Loose fitting clothes - use tight fit, markers.

8. Rick Parent - CIS681 Occlusion problems 3D model, 2D image Underconstrained problems Use multiple cameras.

9. Rick Parent - CIS681 Taxonomy Static v. dynamic processing e.g. use nOptical flow 2D v. 3D Model based or features Edges Regions: colors, textures Single camera v. multi-camera Locate points uniquely Single: underconstrained

10. Rick Parent - CIS681 Processing Variations Background subtraction Contract enhancement Shadow removal Edge detection Region filling Clustering Silhouette formation Dynamic update of background Changing illumination Shadows, brightness Update of static fixtures Body part matching Find appendages, joints Toe - heel curvature Head Speed Real-time only previous frames Interactive Batch Need previous examples? Need initialization? Single frame or temporal info.

11. Rick Parent - CIS681 Cambridge Research lab

12. Rick Parent - CIS681 Cambridge Research lab

13. Rick Parent - CIS681 Cambridge Research lab

14. Rick Parent - CIS681 3D Model Optimization Live action Synthetic scene adjust compare project

15. Rick Parent - CIS681 3D Model Optimization Matching features of synthetic image with live image Temporal coherence Anatomy Area matching Boundary matching

16. Rick Parent - CIS681 Gavrila Multiple cameras Detect edges - remove static ones

17. Rick Parent - CIS681 Gavrila

18. Rick Parent - CIS681 Gavrila

19. Rick Parent - CIS681 MERL Capture motion snippets Baysian formulation

20. Rick Parent - CIS681 Sminchisescu

21. Rick Parent - CIS681 Sminchisescu

22. Rick Parent - CIS681 Sminchisescu

23. Rick Parent - CIS681 Sminchisescu