1. ©2003-2010, The Ohio State University Motion Capture History, Pipeline, and Areas of Interest Advanced Computing Center for the Arts and Design Ohio State University

2. ©2010, The Ohio State University Motion Capture Motion capture (mocap) is sampling and recording motion of humans, animals and inanimate objects as 3d data for analysis, playback and remapping.

3. ©2010, The Ohio State University History of Motion Capture Eadweard Muybridge (1830-1904) Etienne-Jules Marey (1830-1904) Harold Edgerton (1903-1990)

4. ©2010, The Ohio State University Eadward Muybridge The Flying Horse Zoopraxiscope Animal and Human Locomotion © Kingston Museum

5. ©2010, The Ohio State University Etienne-Jules Marey First person to analyze human and animal motion with video Created chronophotographic gun and fixed plate camera

6. ©2010, The Ohio State University Harold Edgerton © Palm Press Inc.

7. ©2010, The Ohio State University Rotoscope Allowed animators to trace cartoon characters over photographed frames of live performances. Invented in 1915 by Max Fleischer Koko the Clown Snow White

8. ©2010, The Ohio State University 1980’s Military and medical research purposes First computer graphics use in research labs First production use o Brilliance by Robert Abel (1985 Superbowl ad) o http://citinite.wordpress.com/2007/12/28/robert-abel- associates/ http://citinite.wordpress.com/2007/12/28/robert-abel- associates/ o Waldo C. Graphic (1988) PDI for Jim Henson tour o Mike the Talking Head (Siggraph 88) o http://mambo.ucsc.edu/psl/mike.html http://mambo.ucsc.edu/psl/mike.html o Don’t Touch Me (1989) o http://www.poetv.com/video.php?vid=15102 http://www.poetv.com/video.php?vid=15102

9. ©2010, The Ohio State University Electromechanical Optical fiber Strobing LED Acoustic Optical Acoustic Motion Capture Systems

10. ©2010, The Ohio State University Electromechanical suits Linked structures Potentiometers determine rotation of each link No occlusion No magnetic or electrical interference Large capture range, portable No global translation Restricted movement Fixed configuration of sensors Low sampling rate Inaccurate joints http://www.metamotion.com/gypsy/gypsy-motion-capture-system-mocap.htm

11. ©2010, The Ohio State University Optical fiber system Fiber-optic sensor Bend and twist sensors measure transmitted light No occlusion Multiple subjects Portable Adjustment to individual proportions is limited Less accurate data http://www.finger-motion-capture.com/

12. ©2010, The Ohio State University Electromagnetic systems Electromagnetic sensors placed on joints of moving object Measures orientation and position of sensor relative to electromagnetic field generated by transmitter No occlusion, multiple subjects Electromagnetic interference, small volume http://www.ascension-tech.com/

13. ©2010, The Ohio State University Acoustic system Set of microphones receive sound wave Other sounds in frequency range can disrupt capture Accuracy not as high as other systems

14. ©2010, The Ohio State University Strobing LED marker system Fewer Occlusion Problems Limited volume http://www.inition.co.uk/inition/product.php?URL_=product_mocaptrack_ascension_reactor2&SubCatID_=59

15. ©2010, The Ohio State University Optical motion capture system Reflective markers Multiple cameras digitize different views of performance High speed capture High data accuracy Occlusion Position sensitive cameras Limited volume http://www.vicon.com

16. Video-based Motion Analysis Research problem for 20+ years Difficulties o Complex varying environment o Segmentation issues o Occlusion ©2010, The Ohio State University

17. Video-based Motion Analysis Research Areas o Tracking o 3D surface reconstruction (surface scanning) o 3D movement reconstruction (video-based motion capture) o Recognition Challenges: o Complex environment variability o Body segmentation o Occlusion

18. ©2010, The Ohio State University 3D Reconstruction Single camera o Model assumptions required Multiple cameras o Require at least 2 cameras, unique with 3 o Camera calibration Motion capture with markers o Use retroreflective markers to simplify video information

19. ©2010, The Ohio State University Optical Mocap Process Calibration o Specifying how each camera is related to the others in reference to origin Tracking o Tracking of points…acceleration and identification problems Data o 3D positional…needs to be converted to rotational data

20. ©2010, The Ohio State University Problems Related to Marker Occlusion

21. ©2010, The Ohio State University Skeletal Fit (remapping mocap data to a character model) How to make markers move a skeleton o Photo reference of performer o CG model o Motion Builder o Vicon Blade o Other methods Problems with detecting joint centers… Organization of joint hierarchies

22. ©2009, The Ohio State University Motion Capture Pipeline Planning Shooting (Vicon IQ) Data processing (ViconIQ, MotionBuilder) Skeleton creation (Maya) Mapping to characters (MotionBuilder) Binding skeleton to a model (Maya, MotionBuilder)

23. ©2010, The Ohio State University Planning Character/Prop setup o Target skeleton/character topology o Ready stance considerations Marker setup o Heavy marker redundancy o Three markers per segment o Place markers close to bone

24. ©2009, The Ohio State University Planning Shot list o Number of characters involved in performance o Performance space dimensions o Interactions in shot o Shots to be blended or looped o Length of shots o Size and location of props o Gross proportional differences for retargeting

25. ©2009, The Ohio State University Planning Technical Issues o Output format o File naming conventions o Frames per second o Target software platform o Database management

26. ©2010, The Ohio State University Other Areas of Interest Re-use of data Generating 3d geometry Scene tracking with motion capture http://accad.osu.edu/~vberezin/bm/lp_anim.html

27. ©2010, The Ohio State University New Optical Technology 16 megapixel cameras ( T-series Vicon and Motion Analysis), 2000 fps 10 bit greyscale – Vicon (better centroid calculation)

28. ©2010, The Ohio State University Applications Medicine Mixed Reality Rehabilitation http://ame2.asu.edu/projects/mrrehab/ http://ame2.asu.edu/projects/mrrehab/ NIH Clinical Movement Analysis Lab http://pdb.cc.nih.gov/ http://pdb.cc.nih.gov/ Military http://www.motionrealityinc.com/ http://www.motionrealityinc.com/ Sports http://www.highwaygames.com/products/view.php?id=2807 http://www.highwaygames.com/products/view.php?id=2807 Video Games http://www.spike.com/video/nba-live-08-mo-cap/2897719 http://www.spike.com/video/nba-live-08-mo-cap/2897719 http://vimeo.com/1096600

29. ©2010, The Ohio State University Other uses Cognitive Psychology http://www.mcgill.ca/spl/projects/motion/ http://www.mcgill.ca/spl/projects/motion/ Historical Preservation http://accad.osu.edu/research/mocap_htmls/NAD.htm Art http://www.openendedgroup.com/index.php/artworks/

30. ©2009, The Ohio State University Feature Films Avatar Curious Case of Benjamin Button Pirates of the Carribean The Incredible Hulk Lord of the Rings Polar Express Matrix Titanic

31. ©2009, The Ohio State University Video Games Kinect Move Wii

32. ©2010, The Ohio State University Future Univesity of South California (Paul Debevec) NYU (Chris Briegler) Stanford University (Vlad Koltun) Carnegie Mellon (Jessica Hodgins) Max Planck Center Organic Motion Mova Image Metrics Captive Motion

33. ©2010, The Ohio State University References 1.Menache, Alberto, “Understanding Motion Capture for Computer Animation and Video Games” 2.http://www.kingston.ac.uk/Muybridge/http://www.kingston.ac.uk/Muybridge/ 3.http://www.anotherscene.com/cinema/firsts/marey.html 4.http://cmp1.ucr.edu/exhibitions/edgerton/edgerton.html 5.Siggraph 2001 Course 51: