1. Motion Pictures and Mathematics Chris Bregler Courant Institute New York University

2. Turing Test: Artificial Intelligence Cogs: Rodney Brooks Simulate Human

3. Turing Test: Computer Graphics Create Illusion On Screen Final Fantasy

4. Mathematical Models Lighting Skin Hair Water Cloth Primary Human Motion ?? (Simulate Brain) (Spider Man II, Debevec et al) (Van Jensen et al) (ILM, Fedkiw et al)

5. Two leading techniques for Animation Performance-Capture: Mocap / Rotoscoping Traditional: Hand-Animated Pixar / Disney Electronic Arts

6. NYU Motion Capture Lab

8. “Satan’s Rotoscope” Controversy Performance-Capture: Mocap / Rotoscoping Traditional: Hand-Animated Pixar / Disney Electronic Arts

9. Disney: Expressive Motion

10. Motion Reference

11. Rotoscope / Mocap: History Disney: Helen Stanley  Sleeping Beauty

12. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

13. Motion Model: Kinematic Chain

14. Motion Capture = Math. Equation 10 = a * 5

15. Optical Flow Kinematic Chains M  * = [v1,v2,…,vn]

16. Motion Capture = Least Squares Sol. M  * = ? [v1,v2,…,vn]

17. Eadweard Muybridge

18. Tracking Kinematic Chains

19. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

20. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

21. Humans have no fixed axis The Digital Human Project Walt Hyneman, Sony / Disney Stuart Sumida, Cal State San Bernardino Elizabeth Rega, Western Univ. of Health Science, Pomona

22. Humans have no fixed axis

23. Optical Flow Kinematic Chains M * = [v1,v2,…,vn]  ?

24. Also estimate Model M  * = ? ? [v1,v2,…,vn]

25. Estimate Kinematics Kathy Pullen

26. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

27. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

28. - Acquisition of Nonrigid 3D Kinematics With Gene Alexander, Henning Biermann, Aaron Hertzmann, Lorenzo Torresani, Danny Yang, Hrishi Deshpande

29. 3D Shape Model Linear Interpolation between 3D Key-Shapes: S 1 S 2 S S = 40% S1 + 60% S2

30. Basis Shape Factorization Complete 2D Tracks or FlowMatrix-Rank <= 3*K M * =  [v1,v2,…,vn]

31. Nonrigid 3D Kinematics from point tracks -

32. - Nonrigid 3D Kinematics results

33. -

34. -

35. Motion Capture Modeling Synthesis From Pixels to 3D Blend Shapes (Torresani et al)

36. Motion Capture Modeling Synthesis From Pixels to 3D Blend Shapes (Torresani et al)

37. CHANGING Motion (Facial Speech) Video Rewrite Bregler, Slaney, Covell

38. CHANGING Age Disney Feature Animation: Gemeni Man Hoyt Yeatman, Director / Visual Effects Supervisor Jinko Gotoh, Producer Andrew Millstein, Exec. Producer Lance Williams, Technical Architect Price Pethel, Creative Director / Actor Walter Hyneman, Digital Effects Supervisor Shari Judson, Production Coordinator, Karla Ventocilla, Production Asst. Hioki Itokazu, Modeler Xinmin Zhao, John Lewis, Sean Jenkins, Dan Ruderman, Eric Enderton, Chris Bregler, Jim Rothrock, Rasmus Tamstorf, Heather Pritchett, Shyh-Chyuan Huang, Developers Sergi Sagas, Jason MacLeod, Manuel Kraemer, Lewis Siegel, Kyle Strawitz Richard Lehman, Technical Directors

39. Disney Face Project

40. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

41. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

42. Turning to the Masters: Motion Capturing Cartoons Chris Bregler, Lorie Loeb, Erika Chuang, Hrishi Deshpande Stanford University [CB: Other picture coming]

43. Turning to the Masters: Expressive Animations

44. Turning to the Masters: Expressive Animations

45. Examples © Warner Bros.

46. Mathematical Motion Models Robot Skeleton Model Human Skeleton Model Non-Rigid Model Style Model

47. Laban Capture With Peggy Hackney and Ted Warburton and Alyssa Lees and Lorenzo Torresani Laban Movement Analysis Capturing Effort Data Driven

48. CHANGING Motion (Style) “Effort Space” Free & Light Bound & Strong Torressani et al

49. CHANGING Motion (Style) “Effort Space” Normal 125% Peggy Lees et al

50. CHANGING Motion (Facial Speech) Video Rewrite Interval Corp. Bregler, Slaney Covell

51. CHANGING Motion (Body Speech) M. Stone et al Rutgers, NYU, Electronic Arts

52. CHANGING Age Disney Feature Animation: Gemeni Man Hoyt Yeatman, Director / Visual Effects Supervisor Jinko Gotoh, Producer Andrew Millstein, Exec. Producer Lance Williams, Technical Architect Price Pethel, Creative Director / Actor Walter Hyneman, Digital Effects Supervisor Shari Judson, Production Coordinator, Karla Ventocilla, Production Asst. Hioki Itokazu, Modeler Xinmin Zhao, John Lewis, Sean Jenkins, Dan Ruderman, Eric Enderton, Chris Bregler, Jim Rothrock, Rasmus Tamstorf, Heather Pritchett, Shyh-Chyuan Huang, Developers Sergi Sagas, Jason MacLeod, Manuel Kraemer, Lewis Siegel, Kyle Strawitz Richard Lehman, Technical Directors

53. Disney Face Project

54. Scaling Up Pixels 1000’s People

55. Scaling Up Largest Mocap Volume so far, 50x50x10 feet, 6 “players” Electronic Theater this year in LA: 190x190x40 And we have more “players”: 4000 © Motion Analysis for “Nike Gamebreakers”

56. SIGGRAPH Electronic Theater

57. The Core Team Sally Rosenthal Katie Salen, Parsons School of Design Luke Dubois, Cycling74 / NYU-ITP Tom Igoe, NYU-ITP Michael Naimark, USC / NYU-ITP Jon Meyer, NYU-MRL Jeremi Sudol, NYU-MRL Akex Postelnicu, RPI Michael Rabinovich, RPI Kevin Feeley, NYU-MRL Clilly Castiglia, NYU-MRL Bo Wright, NYU-MRL Jessica DeVicnezo, NYU-MRL Robb Bifano, NYU-MRL Gloria Sed, NYU-ITP & Fashion Design Chris Bregler, NYU-MRL Sally Rosenthal Katie Salen, Parsons School of Design Luke Dubois, Cycling74 / NYU-ITP Tom Igoe, NYU-ITP Michael Naimark, USC / NYU-ITP Jon Meyer, NYU-MRL Jeremi Sudol, NYU-MRL Akex Postelnicu, RPI Michael Rabinovich, RPI Kevin Feeley, NYU-MRL Clilly Castiglia, NYU-MRL Bo Wright, NYU-MRL Jessica DeVicnezo, NYU-MRL Robb Bifano, NYU-MRL Gloria Sed, NYU-ITP & Fashion Design Chris Bregler, NYU-MRL

60. From Single Pixels to Thousands of People

61. movement.nyu.edu squidball.net