1. Animating (human) motion Presented by: –Yoram Atir –Simon Adar

2. Applications of computer animation Movies Advertising Games Simulators …

3. General goals of the work presented -New methods aimed to save time/money/skills needed. -Study motion (texture).

4. Agenda -Basic concepts -Motion Synthesis/texture using motion capture -Physics/Biomechanics Motion Synthesis -Cartoon Motion Retargeting.

5. Basic Concepts Basic concepts Animation world (3D) Skeletal model representation Model positioning Keyframes Motion capture Frequency bands Correlations

6. Basic Concepts 3D animation world -(Human) model is animated in Object space -Animated model projected into “global” space -Camera is placed and rotated -Perspective is set -Other…

7. Basic Concepts Skeletal representation -Each model has its own Default Pose -DOF’s – joint angles/translations relative to Default Pose -Hierarchical (tree) skeletal representation of model Picture from Lecture in Computer Graphics course Department of computer science University of Washington

8. Basic Concepts Creating motion -Skeletal variations between frames -Overall rotation/Translation between frames -Correlate. General Problem: A LOT of work due to the large number of DOFS & high frame rate

9. Basic Concepts Figure positioning -Forward kinematics (simplified): Figure positioning by joint data specification. Problem: -Tedious trial and error.

10. Basic Concepts Figure positioning Inverse kinematics (simplified) -Joint data is acquired by solving for the final position -In general, This is an optimization problem with a large system of variables and constraints -Problems often are expressed as minimization problems, and solved using standard algorithms (gradient decent etc). -Usually, infinite number of possible solutions. -A “good” solution has to be more than “feasible” -Often one is obtained by embedding specific knowledge as additional constraints, and/or -Using Inverse kinematics as a part of a specific solution.

11. Basic Concepts Basic methods for saving labor Motion capture KeyFrames

12. Basic Concepts Keyframes –Specifying only part of DOFs and frames –Computer interpolation between them Problem: “smooth” interpolation looks unreal There are methods to apply “specific noise” –Term has historical roots

13. Basic Concepts Motion capture –Acquired from “live action” –Copied onto animated character Problem: Hard to adapt. “Motion Editing” – methods to adapt mocap –Done in studios –Mocap libraries exist

14. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control

15. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive

16. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive Detail hard

17. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive Detail hard Many DOF

18. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive Detail hard Many DOF Detail easy

19. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive Detail hard Many DOF Detail easy All DOF

20. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive No control Detail hard Many DOF Detail easy All DOF

21. Basic Concepts Keyframing vs. Mocap Keyframing Mocap DisadvantagesAdvantages Control Intuitive No control Not intuitive Detail hard Many DOF Detail easy All DOF

22. Keyframe Data vs. Motion Capture Data

23. Basic Concepts Frequency Bands Right flatRight toeLeft flatLeft toe

24. Basic Concepts Frequency Bands Simplifies the form of the data –Low frequency Variations: Large scale motions. –Higher frequency variations: individual “noise” / Jitter Both are important to preserve in order to capture the essence of motion

25. Basic Concepts Correlations Joints angle/translation data is related to each other Joint angles are correlated over time Correlation “plot” is –(somewhat) Specific to the type of motion –Carries “personality” information (style)

26. More information… INTRODUCTION TO COMPUTER ANIMATION – Rick parent http://www.cis.ohio-state.edu/~parent/book/outline.html Splines http://www.people.nnov.ru/fractal/splines/Intro.html Hash Inc - Animation software (Movies, tutorials…) http://www.hash.com Google…

27. Motion Capture Assisted Animation: Texturing and Synthesis Kathy Pullen Chris Bregler SIGGRAPH 2002 Motion Capture Assisted Animation – Pullen/Bregler

28. Agenda -Basic concepts -Motion Synthesis/texture using motion capture -Physics/Biomechanics Motion Synthesis -Cartoon Motion Retargeting

29. Goal: Motion Capture Assisted Animation Create a method that allows an artist low- level control of the motion Combine the strengths of keyframe animation with those of mocap Create a method that allows an artist low- level control of the motion Combine the strengths of keyframe animation with those of mocap Motion Capture Assisted Animation – Pullen/Bregler

30. Goal: Motion Capture Assisted Animation “Sketch” an animation by keyframing Animate only a few degrees of freedom Set few keyframes “Enhance” the result with mocap data Synthesize missing degrees of freedom Texture keyframed degrees of freedom “Sketch” an animation by keyframing Animate only a few degrees of freedom Set few keyframes “Enhance” the result with mocap data Synthesize missing degrees of freedom Texture keyframed degrees of freedom Motion Capture Assisted Animation – Pullen/Bregler

31. What is a Motion Texture? Every individual’s movement is unique Synthetic motion should capture the texture To “texture” means to add style to a pre- existing motion Technically, texturing is a special case of synthesis

32. Goal: Motion Capture Assisted Animation Blue = Keyframed Purple = Textured/Synthesized Motion Capture Assisted Animation – Pullen/Bregler

33. How an Animator Works A few degrees of freedom at first Not in detail Fill in detail with more keyframes later A few degrees of freedom at first Not in detail Fill in detail with more keyframes later Motion Capture Assisted Animation – Pullen/Bregler

34. The Method in Words Choose degrees of freedom to drive the animation Compare these degrees of freedom from the keyframed data to mocap Find similar regions Look at what the rest of the body is doing in those regions Put that data onto the keyframed animation Choose degrees of freedom to drive the animation Compare these degrees of freedom from the keyframed data to mocap Find similar regions Look at what the rest of the body is doing in those regions Put that data onto the keyframed animation Motion Capture Assisted Animation – Pullen/Bregler

35. Choices the Animator Must Make 1.Which DOF to use as matching angles 2.Which DOF to texture, which to synthesize 3.Which frequency band to use in matching 4.How many frequency bands to use in texturing 5.How many matches to keep 6.How many best paths to keep 1.Which DOF to use as matching angles 2.Which DOF to texture, which to synthesize 3.Which frequency band to use in matching 4.How many frequency bands to use in texturing 5.How many matches to keep 6.How many best paths to keep Motion Capture Assisted Animation – Pullen/Bregler

36. Before Beginning: Choose Matching Angles Left Hip x Left Hip y Left Hip z Left Knee x Left Knee y Left Knee z Left Ankle x Left Ankle y Left Ankle z Left Ball x Left Ball y Left Ball z Right Hip x Right Hip y Right Hip z Right Knee x Right Knee y Right Knee z Right Ankle x Right Ankle y Right Ankle z Right Ball x Right Ball y Right Ball z Root x trans Root y trans Root z trans Root x rot Root y rot Root z rot Spine1 x Spine1 y Spine1 z Spine2 x Spine2 y Spine2 z Spine3 x Spine3 y Spine3 z Neck x Neck y Neck z Head x Head y Head z Head Aim x Head Aim y Head Aim z Left Clavicle x Left Clavicle y Left Clavicle z Left Shoulder x Left Shoulder y Left Shoulder z Left Elbow x Left Elbow y Left Elbow z Left Wrist x Left Wrist y Left Wrist z Right Clavicle x Right Clavicle y Right Clavicle z Right Shoulder x Right Shoulder y Right Shoulder z Right Elbow x Right Elbow y Right Elbow z Right Wrist x Right Wrist y Right Wrist z Time

37. Matching Angles Drive the Synthesis Left Hip x Left Hip y Left Hip z Left Knee x Left Knee y Left Knee z Left Ankle x Left Ankle y Left Ankle z Left Ball x Left Ball y Left Ball z Right Hip x Right Hip y Right Hip z Right Knee x Right Knee y Right Knee z Right Ankle x Right Ankle y Right Ankle z Right Ball x Right Ball y Right Ball z Root x trans Root y trans Root z trans Root x rot Root y rot Root z rot Spine1 x Spine1 y Spine1 z Spine2 x Spine2 y Spine2 z Spine3 x Spine3 y Spine3 z Neck x Neck y Neck z Head x Head y Head z Head Aim x Head Aim y Head Aim z Left Clavicle x Left Clavicle y Left Clavicle z Left Shoulder x Left Shoulder y Left Shoulder z Left Elbow x Left Elbow y Left Elbow z Left Wrist x Left Wrist y Left Wrist z Right Clavicle x Right Clavicle y Right Clavicle z Right Shoulder x Right Shoulder y Right Shoulder z Right Elbow x Right Elbow y Right Elbow z Right Wrist x Right Wrist y Right Wrist z Time

38. Motion Capture Data Left Hip x Left Hip y Left Hip z Left Knee x Left Knee y Left Knee z Left Ankle x Left Ankle y Left Ankle z Left Ball x Left Ball y Left Ball z Right Hip x Right Hip y Right Hip z Right Knee x Right Knee y Right Knee z Right Ankle x Right Ankle y Right Ankle z Right Ball x Right Ball y Right Ball z Root x trans Root y trans Root z trans Root x rot Root y rot Root z rot Spine1 x Spine1 y Spine1 z Spine2 x Spine2 y Spine2 z Spine3 x Spine3 y Spine3 z Neck x Neck y Neck z Head x Head y Head z Head Aim x Head Aim y Head Aim z Left Clavicle x Left Clavicle y Left Clavicle z Left Shoulder x Left Shoulder y Left Shoulder z Left Elbow x Left Elbow y Left Elbow z Left Wrist x Left Wrist y Left Wrist z Right Clavicle x Right Clavicle y Right Clavicle z Right Shoulder x Right Shoulder y Right Shoulder z Right Elbow x Right Elbow y Right Elbow z Right Wrist x Right Wrist y Right Wrist z Time

39. Overview Steps in texture/synthesis method Frequency analysis Matching Path finding Joining Steps in texture/synthesis method Frequency analysis Matching Path finding Joining Motion Capture Assisted Animation – Pullen/Bregler

40. In the following series of slides: Hip angle = matching angle Spine angle = angle being synthesized In the following series of slides: Hip angle = matching angle Spine angle = angle being synthesized Example Motion Capture Assisted Animation – Pullen/Bregler

41. Frequency Analysis: Break into Bands Motion Capture Assisted Animation – Pullen/Bregler

42. Frequency Time Band-pass decomposition of matching angles Keyframed DataMotion Capture Data Frequency Analysis Motion Capture Assisted Animation – Pullen/Bregler

43. Frequency Time Keyframed DataMotion Capture Data Chosen low frequency band Frequency Analysis Motion Capture Assisted Animation – Pullen/Bregler

44. Keyframed DataMotion Capture Data Hip angle data (a matching angle) Chosen Low Frequency Band Motion Capture Assisted Animation – Pullen/Bregler

45. Keyframed DataMotion Capture Data Making Fragments Break where first derivative changes sign Motion Capture Assisted Animation – Pullen/Bregler

46. Keyframed DataMotion Capture Data Making Fragments Step through fragments one by one Motion Capture Assisted Animation – Pullen/Bregler

47. Matching Keyframed Fragment Motion Capture Assisted Animation – Pullen/Bregler

48. Matching Keyframed Fragment Motion Capture Data Motion Capture Assisted Animation – Pullen/Bregler

49. Matching Keyframed Fragment Motion Capture Data Motion Capture Assisted Animation – Pullen/Bregler

50. Matching Compare to all motion capture fragments Angle in degrees Time Keyframed Mocap

51. Matching Resample mocap fragments to be same length Angle in degrees Time Keyframed Mocap

52. Matching Using some metric on all matching angles and on their first derivatives: Keep the K closest matches Angle in degrees Time Keyframed Mocap

53. Matching Keyframed Fragment Motion Capture Data Motion Capture Assisted Animation – Pullen/Bregler

54. Matching Keyframed Fragment Motion Capture Data Close Matches Motion Capture Assisted Animation – Pullen/Bregler

55. Matching Hip Angle (Matching Angle) Spine Angle (For Synthesis) Motion Capture Assisted Animation – Pullen/Bregler

56. Matching and Synthesis Low frequency hip angle data (a matching angle) Spine angle data to be synthesized Motion Capture Assisted Animation – Pullen/Bregler

57. Matching and Synthesis Low frequency hip angle data (a matching angle) Spine angle data to be synthesized Motion Capture Assisted Animation – Pullen/Bregler

58. Matching and Synthesis Low frequency hip angle data (a matching angle) Spine angle data to be synthesized Motion Capture Assisted Animation – Pullen/Bregler

59. Matching and Synthesis Low frequency hip angle data (a matching angle) Spine angle data to be synthesized Motion Capture Assisted Animation – Pullen/Bregler

60. Matching and Synthesis Low frequency hip angle data (a matching angle) Spine angle data to be synthesized Motion Capture Assisted Animation – Pullen/Bregler

61. Matching and Synthesis Low frequency hip angle data (a matching angle) Spine angle data to be synthesized Motion Capture Assisted Animation – Pullen/Bregler

62. Possible Synthetic Spine Angle Data Angle in degrees Time

63. Path Finding Angle in degrees Time We would like to: Use as much consecutive fragments as possible Stay as close as possible to best fit

64. Path Finding Angle in degrees Time

65. Path Finding Angle in degrees Time

66. Path Finding Angle in degrees Time

67. Path Finding Angle in degrees Time

68. Joining Angle in degrees Time

69. Enhancing Animations: Texturing and Synthesis Keyframed Textured Synthesized Not keyframed

70. Texturing Synthesize upper frequency bands Motion Capture Assisted Animation – Pullen/Bregler

71. Texturing Band-pass decomposition of keyframed data Frequency Time

72. Texturing Synthesize upper frequency bands Frequency Time

73. Walking animations Texturing and Synthesis Keyframed Sketch

74. Walking animations Texturing and Synthesis Motion Capture Data Two different styles of walk

75. Walking animations Texturing and Synthesis Enhanced Animation Upper body is synthesized Lower body is textured

76. Otter Animations: Texturing Keyframed data

77. Otter Animations: Texturing Textured animation

78. Dance Animations: Texturing and Synthesis Lazy Keyframed Sketch

79. Dance Animations: Texturing and Synthesis Motion Capture Data

80. Dance Animations: Texturing and Synthesis Enhanced Animation Blue = Keyframed Purple = Textured/Synthesized

81. Dance Animations: Texturing Keyframed Sketch With More Detail

82. Dance Animations: Texturing Textured Animation Blue = Keyframed Purple = Textured

83. Summary of the Method Keyframed data Mocap Data Keyframed Data Mocap DataPossible Synthetic Data Matching Angles Sketch + Mocap Frequency Analysis Matching Path FindingJoining Enhanced Animation

84. Choices the Animator Must Make 1.Which DOF to use as matching angles 2.Which DOF to texture, which to synthesize 3.Which frequency band to use in matching 4.How many frequency bands to use in texturing 5.How many matches to keep 6.How many best paths to keep 1.Which DOF to use as matching angles 2.Which DOF to texture, which to synthesize 3.Which frequency band to use in matching 4.How many frequency bands to use in texturing 5.How many matches to keep 6.How many best paths to keep

85. Conclusions and Applications Appropriate for an artist interested in a very particular style of motion The artist may have a relatively small motion capture set of that style The artist may want precise control over parts of the motion Appropriate for an artist interested in a very particular style of motion The artist may have a relatively small motion capture set of that style The artist may want precise control over parts of the motion

86. Conclusions and Further Work Direct incorporation of hard constraints Fundamental units of motion Direct incorporation of hard constraints Fundamental units of motion

87. For more info... http://graphics.stanford.edu/~pullen Special Thanks to: Reardon Steele, Electronic Arts

88. Agenda -Basic concepts -Motion Synthesis/texture using motion capture -Physics, Biomechanics Motion Synthesis -Cartoon Motion Retargeting

89. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Agenda Motivation Scope (very dynamic motions, constrains are very important, automatic constrain detection) Overview of the process Constraint and stage detection. Transition pose generation. Momentum control Objective function generation Putting it all together

90. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Motivation Generate rapid prototyping of realistic character motion Avoid simulated human models, that are very complex, and don’t always look realistic

91. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Scope Highly dynamic movement such as jumping, kicking, running, and gymnastics. Less energetic motions such as walking or reaching will not work well in this framework

92. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

93. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process The objective is to transforms simple animations into realistic character motion by applying laws of physics and the biomechanics domain The unknowns are: values of joint angles and parameters of angular and linear momentum

94. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

95. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Constraint and stage detection Each input sequence has two parts: –The part that needs to be improved –The part that needs to kept intacked Automatically extract the positional and sliding constrains

96. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Positional constraint detection A positional constraint fixes a specific point on the character to a stationary location for a period of time We need to find if all these points lie on a line, plane In an articulated character we find the constraints on each body part

97. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Positional constraint detection The algorithm looks for fixed points (point, line, plane)

98. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Positional constraint detection The algorithm looks for fixed points (point, line, plane)

99. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Sliding constraints

100. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

101. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Transition pose generation A transition pose separates constrained and unconstrained stages. Two possibilities: –We ask the animator to draw the transition poses –We have an estimator to suggest a transition pose

102. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Transition Pose Estimator DB contains examples of different motions The input of that DB are the motion parameters like: flight distance, flight height, takeoff angle, landing angle, spin angle.. The DB has a simplified representation of the transition poses by three COM’s We use IK to obtain the full character’s pose from those three COM’s The KNN - K nearest neighbor algorithm The pose estimator predicts the candidate pose by interpolating the KNN with the weights that describe the similarity to the input.

103. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

104. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Momentum control Transition poses constrain the motion at few key points of the animation Dynamic constraints ensure realistic motion of each segment Linear and angular momentum give us these dynamic constraints

105. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Momentum during unconstrained and constrained stages linear momentum - During “flight” the only force is gravity Angular momentum - During “flight” there is no change in Angular momentum During “ground” stage we avoid computing the momentums and use empirical characteristics

106. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

107. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Objective functions There are three Objective functions, the basic idea behind them is power consumption –Minimum mass displacement –Minimal velocity of DOFs –Static balance

108. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

109. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Putting it all together Environment constraints (Ce) Transition pose constraints (Cp) Momentum constraints (Cm) Q are character’s DOFs subject to

110. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Overview of the process Motion sketch Character description Motion DB User interaction Constraint & phase detection Transition pose synthesis Objective functions OptimizationAnimation Momentum control

111. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Some Results Wide variety of figures: male, female, child 51 DOFs The body dimensions and mass distribution is taken from biomechanics literature In some of the cases the animator selects the body parts to be constraints The animator can change relative timing between each phase The optimization was solved by using SNOPT a general nonlinearly-constrained optimization package The optimization time depends on the duration of the animation All of the simple animation took less than five minutes to sketch For all examples the synthesis process took less than five minutes

112. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Broad jump Only 3 keyframes at takeoff, peak and landing

113. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Running The angular momentum constraint creates a counter- body movement by the shoulders and arms to counteract the angular momentum generated by the legs. Keyframing 7 DOFs

114. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Hopscotch Each hop requires 3 keyframes and has fewer than 7 DOFs

115. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Handspring There were no handstands within the DB so the user had to modify the result

116. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" High-bar Two constraints stages: the bar and ground

117. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Karate kick A second synthesis add a keyframe in the peak

118. C. Karen Liu & Zoran Popovi´c "Synthesis of Complex Dynamic Character Motion from Simple Animations" Twist jumps

119. Agenda -Basic concepts -Motion Synthesis/texture using motion capture -Physics/Biomechanics Motion Synthesis -Cartoon Motion Retargeting

120. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Cartoon motion capture & retargeting What is Cartoon Capture & Retargeting Motivation Scope Challenges Modeling Cartoon motion cartoon capture –contour capture –video capture Retargeting Additional constrains & post processing Performance

121. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” What is Cartoon Capture & Retargeting Cartoon Capture –Track the motion From 2D Animation –Represent the motion & save Retargeting –Translate the motion representation to another output media

122. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Digitized video Key shapes Cartoon capture Output corresponding key shapes Motion representation retargeting Output video Cartoon motion capture & retargeting scheme

123. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Motivation

124. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Motivation (cont.) Animation has high exaggeration drawings & motion, which cannot be extracted from the motion capture Highly trained Animators are rare Fast motion style creation

125. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Scope The pink area

126. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Challenges - Capture Cartoon characters have no markers Standard skeletal model-based capture techniques are not suitable The low frame rate makes tracking difficult

127. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Challenges - Retargeting Current retargeting are based on skeletal models Retarget 2D information to 3D

128. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Modeling Cartoon motion Digitized video Key shapes Cartoon capture Output corresponding key shapes Motion representation retargeting Output video

129. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Modeling Cartoon motion Two types of deformations –Affine deformation –Key shape deformation

130. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Affine Deformation Affine parameters y x

131. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Key-Shape Deformation S k are the key shapes

132. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Modeling Cartoon motion In total there are 6+K variables that represent the motion

133. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Cartoon motion capture Digitized video Key shapes Cartoon capture Output corresponding key shapes Motion representation retargeting Output video

134. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Cartoon motion capture contour capture: the input is a sequence of contours video capture: the input is the video sequence

135. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” contour capture Two step minimization: –Find Affine parameters –Find Key-Shape weights Iterate

136. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Video capture The output is the same as in contour capture The input is the video sequence instead of the contour sequence

137. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Retargeting Digitized video Key shapes Cartoon capture Output corresponding key shapes Motion representation retargeting Output video

138. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Retargeting For each Input key-shape an Output key- shape is drawn.

139. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Retargeting Process Key shapes Interpolation Apply Affine transformation From motion capture Retargeted media Retarget Motion capture

140. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Examples

141. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Additional constrains & post processing Undesirable effects may still appear Determine constraints that force the character go through certain position at certain time Apply ad-hoc global transformation that fulfill these constraints

142. C.Bregler, L.Loeb, E.Chuang, H.Deshpande; "Turning to the Masters: Motion Capturing Cartoons” Performance Quantative performance wasn’t mentioned The more complex the motion of the character is, the more key-shapes are needed Many of the animations contain jitter, but the overall exaggerated motion dominates