1. Rick Parent - CIS681 Motion Capture Use digitized motion to animate a character

2. Rick Parent - CIS681 Animation Techniques Digitize motion Procedural - simuleate Model or rules Set initial conditions Keyframe Surface model Motion libraries Stitch together e.g., games

3. Rick Parent - CIS681 Technologies Passive optical - reflective markers Multiple cameras Need non-reflective environ. Active optical - LEDs pulse ID Need power supply Multiple cameras Magnetic - Active sensors Cheaper Need magnetic-neutral environment Need power supply Tether or wireless comm. Smaller work area Mechanical - Rotary Only poses Cheapest

4. Rick Parent - CIS681 Magnetic Up to 144Hz 13-18 6 DoF sensors

5. Rick Parent - CIS681 Passive Optical Motion Capture

6. Rick Parent - CIS681 Active Optical Motion Capture

7. Rick Parent - CIS681 Optical Motion Capture Instrument “talent” Compute joint positions from 3D positions of multiple surface markers Convert multiple 2D marker positions to 3D positions Establish correspondence of markers in images Triangulate 3D position using two or more images Create synthetic character with those dimensions and animation Determine limb lengths from joint coordinates Capture 2D position of markers in multiple cameras

8. Rick Parent - CIS681 Hand Motion Capture Movement too intricate for optical Types of sensors Mechanical Fiber optic

9. Rick Parent - CIS681 Facial Motion Capture Motion too detailed for sensing at distance

10. Rick Parent - CIS681 Motion Capture Procedure 1. Plan 2. Capture 3. Clean 4. Edit 5. Map

11. Rick Parent - CIS681 Problems Dimensions of instrumented talent has to match synthetic charcter Motion captured is what you get - period Limitations on spatial extent of motion Restricted movement because of instrumentation (harness, power supply, physical attachments, etc.) Limitation on complexity of motion because of sensors (e.g., severe occlusions, etc.)

12. Rick Parent - CIS681 Research Blending Blending from one motion to another Signal processing Space-time constriants Retargeting Modify angles based on limb lengths Maintain constraints (e.g. foot on floor) Adapting Modify secondary DoFs e.g. wave arm while walking

13. Rick Parent - CIS681 Motion Warping Motion curves Sparse keyframe-like constraints Keep similar to original Warp each curve independently Constraints (  i,t i ) Time warp constraints (t’ j,t j )  ’(t’) defined by  ’(t) = f( ,t) t= g(t’) g(t’): smooth, well-behaved

14. Rick Parent - CIS681 Motion Warping  ’(t) = a(t)  (t) + b(t) User specifies amounts of a(t i ) and b(t i ) Use interpolating spline to define a(t) and b(t) over length of curve t  (t)

15. Rick Parent - CIS681 Motion Warping - time warp t’ t t  (t) Stretch and compress to overlap Use interpolating spline Linearly interpolate  (t) =   (t) + (1-  )   (t)

16. Rick Parent - CIS681 Motion Warping

17. Rick Parent - CIS681 Motion Warping

18. Rick Parent - CIS681 Motion Warping

19. Rick Parent - CIS681 Motion Warping

20. Rick Parent - CIS681 Motion Editing Multiresolution filtering Cascade of lowpass filters Convolve w/ B-spline 5x5 filter kernel Subsample image by factor of 2 Until 1 pixel - DC component Bandpass pyramid Repeatedly differencing 2 successive lowpass images Expand subtracted images Image reconstruction: add up all bandpass plus DC

21. Rick Parent - CIS681 Motion Editing Motion Low frequency; general gross motion High frequency; detail, subtleties, noise Each motion parameter => signal Calculate lowpass & bandpass Over # frames by expanding filter Lowpass G0: solid G3: dashed Bandpass L0: solid L2: dashed

22. Rick Parent - CIS681 Motion Editing 1.Calculate lowpass sequence 2.Calculate bandpass filter bands 3.Adjust gains and multiply L k ’s by gain values 4.Blend bands 5.Reconstruct signal

23. Rick Parent - CIS681 Motion Editing Adjusting gains of bands for joint positions

24. Rick Parent - CIS681 Motion Editing Multitarget interpolation (like Motion Warping)

25. Rick Parent - CIS681 Motion Editing Multitarget motion interpolation using frequency bands

26. Rick Parent - CIS681 Motion Editing Blending between two walks without (top) and with (bottom) correspondence in time

27. Rick Parent - CIS681 Motion Editing Blending two waves without (top) and with (bottom) corresponence in time

28. Rick Parent - CIS681 Motion Editing 1.Keyframes 2.Interpolate 3.Adjust 4.Recalculate 5.Apply

29. Rick Parent - CIS681 Motion Editing Example of applying displacement curves

30. Rick Parent - CIS681 Retargeting Assume identical structure Use motion displacements m(t) = m0(t) + d(t) IK to enforce constraints can add High-frequence components Use spacetime constraints

31. Rick Parent - CIS681 Retargeting Minimize difference from original motion Constraints (kinematic) Joint limits Footplants Point at same place Point on character follows another point 2 points at specified distance Vector between points at specified orientation

32. Rick Parent - CIS681 Spacetime Constraints Consider spacetime particle Position: x(t) Jet-force: f(t) Motion equation: mx -f - mg = 0 Constraints x(t0) = a x(t1) = b Minimize Find f such that x satisfies the boundary conditions and R is minimized Numerical solution: sequential quadratic programming

33. Rick Parent - CIS681 Sequential Quadratic Programming Compute second order Newton-Raphson step in R Compute first order Newton-Raphson step in the C’s Combine by projecting the first onto the null space of the second Requires Jacobian of the constraint function and Hessian of the objective function Final update = S j + T j Until C’s = 0 and R at minimum

34. Rick Parent - CIS681 Linear System Solving Use pseudo inverse Use conjugate gradient to compute the pseudo inverse

35. Rick Parent - CIS681 Retargeting

36. Rick Parent - CIS681 Retargeting

37. Rick Parent - CIS681 Retargeting

38. Rick Parent - CIS681 Retargeting