1. Interactive Control of Avatars Animated with Human Motion Data Jehee Lee Carnegie Mellon University Seoul National University Jehee Lee Carnegie Mellon University Seoul National University Jinxiang Chai Carnegie Mellon University Paul S. A. Reitsma Brown University Jessica K. Hodgins Carnegie Mellon University Nancy S. Pollard Brown University

2. Avatars: Controllable, Responsive Animated Characters Realistic behavior Realistic behavior Non-trivial environment Non-trivial environment Intuitive user interface Intuitive user interface

3. Interactive Avatar Control How to create a rich set of behaviors ? How to create a rich set of behaviors ? How to direct avatars ? How to direct avatars ? How to animate avatar motion ? How to animate avatar motion ? How to create a rich set of behaviors ? How to create a rich set of behaviors ? How to direct avatars ? How to direct avatars ? How to animate avatar motion ? How to animate avatar motion ? Motion Database PreprocessSearch MotionSensorData AnimateAvatars User Interface

4. Related Work (Probabilistic/Statistical Models) Statistical models Bradley & Stuate 97 Bradley & Stuate 97 Brand & Hertzmann 00 Brand & Hertzmann 00 Pullen & Bregler 00 Pullen & Bregler 00 Bowden 00 Bowden 00 Galata, Johnson & Hogg 01 Galata, Johnson & Hogg 01 Li, Wang & Shum 02 Li, Wang & Shum 02 Search and playback original motion data Molina-Tanco & Hilton 00 Molina-Tanco & Hilton 00 Pullen & Bregler 02 Pullen & Bregler 02 Arikan & Forsyth 02 Arikan & Forsyth 02 Kovar, Gleicher & Pighin 02 Kovar, Gleicher & Pighin 02 This work This work

5. Motion Database In video games Many short, carefully planned, labeled motion clips Many short, carefully planned, labeled motion clips Manual processing Manual processing In video games Many short, carefully planned, labeled motion clips Many short, carefully planned, labeled motion clips Manual processing Manual processing

6. Walk Cycle StopStart Left Turn Right Turn

7. Motion Database Our approach Extended, unlabeled sequences Extended, unlabeled sequences Automatic processing Automatic processing Our approach Extended, unlabeled sequences Extended, unlabeled sequences Automatic processing Automatic processing

8. Motion Data Acquisition

9. Maze - Sketch Interface

10. Re-sequenceRe-sequence Motion capture regionVirtual environment Obstacles Sketched path

11. Re-sequenceRe-sequence Motion capture regionVirtual environment

12. Data Acquisition “Poles and Holes” rough terrain

13. Terrain Navigation

14. Unstructured Input Data A number of motion clips Each clip contains many frames Each clip contains many frames Each frame represents a pose Each frame represents a pose A number of motion clips Each clip contains many frames Each clip contains many frames Each frame represents a pose Each frame represents a pose

15. Unstructured Input Data Connecting transition Between similar frames Between similar frames Connecting transition Between similar frames Between similar frames

16. Graph Construction

17. Distance between Frames Weighted differences of joint angles Weighted differences of joint velocities

18. Pruning Transitions Reduce storage space O(n^2) will be prohibitive O(n^2) will be prohibitive Better quality Pruning “bad” transitions Pruning “bad” transitions Efficient search Sparse graph Sparse graph i j

19. Pruning Transition Contact state: Avoid transition to dissimilar contact state Contact state: Avoid transition to dissimilar contact state Likelihood: User-specified threshold Likelihood: User-specified threshold Similarity: Local maxima Similarity: Local maxima Avoid dead-ends: Strongly connected components Avoid dead-ends: Strongly connected components

20. Graph Search Best-first graph traversal Path length is bounded Path length is bounded Fixed number of frames at each frame Fixed number of frames at each frame Comparison to global search Intended for interactive control Intended for interactive control Not for accurate global planning Not for accurate global planning

21. Comparison to Real Motion Environment with physical obstacles

22. Comparison to Real Motion

23. Global vs. Local Coordinates Local, moving, body-relativecoordinates Global, fixed, object-relativecoordinates

24. User Interface In maze and terrain environments Sketch interface was effective Sketch interface was effective In maze and terrain environments Sketch interface was effective Sketch interface was effective

25. User Interface In playground A broader variety of motions are available A broader variety of motions are available In playground A broader variety of motions are available A broader variety of motions are available

26. Choice Interface What is available in database ? Provide with several options Provide with several options Select among available behaviors Select among available behaviors What is available in database ? Provide with several options Provide with several options Select among available behaviors Select among available behaviors

27. Choice Interface

28. What to Show Space and Time Windows

29. How to Create Choices

30. ClusteringClustering AA A BC C D D E E

31. How to Capture Transitions AA A BC C D D E E

32. A A BC C D D E E A

33. Cluster Tree A BC C D D E E AB A D C A Three possible actions: ABA, ABC, ABD A

34. Cluster Forest AA A BC C D D E E DB AC D C A E B C AB A D C

35. Performance Interface Motion Database Search Vision-based Interface

36. Vision Interface – Single Camera

37. SearchSearch 3 sec AA A BC C D D E E Video buffer

38. SearchSearch 3 sec AB A D AA A BC C D D E E

39. SummarySummary Graph representation Flexibility in motion Flexibility in motion Cluster forest A map for avatar’s behavior A map for avatar’s behavior User interfaces Graph representation Flexibility in motion Flexibility in motion Cluster forest A map for avatar’s behavior A map for avatar’s behavior User interfaces

40. Future Work Body-relative vs. object-relative Assemble objects in new configurations Assemble objects in new configurations Interactions among avatars Interactions among avatars Evaluate user interface User test for effectiveness User test for effectiveness Combine with existing techniques Motion editing and style modifications Motion editing and style modifications

41. Acknowledgements Thank All of our motion capture subjects All of our motion capture subjects Rory and Justin Macey Rory and Justin MaceySupport NSF NSF Project web page http://graphics.snu.ac.kr/~jehee/Avatar/avatar.htm

43. Similarity between Frames Our Work Arikan & Forsyth Kovar & Gleicher & Pighin Joint Angle/Position AnglePosition Pose OOO Velocity OO Implicitly Acceleration X Translation Only Implicitly

44. Pruning Transitions Our Work Arikan & Forsyth Kovar & Gleicher & Pighin Contact OXX Likelihood OOO Similarity OXO Avoid dead ends OXO

45. Related Work (Character Animation) Rule-basedControl system Bruderlin & Calvert 96 Perlin & Goldberg 96 Chi et al. 00 Cassell et al. 01 Hodgins et al. 95 Wooten and Hodgins 96 Laszlo et al. 96 Faloutsos et al. 01 Example-based Probabilistic/Statistical Models Popovic & Witkin 95 Bruderlin & Willams 95 Unuma et al. 95 Lamouret & van de Panne 96 Rose et al. 97 Wiley & Hahn 97 Gleicher 97, 98, 01 Sun & Mataxas 01 Bradley & Stuart 97 Pullen & Bregler 00, 02 Tanco & Hilton 00 Brand & Hertzmann 00 Galata & Johnson & Hogg 01 Arikan & Forsyth 02 Kovar & Gleicher & Pighin 02 Li & Wang & Shum 02 (THIS WORK)

46. Related Work (User Interfaces) Graphical User Interfaces Performance (Motion capture devices) Performance(Vision-based) Bruderlin & Calvert 96 Laszlo et al. 96 Rose et al. 97 Chi et al. 00 Badler et al. 93 Semwal et al. 98 Blumberg 98 Molet et al. 99 “ Mocap Boxing ” (Konami) Blumberg & Galyean 95 Brand 99 Rosales et al. 01 Ben-Arie et al. 01