1. Multiresolution Motion Analysis with Applications Jehee Lee Sung Yong Shin Dept of EE&CS, KAIST Jehee Lee Sung Yong Shin Dept of EE&CS, KAIST

2. Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

3. Character Animation Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on stimulates the need for a unified approachstimulates the need for a unified approach Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on stimulates the need for a unified approachstimulates the need for a unified approach

4. Multiresolution Analysis Representing a signal at multiple resolutions gives hierarchy of successively smoother signalsgives hierarchy of successively smoother signals facilitates a variety of signal processing tasksfacilitates a variety of signal processing tasks Representing a signal at multiple resolutions gives hierarchy of successively smoother signalsgives hierarchy of successively smoother signals facilitates a variety of signal processing tasksfacilitates a variety of signal processing tasks

5. Previous Work Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95] Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95]

6. Issues in Motion Analysis Difficulties in handling motion data Inherent non-linearity of orientation spaceInherent non-linearity of orientation spaceCoordinate-invariance Independent of the choice of coordinate framesIndependent of the choice of coordinate frames Difficulties in handling motion data Inherent non-linearity of orientation spaceInherent non-linearity of orientation spaceCoordinate-invariance Independent of the choice of coordinate framesIndependent of the choice of coordinate frames

7. Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

8. Motion Representation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation

9. Decomposition Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Reduction Expansion

10. Decomposition and Reconstruction DecompositionReconstructionDecompositionReconstruction

11. Our Approach Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion

12. Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

13. Motion Displacement global (fixed) reference frame

14. Motion Displacement global (fixed) reference frame

15. Hierarchical Displacement Mapping

19. A series of successively refined motions Coordinate-independentCoordinate-independent –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization A series of successively refined motions Coordinate-independentCoordinate-independent –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization Hierarchical Displacement Mapping

20. Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

21. Spatial Filtering for Orientation Data Linear shift-invariant (LSI) filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints Linear shift-invariant (LSI) filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints

22. Previous Work Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98) –lack of crucial filter properties Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98) –lack of crucial filter properties

23. Basic Idea Exploit correspondence in differential spaces Linear motion :Linear motion : Angular motion :Angular motion : Exploit correspondence in differential spaces Linear motion :Linear motion : Angular motion :Angular motion : Velocity Acceleration

24. Transformation Transformation between linear and angular signals

25. Filter Design Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed

26. Filter Design Given: spatial filter F Output: spatial filter H for orientation data Local supportLocal support –#support( H ) = #support( F ) Given: spatial filter F Output: spatial filter H for orientation data Local supportLocal support –#support( H ) = #support( F )

27. Properties of Orientation Filters Coordinate-invarianceTime-invarianceSymmetryCoordinate-invarianceTime-invarianceSymmetry

28. Examples (1) Blurring by binomial masks Original Angular acceleration Filtered Original Filtered Original Filtered

29. Examples (2) SmoothingSmoothing Original Angular acceleration Filtered Original Filtered Original Filtered

30. Examples (3) High-frequency boosting Original Angular acceleration Filtered Original Filtered Original Filtered

31. Our scheme vs. Re-normalization Re-normalizationOur scheme Filtering with an average filter

32. Coordinate Frame-Invariance Decomposition Reconstruction

33. Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

34. Enhancement / Attenuation Level-wise scaling of coefficients

35. Motion Blending Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples straight walking straight walking turning with a walk turning with a walk straight limping straight limping turning with a limp turning with a limp

36. Motion Blending

37. Motion Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

38. Motion Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

39. Motion Stitching Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly

40. Motion Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

41. Motion Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

42. Motion Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level stub a toelimpstitching

43. System Demonstration

44. Conclusion Multiresolution motion Analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invarianceCoordinate-invariance Multiresolution motion Analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invarianceCoordinate-invariance