1. Coordinate-Invariant Methods For Motion Analysis and Synthesis Jehee Lee Dept. Of Electric Engineering and Computer Science Korea Advanced Institute of Science and Technology Jehee Lee Dept. Of Electric Engineering and Computer Science Korea Advanced Institute of Science and Technology

2. Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications

3. Character Animation Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on

4. Motion Signal Processing Difficulties in handling motion data SingularitySingularity Inherent non-linearity of orientation spaceInherent non-linearity of orientation space General issues in motion signal processing Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance Difficulties in handling motion data SingularitySingularity Inherent non-linearity of orientation spaceInherent non-linearity of orientation space General issues in motion signal processing Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance

5. Coordinate-Invariance Independent of the choice of coordinate frames

6. Time-Invariance Independent of the position on the signal Time

7. Overview Generalize conventional methods Designing spatial filters for orientation dataDesigning spatial filters for orientation data Multiresolution analysis for rigid motionMultiresolution analysis for rigid motionRequirements Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance Computationally optimalComputationally optimal Generalize conventional methods Designing spatial filters for orientation dataDesigning spatial filters for orientation data Multiresolution analysis for rigid motionMultiresolution analysis for rigid motionRequirements Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance Computationally optimalComputationally optimal

8. Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications

9. Spatial Filtering for Orientation Data Linear Time-Invariant filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints Linear Time-Invariant filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints

10. Previous Work Euler angle parameterization Bodenheimer et al. (’97)Bodenheimer et al. (’97)Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98) Euler angle parameterization Bodenheimer et al. (’97)Bodenheimer et al. (’97)Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98)

11. Exp and Log

14. logexp

15. Linear and Angular Displacement

18. Transformation Transformation between linear and angular signals

19. Filter Design Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed

20. Filter Design Given: spatial filter F Output: spatial filter H for orientation data Given: spatial filter F Output: spatial filter H for orientation data

21. Examples

25. Example

26. Examples Original Angular acceleration Filtered Original Filtered Original Filtered

27. Properties of Orientation Filters Coordinate-invarianceTime-invarianceSymmetryCoordinate-invarianceTime-invarianceSymmetry

28. Computation Computefor i=1 … N(# of log = N)Computefor i=1 … N(# of log = N) Computefor i=1 … N(# of exp = N)Computefor i=1 … N(# of exp = N) Computefor i=1 … N(# of log = N)Computefor i=1 … N(# of log = N) Computefor i=1 … N(# of exp = N)Computefor i=1 … N(# of exp = N)

29. Our scheme vs. Re-normalization Re-normalizationOur scheme Filtering with average filter

30. Local vs. Global Parameterization Global Log parameterization Transform toTransform to Apply a filterApply a filter Transform toTransform to Global Log parameterization Transform toTransform to Apply a filterApply a filter Transform toTransform to

31. Transform into a Hemi-Sphere Antipodal equivalence

32. Cumulative vs. Non-cumulative Cumulative local parameterization ComputeCompute Apply a filter, and then IntegrateApply a filter, and then Integrate Cumulative local parameterization ComputeCompute Apply a filter, and then IntegrateApply a filter, and then Integrate

33. Coordinate- and Time-invariant Alternatives Geometric construction Slerp (spherical linear interpolation)Slerp (spherical linear interpolation) Bezier curve construction of Shoemake (‘85)Bezier curve construction of Shoemake (‘85) Algebraic construction on tangent space Local parameterization (coordinate-invariant)Local parameterization (coordinate-invariant) Local support (time-invariant)Local support (time-invariant) Geometric construction Slerp (spherical linear interpolation)Slerp (spherical linear interpolation) Bezier curve construction of Shoemake (‘85)Bezier curve construction of Shoemake (‘85) Algebraic construction on tangent space Local parameterization (coordinate-invariant)Local parameterization (coordinate-invariant) Local support (time-invariant)Local support (time-invariant)

34. Summary (Motion Filtering) Designing spatial filters for orientation data Satisfy desired propertiesSatisfy desired properties –Coordinate-invariance –Time-invariance –Symmetry Simple, efficient, easy to implementSimple, efficient, easy to implement Designing spatial filters for orientation data Satisfy desired propertiesSatisfy desired properties –Coordinate-invariance –Time-invariance –Symmetry Simple, efficient, easy to implementSimple, efficient, easy to implement

35. Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications

36. Multiresolution Analysis Representing a signal at multiple resolutions facilitate a variety of signal processing tasksfacilitate a variety of signal processing tasks give hierarchy of successively smoother signalsgive hierarchy of successively smoother signals Representing a signal at multiple resolutions facilitate a variety of signal processing tasksfacilitate a variety of signal processing tasks give hierarchy of successively smoother signalsgive hierarchy of successively smoother signals

37. Previous Work Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95] Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95]

38. Decomposition Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Reduction Expansion

39. Decomposition and Reconstruction DecompositionReconstructionDecompositionReconstruction

40. Our Approach Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Motion displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Motion displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion

41. Motion Representation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation

42. Motion Displacement global (fixed) reference frame

43. Motion Displacement global (fixed) reference frame

44. Hierarchical Displacement Mapping

48. A series of successively refined motions Coordinate-independenceCoordinate-independence –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization A series of successively refined motions Coordinate-independenceCoordinate-independence –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization Hierarchical Displacement Mapping

49. Coordinate Frame-Invariance Decomposition Reconstruction

50. Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Analysis 4.Applications

51. Enhancement / Attenuation Level-wise scaling of coefficients

52. Enhancement / Attenuation Level-wise scaling of coefficients

53. Extrapolation Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples walking running limping running with a limp running with a limp

54. Extrapolation Walking Limping Turning

55. Extrapolation Walking Strutting Running

56. Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

57. Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

58. Stitching Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly

59. Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

60. Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

61. Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level stub a toelimpstitching

62. Frequency-based motion editing Edit the global pattern of example motions without explicit segmentationwithout explicit segmentation Edit the global pattern of example motions without explicit segmentationwithout explicit segmentation

63. Shuffling and Reconstruction Multiresolution representation of example motion Multiresolution representation of example motion

64. Shuffling and Reconstruction Shuffling The base signal of new motion

65. Shuffling and Reconstruction Multiresolution Sampling Shuffling Reconstruct detail coefficients Reconstruct detail coefficients

66. Shuffling and Reconstruction Multiresolution Sampling Shuffling

67. Multiresolution Sampling InputShufflingOutput

68. Multiresolution Sampling Feature matching –example) the change of linear and angular velocities Feature matching –example) the change of linear and angular velocities Matching

69. Multiresolution Sampling Feature matching –example) the change of linear and angular velocities Feature matching –example) the change of linear and angular velocities Matching Reconstruct

70. Multiresolution Sampling Matching features at multiple resolutions Matching Reconstruct Matching

71. Summary Multiresolution motion analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invariance and Time-invarianceCoordinate-invariance and Time-invariance Multiresolution motion analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invariance and Time-invarianceCoordinate-invariance and Time-invariance