Multiresolution Motion Analysis with Applications Jehee Lee Sung Yong Shin Dept of EE&CS, KAIST Jehee Lee Sung Yong Shin Dept of EE&CS, KAIST

Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

Character Animation Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on stimulates the need for a unified approachstimulates the need for a unified approach Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on stimulates the need for a unified approachstimulates the need for a unified approach

Multiresolution Analysis Representing a signal at multiple resolutions gives hierarchy of successively smoother signalsgives hierarchy of successively smoother signals facilitates a variety of signal processing tasksfacilitates a variety of signal processing tasks Representing a signal at multiple resolutions gives hierarchy of successively smoother signalsgives hierarchy of successively smoother signals facilitates a variety of signal processing tasksfacilitates a variety of signal processing tasks

Previous Work Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95] Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95]

Issues in Motion Analysis Difficulties in handling motion data Inherent non-linearity of orientation spaceInherent non-linearity of orientation spaceCoordinate-invariance Independent of the choice of coordinate framesIndependent of the choice of coordinate frames Difficulties in handling motion data Inherent non-linearity of orientation spaceInherent non-linearity of orientation spaceCoordinate-invariance Independent of the choice of coordinate framesIndependent of the choice of coordinate frames

Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

Motion Representation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation

Decomposition Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Reduction Expansion

Decomposition and Reconstruction DecompositionReconstructionDecompositionReconstruction

Our Approach Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion

Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

Motion Displacement global (fixed) reference frame

Motion Displacement global (fixed) reference frame

Hierarchical Displacement Mapping

A series of successively refined motions Coordinate-independentCoordinate-independent –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization A series of successively refined motions Coordinate-independentCoordinate-independent –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization Hierarchical Displacement Mapping

Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

Spatial Filtering for Orientation Data Linear shift-invariant (LSI) filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints Linear shift-invariant (LSI) filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints

Previous Work Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98) –lack of crucial filter properties Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98) –lack of crucial filter properties

Basic Idea Exploit correspondence in differential spaces Linear motion :Linear motion : Angular motion :Angular motion : Exploit correspondence in differential spaces Linear motion :Linear motion : Angular motion :Angular motion : Velocity Acceleration

Transformation Transformation between linear and angular signals

Filter Design Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed

Filter Design Given: spatial filter F Output: spatial filter H for orientation data Local supportLocal support –#support( H ) = #support( F ) Given: spatial filter F Output: spatial filter H for orientation data Local supportLocal support –#support( H ) = #support( F )

Properties of Orientation Filters Coordinate-invarianceTime-invarianceSymmetryCoordinate-invarianceTime-invarianceSymmetry

Examples (1) Blurring by binomial masks Original Angular acceleration Filtered Original Filtered Original Filtered

Examples (2) SmoothingSmoothing Original Angular acceleration Filtered Original Filtered Original Filtered

Examples (3) High-frequency boosting Original Angular acceleration Filtered Original Filtered Original Filtered

Our scheme vs. Re-normalization Re-normalizationOur scheme Filtering with an average filter

Coordinate Frame-Invariance Decomposition Reconstruction

Contents 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications 1.Introduction 2.Multiresolution Analysis 3.Hierarchical Displacement Mapping 4.Spatial Filtering for Motion Data 5.Applications

Enhancement / Attenuation Level-wise scaling of coefficients

Motion Blending Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples straight walking straight walking turning with a walk turning with a walk straight limping straight limping turning with a limp turning with a limp

Motion Blending

Motion Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

Motion Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

Motion Stitching Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly

Motion Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

Motion Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

Motion Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level stub a toelimpstitching

System Demonstration

Conclusion Multiresolution motion Analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invarianceCoordinate-invariance Multiresolution motion Analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invarianceCoordinate-invariance