Coordinate-Invariant Methods For Motion Analysis and Synthesis Jehee Lee Dept. Of Electric Engineering and Computer Science Korea Advanced Institute of Science and Technology Jehee Lee Dept. Of Electric Engineering and Computer Science Korea Advanced Institute of Science and Technology

Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications

Character Animation Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on Realistic motion data Motion capture technologyMotion capture technology Commercial librariesCommercial libraries Producing animation from available motion clips requires specialized toolsrequires specialized tools –interactive editing, smoothing, enhancement, blending, stitching, and so on

Motion Signal Processing Difficulties in handling motion data SingularitySingularity Inherent non-linearity of orientation spaceInherent non-linearity of orientation space General issues in motion signal processing Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance Difficulties in handling motion data SingularitySingularity Inherent non-linearity of orientation spaceInherent non-linearity of orientation space General issues in motion signal processing Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance

Coordinate-Invariance Independent of the choice of coordinate frames

Time-Invariance Independent of the position on the signal Time

Overview Generalize conventional methods Designing spatial filters for orientation dataDesigning spatial filters for orientation data Multiresolution analysis for rigid motionMultiresolution analysis for rigid motionRequirements Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance Computationally optimalComputationally optimal Generalize conventional methods Designing spatial filters for orientation dataDesigning spatial filters for orientation data Multiresolution analysis for rigid motionMultiresolution analysis for rigid motionRequirements Coordinate-invarianceCoordinate-invariance Time-invarianceTime-invariance Computationally optimalComputationally optimal

Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications

Spatial Filtering for Orientation Data Linear Time-Invariant filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints Linear Time-Invariant filter filter mask :filter mask : vector-valued signal :vector-valued signal : Not suitable for unit quaternion data unit-length constraintsunit-length constraints

Previous Work Euler angle parameterization Bodenheimer et al. (’97)Bodenheimer et al. (’97)Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98) Euler angle parameterization Bodenheimer et al. (’97)Bodenheimer et al. (’97)Re-normalization Azuma and Bishop (‘94)Azuma and Bishop (‘94) Exploit a local parameterization Lee and Shin (‘96)Lee and Shin (‘96) Welch and Bishop (‘97)Welch and Bishop (‘97) Fang et al. (‘98)Fang et al. (‘98) Hsieh et al. (‘98)Hsieh et al. (‘98)

Exp and Log

logexp

Linear and Angular Displacement

Transformation Transformation between linear and angular signals

Filter Design Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed Given: spatial filter F Output: spatial filter H for orientation data “Unitariness” is guaranteed“Unitariness” is guaranteed

Filter Design Given: spatial filter F Output: spatial filter H for orientation data Given: spatial filter F Output: spatial filter H for orientation data

Examples

Example

Examples Original Angular acceleration Filtered Original Filtered Original Filtered

Properties of Orientation Filters Coordinate-invarianceTime-invarianceSymmetryCoordinate-invarianceTime-invarianceSymmetry

Computation Computefor i=1 … N(# of log = N)Computefor i=1 … N(# of log = N) Computefor i=1 … N(# of exp = N)Computefor i=1 … N(# of exp = N) Computefor i=1 … N(# of log = N)Computefor i=1 … N(# of log = N) Computefor i=1 … N(# of exp = N)Computefor i=1 … N(# of exp = N)

Our scheme vs. Re-normalization Re-normalizationOur scheme Filtering with average filter

Local vs. Global Parameterization Global Log parameterization Transform toTransform to Apply a filterApply a filter Transform toTransform to Global Log parameterization Transform toTransform to Apply a filterApply a filter Transform toTransform to

Transform into a Hemi-Sphere Antipodal equivalence

Cumulative vs. Non-cumulative Cumulative local parameterization ComputeCompute Apply a filter, and then IntegrateApply a filter, and then Integrate Cumulative local parameterization ComputeCompute Apply a filter, and then IntegrateApply a filter, and then Integrate

Coordinate- and Time-invariant Alternatives Geometric construction Slerp (spherical linear interpolation)Slerp (spherical linear interpolation) Bezier curve construction of Shoemake (‘85)Bezier curve construction of Shoemake (‘85) Algebraic construction on tangent space Local parameterization (coordinate-invariant)Local parameterization (coordinate-invariant) Local support (time-invariant)Local support (time-invariant) Geometric construction Slerp (spherical linear interpolation)Slerp (spherical linear interpolation) Bezier curve construction of Shoemake (‘85)Bezier curve construction of Shoemake (‘85) Algebraic construction on tangent space Local parameterization (coordinate-invariant)Local parameterization (coordinate-invariant) Local support (time-invariant)Local support (time-invariant)

Summary (Motion Filtering) Designing spatial filters for orientation data Satisfy desired propertiesSatisfy desired properties –Coordinate-invariance –Time-invariance –Symmetry Simple, efficient, easy to implementSimple, efficient, easy to implement Designing spatial filters for orientation data Satisfy desired propertiesSatisfy desired properties –Coordinate-invariance –Time-invariance –Symmetry Simple, efficient, easy to implementSimple, efficient, easy to implement

Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Motion Analysis 4.Applications

Multiresolution Analysis Representing a signal at multiple resolutions facilitate a variety of signal processing tasksfacilitate a variety of signal processing tasks give hierarchy of successively smoother signalsgive hierarchy of successively smoother signals Representing a signal at multiple resolutions facilitate a variety of signal processing tasksfacilitate a variety of signal processing tasks give hierarchy of successively smoother signalsgive hierarchy of successively smoother signals

Previous Work Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95] Image and signal processing Gauss-Laplacian pyramid [Burt and Adelson 83]Gauss-Laplacian pyramid [Burt and Adelson 83] Texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so onTexture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on Motion synthesis and editing Hierarchical spacetime control [Liu, Gortler and Cohen 94]Hierarchical spacetime control [Liu, Gortler and Cohen 94] Motion signal processing [Bruderlin and Williams 95]Motion signal processing [Bruderlin and Williams 95]

Decomposition Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Expansion : up-sampling followed by smoothing Reduction : smoothing followed by down-sampling Reduction Expansion

Decomposition and Reconstruction DecompositionReconstructionDecompositionReconstruction

Our Approach Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Motion displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion Multiresolution Motion Analysis Hierarchical displacement mappingHierarchical displacement mapping –How to represent –Motion displacement mapping [Bruderlin and Williams 95] –Motion warping [Popovic and Witkin 95] Spatial filtering for motion dataSpatial filtering for motion data –How to construct –Implement reduction and expansion

Motion Representation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation Configuration of articulated figures Bundle of motion signalsBundle of motion signals Each signal represents time-varying positions and orientationsEach signal represents time-varying positions and orientations Rigid transformationRigid transformation

Motion Displacement global (fixed) reference frame

Motion Displacement global (fixed) reference frame

Hierarchical Displacement Mapping

A series of successively refined motions Coordinate-independenceCoordinate-independence –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization A series of successively refined motions Coordinate-independenceCoordinate-independence –measured in a body-fixed coordinate frame UniformityUniformity –through a local parameterization Hierarchical Displacement Mapping

Coordinate Frame-Invariance Decomposition Reconstruction

Contents 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Analysis 4.Applications 1.Issues in Motion Analysis and Synthesis 2.Spatial Filtering for Motion Data 3.Multiresolution Analysis 4.Applications

Enhancement / Attenuation Level-wise scaling of coefficients

Enhancement / Attenuation Level-wise scaling of coefficients

Extrapolation Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples Combine multiple motions together select a base signal and details from different examplesselect a base signal and details from different examples walking running limping running with a limp running with a limp

Extrapolation Walking Limping Turning

Extrapolation Walking Strutting Running

Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

Stitching A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation A simple approach Estimate velocities at boundaries, thenEstimate velocities at boundaries, then Perform -interpolationPerform -interpolation

Stitching Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly Difficulties of the simple approach Hard to estimate velocity robustlyHard to estimate velocity robustly

Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level WalkingRunning

Stitching Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level Stitching motion clips seamlessly Merging coefficients level-by-levelMerging coefficients level-by-level stub a toelimpstitching

Frequency-based motion editing Edit the global pattern of example motions without explicit segmentationwithout explicit segmentation Edit the global pattern of example motions without explicit segmentationwithout explicit segmentation

Shuffling and Reconstruction Multiresolution representation of example motion Multiresolution representation of example motion

Shuffling and Reconstruction Shuffling The base signal of new motion

Shuffling and Reconstruction Multiresolution Sampling Shuffling Reconstruct detail coefficients Reconstruct detail coefficients

Shuffling and Reconstruction Multiresolution Sampling Shuffling

Multiresolution Sampling InputShufflingOutput

Multiresolution Sampling Feature matching –example) the change of linear and angular velocities Feature matching –example) the change of linear and angular velocities Matching

Multiresolution Sampling Feature matching –example) the change of linear and angular velocities Feature matching –example) the change of linear and angular velocities Matching Reconstruct

Multiresolution Sampling Matching features at multiple resolutions Matching Reconstruct Matching

Summary Multiresolution motion analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invariance and Time-invarianceCoordinate-invariance and Time-invariance Multiresolution motion analysis Coherency in positions and orientationsCoherency in positions and orientations Coordinate-invariance and Time-invarianceCoordinate-invariance and Time-invariance