Automating Graph-Based Motion Synthesis Lucas Kovar Michael Gleicher University of Wisconsin-Madison

Working with Large Data Sets Goal: convert motion data into a usable form. Can we automate this? –Must preserve realism and provide control Motion model User specifications Motion Automated process Motion data

Graph-Based Synthesis Motion data is static clips of fixed length. How can we make long sequences of motion? walk Edge = clip Node = choice point Graph walk = motion stand stop start walking Typically one uses move trees, built by hand. How can we automate this?

Outline Motion graphs Snap-together motion

Outline Motion graphs –Building motion graphs –Using motion graphs Snap-together motion

Motion Graphs Idea: automatically add transitions to a data set. Quality: Only transition when motions are similar (but wherever they are similar). Motion 1 Motion 2 Motion 1 Motion 2 Control: Search for “optimal” sequences of edges

Related Work Statistical models –Brand and Hertzmann ’01, Li et al. ’02 –Looser quality guarantees, less focus on control Other motion representations –Simulation (Lamouret and van de Panne ’96) –Video (Shödl et al. ’01, ‘02) Concurrent work –Arikan and Forsyth ’02, Lee et al. ‘02

Motion Graphs: An Example

Adding Transitions For arbitrary motions, transitions are hard. Create transitions where motions are similar. ?

Finding Similar Frames Need derivatives (velocity, acceleration, etc.) Joint angles are hard to compare directly Must account for coordinate invariance –Effect of perturbation (e.g., rotate shoulder) depends on pose –Different camera ≠ different motion!

Distance Metric Derivative information Align coordinate systems “Body shape”, not joint angles Initial frames

Finding Transition Points Every pair of frames now has a distance. Transitions are local minima below a threshold. Motion 2 Frames Motion 1 Frames

Finding Transition Points Transition thresholds control quality vs. flexibility tradeoff. Threshold = 0 cmThreshold = 8 cmThreshold = 16 cm

Creating Transitions Motion data is interpolated with C 1 continuity. Initial signals Interpolation weights Result Constraints are also “blended” and enforced as a post-process (Kovar et al. ’02).

Outline Motion graphs Building motion graphs –Using motion graphs Snap-together motion

Interacting With Motion Graphs Any sequence of edges is a motion! Control is harder – motion graphs are complex. Find graph walks that minimize a cost function.

Searching for Motion Branch-and-bound speeds the search. Incrementally search for the optimal motion.

Path Synthesis Goal: extract motion that follows a path. Can also restrict motion style. User’s path ( ) Motion’s path ( ) Minimize

Results

Outline Motion graphs Snap-together motion

Motion Graphs: Advantages/Drawbacks But, their complicated structure is problematic. –Must use costly search methods –Synthesized motion is only a “best fit” –Graph is hard to reason about Motion graphs allow one to create lengthy, complicated motions with little effort.

Structure vs. Unstructured Graphs

Snap-Together Motion (STM) Idea: help build graphs with simple structure. –Control: small number of “hub” nodes –Quality: smooth transitions, constraints enforced –Automation: build hubs just by selecting a pose

STM: Preview

STM: Overview Find groups of similar frames (match sets) and make multi-way transitions 1.Pick a “seed” frame or have the system suggest one 2.Find similar frames and add displacement maps so motions are identical 3.Ensure constraints remain enforced

STM: Overview Snappable Motion Original Motion … Synthesized Motion

Creating Match Sets Given a pose, find similar poses –Build distance grid (same metric as before) –Pose = row; find 1D local minima below threshold

Making Transitions Goal: make each frame in the match set identical. Add displacement maps so each frame has the average pose and velocity.

Making Transitions(cont.) Original motionPosture fittingVelocity fitting Use a two-level displacement map –Coarse knots for pose –Denser knots for velocity

Enforcing Constraints Two-step process to preserve pose at each hub 1.Enforce constraints on frames in the match set. 2.Fix these frames, enforce in rest of each clip. Problem: conflicting constraints Hub 1 Hub 2 Solution: group hub nodes with conflicts and find single constraint position over group. Foot positions don’t match! constraint spans clip

Results

Conclusion Simplify graph-based synthesis by automating the identification and tuning of transitions. Different strategies different kinds of graphs –Motion graphs: “greedy” strategy is general and highly automated, but synthesis requires search –STM: clustering makes simpler graphs for improved control, but assumes hub poses exist.