Computer Graphics Lab Electrical Engineering, Technion, Israel June 2009 [1] [1] Xuemiao Xu, Animating Animal Motion From Still, Siggraph 2008

Create an animation movie by reconstructing the animal motion from a still picture Algorithm Still Image Animation Movie

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Extract the snapshots using a tool called “MVP-PIE” developed at the CGM lab at the Technion. This stage is not part of our project but it’s a necessary step in order to extract the snapshots from the source image.

Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction Shape Context Descriptor

User Interaction: choose two anchor points that define the movement direction of the animal. Rotate the snapshot in the angle defined by the user. Find the contour levels of the snapshot using the Matlab Image Proccesing Toolbox. Take the last contour level and decimate it, and display it on the XY plane.

Find the contour of the snapshot For every point on the contour draw the circles and bins as shown Create the “matrix descriptor “ Invariant to translation, rotation and scale.

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Distance between Snapshot-K and Snapshot-L: Point on snapshot K Closest point on snapshot L How do we find ? Distance between two matrix descriptors

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Find the optimal path between the snapshots using the distances matrix Looking to find a path which minimizes the Energy function: Local similarity: Sampling uniformity: Global distinction: We use the Simulated Annealing Optimization Algorithm Avoid “getting stuck” on a local minimum, because of the Temperature factor. Ignore outliers that don’t belong in the motion cycle. Ignore snapshots that are too similar to other snapshots.

Initialize a path While (T>Limit) Loop K times Choose a new path length L curr Generate new Path P curr at length L curr and price C curr If (C curr – C old ) < 0 Accept current path (trivial). Update parameters. Else if exp{(C curr – C old )/T} < rand[0,1] Accept current path. Update parameters. Else Reject current path. Decrease Temprature T = T*Annealing_Factor End

Full Cycle: Half Cycle:

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Pivot In order to create smooth and realistic animation, all the animals have to be in the same pose relative to the camera. Affine transformation includes: Translation scale rotation All snapshots are translated relative to a pivot snapshot.

Pivot histogram: Every snapshot has its own color and texture that can vary from one snapshot to another. Histogram standardization of all snapshots will make the final animation look smoother.

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Finding animation points: Points which are not static during the animal motion Morph between the source points and target points Source Image is registered to the green points Destination Image is registered to the red points.

Shape Context Descriptor Find distances Path Finding 6,3,4,1,2,5 Consistency Refinement TPS Morphing TPS Morphing Creating Animation snapshots Shape context descriptors Distances matrix Motion Path Consistent snapshots Animation between two snapshots Source Image Snapshot Extraction

Morph three frames between two snapshots Source Destination

Blend the morphed frames into the background image, by the following formula: The Threshold Value (T) was empirically found and set to T=120. T=50T=120 T=150T=200

Output Animation: Input Images:

Output Animation: Input Images:

Output animation: Input Image: