2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang

2/14/00 The Standard Vision Pipeline Image / Video Recognition Modeling Capture

2/14/00 Vision for Graphics / Animation Image / Video Synthesis Modeling Capture

2/14/00 Vision for Graphics / Animation Image / Video Expressive Animations Modeling Capture

2/14/00 The Standard Analogy

2/14/00 The Graphics Problem

2/14/00 The Vision Problem

2/14/00 Vision Based Animation

2/14/00 Vision for Graphics Image / Video Synthesis Modeling Capture

2/14/00 Vision for Graphics Image / Video Synthesis Richer Modeling Capture

2/14/00 Vision for Animation Image / Video Kinematics Dynamics Synthesis Capture Richer Modeling

2/14/00 Models: 2 Issues Explicit / Implicit tradeoff: Hand CodedLearned

2/14/00 Models: 2 Issues Explicit / Implicit tradeoff: Hand CodedLearned M X Y Analytically derived Fit Black-Box

2/14/00 Models: 2 Issues Explicit / Implicit tradeoff: Model / Data tradeoff: Pure ModelPure Data AnalyticalLearned

2/14/00 Models: 2 Issues Explicit / Implicit tradeoff: Model / Data tradeoff: Pure ModelPure Data AnalyticalLearned

2/14/00 Case #1: Models of Kinematics - Approximate Kinematic Chains. - Fine-tune Kinematic Chains. - Acquisition of Nonrigid Models.

2/14/00 Approximate Kinematic Chains - Enough for Animation ?

2/14/00 Case #1: Models of Kinematics - Approximate Kinematic Chains. - Fine-tune Kinematic Chains. - Acquisition of Nonrigid Models.

2/14/00 Fine-Tune Kinematics Anna Rabinowicz

2/14/00 Fine-Tune Kinematics Kathy Pullen

2/14/00 Case #1: Models of Kinematics - Approximate Kinematic Chains. - Fine-tune Kinematic Chains. - Acquisition of Nonrigid Models.

2/14/00 - Acquisition of Nonrigid 3D Kinematics With Gene Alexander, Henning Biermann, Aaron Hertzmann, Lorenzo Torresani, Danny Yang

2/14/00 Solution for Non-rigid Tracking / Reconstruction - Prior Model: 2D: PCA-based (Cootes et al 95) 3D: Blanz+Vetter-99

2/14/00 3D Model Acquisition - Multi-view input: Pighin et al 98

2/14/00 Solution for Rigid 3D Acquisition Structure from Motion: - Tomasi-Kanade-92 Factorization 3D Pose 3D rigid Object

2/14/00 Acquisition without prior model ? No Model available ? Model too generic/specific ? Stock-Footage only in 2D ?

2/14/00 Solution based on Factorization - We want 3 things: - 3D non-rigid shape model - for each frame: - 3D Pose - non-rigid configuration (deformation) -> Tomasi-Kanade-92: W = P S Rank 3

2/14/00 Solution based on Factorization - We want 3 things: - 3D non-rigid shape model - for each frame: - 3D Pose - non-rigid configuration (deformation) -> PCA-based representations: W = P non-rigid S Rank K

2/14/00 Factoring twice -

2/14/00 Nonrigid 3D Kinematics from point tracks -

2/14/00 - Nonrigid 3D Kinematics from dense flow

2/14/00 - Nonrigid 3D Kinematics from dense flow

2/14/00 - Nonrigid 3D Kinematics from dense flow

2/14/00 Motion Capture Modeling Synthesis Nonrigid 3D Kinematics from dense flow

2/14/00 Case #2: Models of Dynamics Pure ModelPure Data

2/14/00 Case #2: Models of Dynamics Pure ModelPure Data Learning ControllerMocap. Library

2/14/00 Case #2: Models of Dynamics Pure ModelPure Data Parametric Non-Parametric

2/14/00 Case #2: Models of Dynamics Pure ModelPure Data Kernel-Based Densities

2/14/00 Case #2: Models of Dynamics Kathy Pullen

2/14/00 Case #2: Models of Dynamics “Motion Texture” Sampling / Matching

2/14/00 Case #2: Models of Dynamics P(Li | Lj) Sampling / Matching:

2/14/00 Synthesizing Subtleties + Variations Kathy Pullen Motion Capture Modeling Synthesis

2/14/00 Synthesizing Subtleties + Variations Kathy Pullen Motion Capture Modeling Synthesis

2/14/00 Synthesizing Subtleties + Variations Kathy Pullen Motion Capture Modeling Synthesis

2/14/00 Synthesizing Subtleties + Variations Kathy Pullen Motion Capture Modeling Synthesis

2/14/00 Synthesizing Subtleties + Variations Kathy Pullen Motion Capture Modeling Synthesis

2/14/00 Vision Based Animation Animations Kinematics Dynamics Richer Modeling

2/14/00 Human Movement Research Computer Science: Vision / Graphics Bio-Mechanics: Motion Models/Analysis Art: Animation / Dance Medicine/ Neurology: Movement Disorders

2/14/00 Model Data

2/14/00 Model Data Expressive Animation

2/14/00 Human Movement Research Computer Science: Vision / Graphics Bio-Mechanics: Motion Models/Analysis Art: Animation / Dance Medicine/ Neurology: Movement Disorders

2/14/00 Project Mariposa: Animation Principles Lorie Loeb

2/14/00 Project Mariposa: Animation Principles Lorie Loeb Realism is out

2/14/00 Project Mariposa: Animation Principles Lorie Loeb Realism is out Expressiveness is in

2/14/00 Model Data Expressive Animation

2/14/00 Model Data Controversy: Motion-Capture based Animation

2/14/00 Model Data Controversy: Motion-Capture based Animation

2/14/00 Model Data Controversy: Motion-Capture based Animation

2/14/00 Model Data Controversy: Motion-Capture based Animation

2/14/00 Synthesis by Analysis Animations

2/14/00 Enough for Animation ? Expressive Animations Video/Mocap

2/14/00 Expressive Animation Expressive Output Cartoon Capture