Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian.

Slides:



Advertisements
Similar presentations
Part 2: Unsupervised Learning
Advertisements

Bayes rule, priors and maximum a posteriori
Bayesian Belief Propagation
The fundamental matrix F
A Brief Introduction to Bayesian Inference Robert Van Dine 1.
System Integration and Experimental Results Intelligent Robotics Research Centre (IRRC) Department of Electrical and Computer Systems Engineering Monash.
2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 10: The Bayesian way to fit models Geoffrey Hinton.
Reducing Drift in Parametric Motion Tracking
Creating probability forecasts of binary events from ensemble predictions and prior information - A comparison of methods Cristina Primo Institute Pierre.
“ Pixels that Sound ” Find pixels that correspond (correlate !?) to sound Kidron, Schechner, Elad, CVPR
Rob Fergus Courant Institute of Mathematical Sciences New York University A Variational Approach to Blind Image Deconvolution.
Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang 1, Xiaolei Huang 2, Dimitris Metaxas 1 Rutgers University 1, Lehigh University.
SIGGRAPH Course 30: Performance-Driven Facial Animation For Latest Version of Bregler’s Slides and Notes please go to:
Paper Discussion: “Simultaneous Localization and Environmental Mapping with a Sensor Network”, Marinakis et. al. ICRA 2011.
Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004.
Stanford CS223B Computer Vision, Winter 2005 Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and.
Stanford CS223B Computer Vision, Winter 2007 Lecture 8 Structure From Motion Professors Sebastian Thrun and Jana Košecká CAs: Vaibhav Vaish and David Stavens.
Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado.
Tracking using the Kalman Filter. Point Tracking Estimate the location of a given point along a sequence of images. (x 0,y 0 ) (x n,y n )
SIGGRAPH Course 30: Performance-Driven Facial Animation Section: Marker-less Face Capture and Automatic Model Construction Part 1: Chris Bregler, NYU Part.
Probabilistic Robotics Introduction Probabilities Bayes rule Bayes filters.
Structure From Motion Sebastian Thrun, Gary Bradski, Daniel Russakoff
I The meaning of chance Axiomatization. E Plurbus Unum.
An Iterative Optimization Approach for Unified Image Segmentation and Matting Hello everyone, my name is Jue Wang, I’m glad to be here to present our paper.
CSCE 641 Computer Graphics: Image-based Modeling (Cont.) Jinxiang Chai.
Learning the Appearance of Faces: A Unifying Approach for the Analysis and Synthesis of Images. Thomas Vetter Germany University of Freiburg
Computer vision: models, learning and inference Chapter 6 Learning and Inference in Vision.
Genetic Regulatory Network Inference Russell Schwartz Department of Biological Sciences Carnegie Mellon University.
Brief Introduction to Geometry and Vision
Lab 4 1.Get an image into a ROS node 2.Find all the orange pixels (suggest HSV) 3.Identify the midpoint of all the orange pixels 4.Explore the findContours.
EM and expected complete log-likelihood Mixture of Experts
Learning Human Pose and Motion Models for Animation Aaron Hertzmann University of Toronto.
Abstract Developing sign language applications for deaf people is extremely important, since it is difficult to communicate with people that are unfamiliar.
Course Syllabus 1.Color 2.Camera models, camera calibration 3.Advanced image pre-processing Line detection Corner detection Maximally stable extremal regions.
Yu-Wing Tai, Hao Du, Michael S. Brown, Stephen Lin CVPR’08 (Longer Version in Revision at IEEE Trans PAMI) Google Search: Video Deblurring Spatially Varying.
1 Reconstructing head models from photograph for individualized 3D-audio processing Matteo Dellepiane, Nico Pietroni, Nicolas Tsingos, Manuel Asselot,
2/14/00 Computer Vision. 2/14/00 Computer Vision Lecturer: Ir. Resmana Lim, M.Eng. Text: 1) Computer Vision -- A Modern Approach.
Uncertainty Management in Rule-based Expert Systems
Making sense of randomness
Real-Time Simultaneous Localization and Mapping with a Single Camera (Mono SLAM) Young Ki Baik Computer Vision Lab. Seoul National University.
CPSC : Data-driven Computer Graphics Jinxiang Chai.
MURI Annual Review, Vanderbilt, Sep 8 th, 2009 Heterogeneous Sensor Webs for Automated Target Recognition and Tracking in Urban Terrain (W911NF )
4 Proposed Research Projects SmartHome – Encouraging patients with mild cognitive disabilities to use digital memory notebook for activities of daily living.
Paper Reading Dalong Du Nov.27, Papers Leon Gu and Takeo Kanade. A Generative Shape Regularization Model for Robust Face Alignment. ECCV08. Yan.
CSSE463: Image Recognition Day 14 Lab due Weds, 3:25. Lab due Weds, 3:25. My solutions assume that you don't threshold the shapes.ppt image. My solutions.
Computer Vision, CS766 Staff Instructor: Li Zhang TA: Yu-Chi Lai
Probability and Statistics in Vision. Probability Objects not all the sameObjects not all the same – Many possible shapes for people, cars, … – Skin has.
Lecture 9 Feature Extraction and Motion Estimation Slides by: Michael Black Clark F. Olson Jean Ponce.
Probabilistic Robotics Introduction Probabilities Bayes rule Bayes filters.
- 1 - Outline Introduction to the Bayesian theory –Bayesian Probability –Bayes’ Rule –Bayesian Inference –Historical Note Coin trials example Bayes rule.
Monte Carlo Localization for Mobile Robots Frank Dellaert 1, Dieter Fox 2, Wolfram Burgard 3, Sebastian Thrun 4 1 Georgia Institute of Technology 2 University.
ICCV 2007 Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang 1, Xiaolei Huang 2, Dimitris Metaxas 1 Rutgers University 1,
Probabilistic Robotics Probability Theory Basics Error Propagation Slides from Autonomous Robots (Siegwart and Nourbaksh), Chapter 5 Probabilistic Robotics.
Outline Historical note about Bayes’ rule Bayesian updating for probability density functions –Salary offer estimate Coin trials example Reading material:
Ch 1. Introduction Pattern Recognition and Machine Learning, C. M. Bishop, Updated by J.-H. Eom (2 nd round revision) Summarized by K.-I.
CSC321: Lecture 8: The Bayesian way to fit models Geoffrey Hinton.
CENG 789 – Digital Geometry Processing 9- Least-Squares Solutions Asst. Prof. Yusuf Sahillioğlu Computer Eng. Dept,, Turkey.
Machine Learning for Graphics and Vision Aaron Hertzmann University of Toronto Aaron Hertzmann University of Toronto.
Reconstruction of a Scene with Multiple Linearly Moving Objects Mei Han and Takeo Kanade CISC 849.
A Forest of Sensors: Using adaptive tracking to classify and monitor activities in a site Eric Grimson AI Lab, Massachusetts Institute of Technology
Computer Vision, Robotics, Machine Learning and Control Lab
Particle Filtering for Geometric Active Contours
Markov ó Kalman Filter Localization
Dynamical Statistical Shape Priors for Level Set Based Tracking
Structure from motion Input: Output: (Tomasi and Kanade)
Eric Grimson, Chris Stauffer,
PRAKASH CHOCKALINGAM, NALIN PRADEEP, AND STAN BIRCHFIELD
CS639: Data Management for Data Science
Structure from motion Input: Output: (Tomasi and Kanade)
Presentation transcript:

Machine Learning for Computer graphics Aaron Hertzmann University of Toronto Bayesian

Computers are really fast If you can create it, you can render it

How do you create it? Digital Michaelangelo Project Steven Schkolne

Two problems How do we get the data into the computer? How do we manipulate it?

Data fitting

Key questions How do you fit a model to data? –How do you choose weights and thresholds? –How do you incorporate prior knowledge? –How do you merge multiple sources of information? –How do you model uncertainty? Bayesian reasoning provides a solution

Talk outline Bayesian reasoning Facial modeling example Non-rigid modeling from video

What is reasoning? How do people reason? How should computers do it?

Aristotelian Logic If A is true, then B is true A is true Therefore, B is true A: My car was stolen B: My car isn’t where I left it

Real-world is uncertain Problems with pure logic Don’t have perfect information Don’t really know the model Model is non-deterministic

Beliefs Let B(X) = “belief in X”, B(¬X) = “belief in not X” 1.An ordering of beliefs exists 2.B(X) = f(B(¬X)) 3.B(X) = g(B(X|Y),B(Y))

Cox axioms R.T. Cox, “Probability, frequency, and reasonable expectation,” American J. Physics, 14(1):1-13, 1946

“Probability theory is nothing more than common sense reduced to calculation.” - Pierre-Simon Laplace, 1814

Bayesian vs. Frequentist Frequentist (“Orthodox”): –Probability = percentage of events in infinite trials Medicine, biology: Frequentist Astronomy, geology, EE, computer vision: largely Bayesian

Learning in a nutshell Create a mathematical model Get data Solve for unknowns

Face modeling Blanz, Vetter, “A Morphable Model for the Synthesis of 3D Faces,” SIGGRAPH 99

Generative model Faces come from a Gaussian Learning

Bayes Rule Often:

Learning a Gaussian

Maximization trick Maximize minimize

Fitting a face to an image Generative model

Fitting a face to an image Maximize minimize

Why does it work? p(x|model)

General features Models uncertainty Applies to any generative model Merge multiple sources of information Learn all the parameters

Caveats Still need to understand the model Not necessarily tractable Potentially more involved than ad hoc methods

Applications in graphics Shape and motion capture Learning styles and generating new data

Advanced topics Marginalize out some data Keep all uncertainty

Learning Non-Rigid 3D Shape from 2D Motion Joint work with Lorenzo Torresani and Chris Bregler (Stanford, NYU)

Camera geometry Orthographic projection

Non-rigid reconstruction Input: 2D point tracks Output: 3D nonrigid motion Totally ambiguous!

Shape reconstruction Least-squares version minimize

Bayesian formulation maximize

How does it work? Maximize Minimize (actual system is slightly more sophisticated)

Results Least-squares, no reg. Least-squares, reg. Gaussian LDS View 2 Input

Conclusions Bayesian methods provide unified framework Build a model, and reason about it The future of data-driven graphics

Background C. Bregler, A. Hertzmann, H. Biermann, CVPR 2000, “Recovering Non-Rigid 3D Shape from Image Streams” Non-rigid structure from motion

Background L. Torresani et al., CVPR 2001, “Tracking and Modeling Non- Rigid Objects with Rank Constraints” M. Brand, CVPR 2001, “Morphable 3D models from video”, “Flexible flow…” Loop –Track based on non-rigid SFM –Update non-rigid SFM based on tracking

Shape regularization

Our approach Learn Gaussian deformation PDF Simultaneously with reconstruction Similar to Cootes-Taylor, Vetter- Blanz, etc. Similar to factor analysis, PPCA, TCA

Factor analysis model Like factor analysis Use EM [ Ghahramani-Hinton, Roweis, Tipping and Bishop]

Temporal dynamics

Results