Figure-centric averages Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the.

Slides:



Advertisements
Similar presentations
Applications of one-class classification
Advertisements

Shapelets Correlated with Surface Normals Produce Surfaces Peter Kovesi School of Computer Science & Software Engineering The University of Western Australia.
Chapter 3 Image Enhancement in the Spatial Domain.
Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction.
Image Congealing (batch/multiple) image (alignment/registration) Advanced Topics in Computer Vision (048921) Boris Kimelman.
Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović
電腦視覺 Computer and Robot Vision I Chapter2: Binary Machine Vision: Thresholding and Segmentation Instructor: Shih-Shinh Huang 1.
Automatic Feature Extraction for Multi-view 3D Face Recognition
2008 SIAM Conference on Imaging Science July 7, 2008 Jason A. Palmer
Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada.
Face Recognition in Hyperspectral Images Z. Pan, G. Healey, M. Prasad and B. Tromberg University of California Published at IEEE Trans. on PAMI Vol 25,
Feature-Based Image Metamorphosis Thaddeus Beier Shawn Neely SIGGRAPH 1992.
Lecture 5 Template matching
Exchanging Faces in Images SIGGRAPH ’04 Blanz V., Scherbaum K., Vetter T., Seidel HP. Speaker: Alvin Date: 21 July 2004.
Data-driven Methods: Faces : Computational Photography Alexei Efros, CMU, Fall 2007 Portrait of Piotr Gibas © Joaquin Rosales Gomez.
Real-time Embedded Face Recognition for Smart Home Fei Zuo, Student Member, IEEE, Peter H. N. de With, Senior Member, IEEE.
Final Class: Range Data registration CISC4/689 Credits: Tel-Aviv University.
Computing correspondences in order to study spatial and temporal patterns of gene expression Charless Fowlkes UC Berkeley, Computer Science.
Probabilistic video stabilization using Kalman filtering and mosaicking.
Real-time Combined 2D+3D Active Appearance Models Jing Xiao, Simon Baker,Iain Matthew, and Takeo Kanade CVPR 2004 Presented by Pat Chan 23/11/2004.
Face Recognition Based on 3D Shape Estimation
Image Morphing : Rendering and Image Processing Alexei Efros.
Image Morphing : Computational Photography Alexei Efros, CMU, Fall 2005 © Alexey Tikhonov.
Automatic Face Recognition for Film Character Retrieval in Feature-Length Films Ognjen Arandjelović Andrew Zisserman.
Active Appearance Models Computer examples A. Torralba T. F. Cootes, C.J. Taylor, G. J. Edwards M. B. Stegmann.
Face Collections : Rendering and Image Processing Alexei Efros.
Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.
Image Morphing, Triangulation CSE399b, Spring 07 Computer Vision.
Preprocessing II: Between Subjects John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, UK.
Faces: Analysis and Synthesis Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski.
Learning the Appearance of Faces: A Unifying Approach for the Analysis and Synthesis of Images. Thomas Vetter Germany University of Freiburg
Understanding Faces Computational Photography
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
Image processing Lecture 4.
3D-Assisted Facial Texture Super-Resolution Pouria Mortazavian, Josef Kittler, William Christmas 10 September 2009 Centre for Vision, Speech and Signal.
Introduction --Classification Shape ContourRegion Structural Syntactic Graph Tree Model-driven Data-driven Perimeter Compactness Eccentricity.
Computer Science Department Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et.
Image Morphing CSC320: Introduction to Visual Computing
IIS for Image Processing Michael J. Watts
Recognition Part II Ali Farhadi CSE 455.
Last tuesday, you talked about active shape models Data set of 1,500 hand-labeled faces 20 facial features (eyes, eye brows, nose, mouth, chin) Train 40.
Face Detection and Recognition Computational Photography Derek Hoiem, University of Illinois Lecture by Kevin Karsch 12/3/13 Chuck Close, self portrait.
Image Synthesis Rabie A. Ramadan, PhD 3. 2 Our Problem.
Human-Computer Interaction Human-Computer Interaction Tracking Hanyang University Jong-Il Park.
CSE554AlignmentSlide 1 CSE 554 Lecture 5: Alignment Fall 2011.
University of Ioannina - Department of Computer Science Intensity Transformations (Point Processing) Christophoros Nikou Digital Image.
1 ECE 738 Paper presentation Paper: Active Appearance Models Author: T.F.Cootes, G.J. Edwards and C.J.Taylor Student: Zhaozheng Yin Instructor: Dr. Yuhen.
Course 9 Texture. Definition: Texture is repeating patterns of local variations in image intensity, which is too fine to be distinguished. Texture evokes.
The Quotient Image: Class-based Recognition and Synthesis Under Varying Illumination T. Riklin-Raviv and A. Shashua Institute of Computer Science Hebrew.
Understanding Faces 12/1/15 Some slides from Amin Sadeghi, Lana Lazebnik, Silvio Savarese, Fei-Fei Li Chuck Close, self portrait Lucas by Chuck Close Detection,
Multimodal Interaction Dr. Mike Spann
Data-driven Methods: Faces : Computational Photography Alexei Efros, CMU, Fall 2012 Portrait of Piotr Gibas © Joaquin Rosales Gomez.
Visual Computing Computer Vision 2 INFO410 & INFO350 S2 2015
CS 691B Computational Photography
CS 691B Computational Photography Instructor: Gianfranco Doretto Data Driven Methods: Faces.
Point Distribution Models Active Appearance Models Compilation based on: Dhruv Batra ECE CMU Tim Cootes Machester.
Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.
Multimedia Programming 10: Image Morphing
Image Morphing © Zooface Many slides from Alexei Efros, Berkeley.
Data-driven Methods: Faces
IIS for Image Processing
LOCUS: Learning Object Classes with Unsupervised Segmentation
Computational Photography Derek Hoiem, University of Illinois
Recognition: Face Recognition
Dynamical Statistical Shape Priors for Level Set Based Tracking
Common Classification Tasks
Data-driven Methods: Faces
Outline S. C. Zhu, X. Liu, and Y. Wu, “Exploring Texture Ensembles by Efficient Markov Chain Monte Carlo”, IEEE Transactions On Pattern Analysis And Machine.
Computational Photography
Brief Review of Recognition + Context
Presentation transcript:

Figure-centric averages Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images.

More by Jason Salavon More at:

“100 Special Moments” by Jason Salavon Why blurry?

Computing Means Two Requirements: Alignment of objects Objects must span a subspace Useful concepts: Subpopulation means Deviations from the mean

Images as Vectors = m n n*m

Vector Mean: Importance of Alignment = m n n*m = ½ + ½ = mean image

How to align faces?

Shape Vector = 43 Provides alignment!

Average Face 1. Warp to mean shape 2. Average pixels

Objects must span a subspace (1,0) (0,1) (.5,.5)

Example Does not span a subspace mean

Subpopulation means Examples: Happy faces Young faces Asian faces Etc. Sunny days Rainy days Etc. Average male Average female

Deviations from the mean - = Image X Mean X  X = X - X

Deviations from the mean += = X  X = X - X

Manipulating Facial Appearance through Shape and Color Duncan A. Rowland and David I. Perrett St Andrews University IEEE CG&A, September 1995

Face Modeling Compute average faces (color and shape) Compute deviations between male and female (vector and color differences)

Changing gender Deform shape and/or color of an input face in the direction of “more female” original shape colorboth

Enhancing gender more same original androgynous more opposite

Changing age Face becomes “rounder” and “more textured” and “grayer” original shape colorboth

Back to the Subspace

Linear Subspace: convex combinations Any new image X can be obtained as weighted sum of stored “basis” images. Our old friend, change of basis! What are the new coordinates of X?

The Morphable Face Model The actual structure of a face is captured in the shape vector S = (x 1, y 1, x 2, …, y n ) T, containing the (x, y) coordinates of the n vertices of a face, and the appearance (texture) vector T = (R 1, G 1, B 1, R 2, …, G n, B n ) T, containing the color values of the mean-warped face image. Shape S Appearance T

The Morphable face model Again, assuming that we have m such vector pairs in full correspondence, we can form new shapes S model and new appearances T model as: If number of basis faces m is large enough to span the face subspace then: Any new face can be represented as a pair of vectors (  1,  2  m ) T and (  1,  2  m ) T !

Using 3D Geometry: Blinz & Vetter, 1999 show SIGGRAPH video

Computer Science Erik Learned-Miller Joint Alignment: What’s It Good For?

26 Learned-Miller Congealing (CVPR 2000, PAMI 2006)

27 Learned-Miller Five Applications  Image factorizations For transfer learning, learning from one example  Alignment for Data Pooling 3D MR registration EEG registration  Artifact removal Magnetic resonance bias removal  Improvements to recognition algorithms Alignment before recognition  Defining anchor points for registration Find highly repeatable regions for future registrations

28 Learned-Miller Congealing  Process of joint “alignment” of sets of arrays (samples of continuous fields).  3 ingredients A set of arrays in some class A parameterized family of continuous transformations A criterion of joint alignment

29 Learned-Miller Congealing Binary Digits  3 ingredients A set of arrays in some class: Binary images A parameterized family of continuous transformations: Affine transforms A criterion of joint alignment: Entropy minimization

30 Learned-Miller Criterion of Joint Alignment  Minimize sum of pixel stack entropies by transforming each image. A pixel stack

31 Learned-Miller Observed Image “Latent Image” Transform (Previous work by Grenander,, Frey and Jojic.) An Image Factorization

32 Learned-Miller A pixel stack

33 Learned-Miller The Independent Pixel Assumption  Model assumes independent pixels  A poor generative model: True image probabilities don’t match model probabilities. Reason: heavy dependence of neighboring pixels.  However! This model is great for alignment and separation of causes! Why? Relative probabilities of “better aligned” and “worse aligned” are usually correct.  Once components are separated, a more accurate (and computationally expensive) model can be used to model each component.

34 Learned-Miller BeforeAfter Each pair implicitly creates a sample of the transform T. Congealing A transform

35 Learned-Miller Character Models Latent Images Transforms Image Kernel Density Estimator (or other estimator) Transform Kernel Density Estimator (CVPR 2003) Latent Image Probability Density for Zeroes P(I L ) Transform Probability Density for Zeroes P(T) Congealing

36 Learned-Miller How do we line up a new image? Sequence of successively “sharper” models … step 0 step 1 step N … Take one gradient step with respect to each model.

37 Learned-Miller Digit Models from One Example