The content of these slides by John Galeotti, © 2012 Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN276201000580P,

Slides:



Advertisements
Similar presentations
EigenFaces and EigenPatches Useful model of variation in a region –Region must be fixed shape (eg rectangle) Developed for face recognition Generalised.
Advertisements

The content of these slides by John Galeotti, © 2013 Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
Interest points CSE P 576 Ali Farhadi Many slides from Steve Seitz, Larry Zitnick.
Lecture 12 Level Sets & Parametric Transforms sec & ch
Course Syllabus 1.Color 2.Camera models, camera calibration 3.Advanced image pre-processing Line detection Corner detection Maximally stable extremal regions.
This work by John Galeotti and Damion Shelton, © , was made possible in part by NIH NLM contract# HHSN P, and is licensed under a Creative.
Course Syllabus 1.Color 2.Camera models, camera calibration 3.Advanced image pre-processing Line detection Corner detection Maximally stable extremal regions.
Facial feature localization Presented by: Harvest Jang Spring 2002.
This work by John Galeotti and Damion Shelton, © , was made possible in part by NIH NLM contract# HHSN P, and is licensed under a Creative.
Each pixel is 0 or 1, background or foreground Image processing to
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #20.
© 2003 by Davi GeigerComputer Vision September 2003 L1.1 Face Recognition Recognized Person Face Recognition.
Face Recognition Jeremy Wyatt.
3D Geometry for Computer Graphics
Computer Vision I Instructor: Prof. Ko Nishino. Today How do we recognize objects in images?
2007Theo Schouten1 Morphology Set theory is the mathematical basis for morphology. Sets in Euclidic space E 2 (or rather Z 2 : the set of pairs of integers)
The content of these slides by John Galeotti, © 2008 to 2015 Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
CS 485/685 Computer Vision Face Recognition Using Principal Components Analysis (PCA) M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive.
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
Lecture 6 Linear Processing ch. 5 of Machine Vision by Wesley E
SVD(Singular Value Decomposition) and Its Applications
Dimensionality Reduction: Principal Components Analysis Optional Reading: Smith, A Tutorial on Principal Components Analysis (linked to class webpage)
Probability of Error Feature vectors typically have dimensions greater than 50. Classification accuracy depends upon the dimensionality and the amount.
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
Multimodal Interaction Dr. Mike Spann
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
BACKGROUND LEARNING AND LETTER DETECTION USING TEXTURE WITH PRINCIPAL COMPONENT ANALYSIS (PCA) CIS 601 PROJECT SUMIT BASU FALL 2004.
Chapter 9.  Mathematical morphology: ◦ A useful tool for extracting image components in the representation of region shape.  Boundaries, skeletons,
CSE554AlignmentSlide 1 CSE 554 Lecture 5: Alignment Fall 2011.
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
1 Recognition by Appearance Appearance-based recognition is a competing paradigm to features and alignment. No features are extracted! Images are represented.
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
Chapter 3 cont’d. Binary Image Analysis. Binary image morphology (nonlinear image processing)
Face Recognition: An Introduction
The content of these slides by John Galeotti, © Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P,
ECE 8443 – Pattern Recognition LECTURE 08: DIMENSIONALITY, PRINCIPAL COMPONENTS ANALYSIS Objectives: Data Considerations Computational Complexity Overfitting.
CSE 185 Introduction to Computer Vision Face Recognition.
CSSE463: Image Recognition Day 27 This week This week Today: Applications of PCA Today: Applications of PCA Sunday night: project plans and prelim work.
Last update Heejune Ahn, SeoulTech
References Books: Chapter 11, Image Processing, Analysis, and Machine Vision, Sonka et al Chapter 9, Digital Image Processing, Gonzalez & Woods.
CS654: Digital Image Analysis
1 Mathematic Morphology used to extract image components that are useful in the representation and description of region shape, such as boundaries extraction.
An Improved Approach For Image Matching Using Principle Component Analysis(PCA An Improved Approach For Image Matching Using Principle Component Analysis(PCA.
Morphological Image Processing Robotics. 2/22/2016Introduction to Machine Vision Remember from Lecture 12: GRAY LEVEL THRESHOLDING Objects Set threshold.
ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 10: PRINCIPAL COMPONENTS ANALYSIS Objectives:
Morphological Image Processing (Chapter 9) CSC 446 Lecturer: Nada ALZaben.
Face detection and recognition Many slides adapted from K. Grauman and D. Lowe.
CSSE463: Image Recognition Day 25 This week This week Today: Applications of PCA Today: Applications of PCA Sunday night: project plans and prelim work.
Content Based Coding of Face Images
(CMU ECE) : (CMU BME) : BioE 2630 (Pitt)
LECTURE 10: DISCRIMINANT ANALYSIS
Recognition: Face Recognition
Lecture 13 Shape ch. 9, sec. 1-8, of Machine Vision by Wesley E
In summary C1={skin} C2={~skin} Given x=[R,G,B], is it skin or ~skin?
CS Digital Image Processing Lecture 5
Lecture 16: Morphology (ch 7) & Image Matching (ch 13) ch. 7 and ch
CS4670: Intro to Computer Vision
CS4670: Intro to Computer Vision
Midterm Exam Closed book, notes, computer Similar to test 1 in format:
LECTURE 09: DISCRIMINANT ANALYSIS
Where are we? We have covered: Project 1b was due today
Midterm Exam Closed book, notes, computer Similar to test 1 in format:
Lecture 14 Shape ch. 9, sec. 1-8, of Machine Vision by Wesley E
(CMU ECE) : (CMU BME) : BioE 2630 (Pitt)
The “Margaret Thatcher Illusion”, by Peter Thompson
Presentation transcript:

The content of these slides by John Galeotti, © 2012 Carnegie Mellon University (CMU), was made possible in part by NIH NLM contract# HHSN P, and is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License. To view a copy of this license, visit or send a letter to Creative Commons, 171 2nd Street, Suite 300, San Francisco, California, 94105, USA. Permissions beyond the scope of this license may be available either from CMU or by ing The most recent version of these slides may be accessed online via Lecture 14 Morphology (ch 7) & Image Matching (ch 13) ch. 7 and ch. 13 of Machine Vision by Wesley E. Snyder & Hairong Qi Spring 2012 BioE 2630 (Pitt) : (CMU RI) (CMU ECE) : (CMU BME) Dr. John Galeotti

Mathematical Morphology  The study of shape…  Using Set Theory  Most easily understood for binary images. 2

Binary Morphology: Basic Idea 1.Make multiple copies of a shape 2.Translate those copies around 3.Combine them with either their:  Union, , in the case of dilation,   Intersection, , in the case of erosion,  3 Dilation makes things bigger Erosion makes things smaller

Binary Morphology: Basic Idea  Q: How do we designate:  The number of copies to make?  The translation to apply to each copy?  A: With a structuring element (s.e.)  A (typically) small binary image.  We will assume the s.e. always contains the origin.  For each marked pixel in the s.e.:  Make a new copy of the original image  Translate that new copy by the coordinates of the current pixel in the s.e. 4

Dilation Example 5 =  f B = s.e fAfA A = { (2,8),(3,8),(7,8),(8,8),(5,6),(2,4),(3,4 ),(3,3),(4,3),(5,3),( 6,3),(7,3),(7,4),(8, 4) } B = { (0,0),(0,-1) }

Erosion Example  For erosion, we translate by the negated coordinates of the current pixel in the s.e. 6 =  f B = s.e fAfA =

Notation  A (binary) image: f A  The set of marked pixels in f A : A  A = { ( x 1,y 1 ), ( x 2,y 2 ), … }  A translated image or set: f A (dx,dy) or A (dx,dy)  The number of elements in A: #A  Complement (inverse) of A: A c  Reflection (rotation) of A: Ã  Ã = { ( -x,-y ) | ( x,y )  A }

Properties  Dilation:  Commutative, Associative, & Distributive  Increasing: If A  B then A  K  B  K  Extensive: A  A  B  Erosion:  Anti-extensive (A  B  A), … (see the text)  Duality:  (A  B) c = A c  B  (A  B) c = A c  B  Not Inverses:  A ≠ (A  B)  B  A ≠ (A  B)  B 8 ~ ~ This is actually the opening of A by B This is actually the closing of A by B

Opening  f A o f B = ( f A  f B )  f B  Preserves the geometry of objects that are “big enough”  Erases smaller objects  Mental Concept:  “Pick up” the s.e. and place it in f A.  Never place the s.e. anywhere it covers any pixels in f A that are not marked.  f A o f B = the set of (marked) pixels in f A which can be covered by the s.e. 9

Opening Example  Use a horizontal s.e. to remove 1-pixel thick vertical structures: Erosion Dilation

Gray-Scale Morphology  Morphology operates on sets  Binary images are just a set of marked pixels  Gray-scale images contain more information  How can we apply morphology to this extra intensity information?  We need to somehow represent intensity as elements of a set 11

 Gray-scale morphology operates on the umbra of an image.  Imagine a 2D image as a pixilated surface in 3D  We can also “pixilate” the height of that surface  The 2D image is now a 3D surface made of 3D cells 12 The Umbra The umbra of a 1D image The umbra of a 1D s.e Dilation

 Records at each pixel the distance from that pixel to the nearest boundary (or to some other feature).  Used by other algorithms  The DT is a solution of the Diff. Eq.: ||  DT(x) || = 1, DT(x) = 0 on boundary  Can compute using erosion  DT(x) = iteration when x disappears  Details in the book The Distance Transform (DT) DT of a region’s interior

Voronoi Diagram  Divides space  Related to DT  Q: To which of a set of regions (or points) is this point the closest?  Voronoi Diagram’s boundaries = points that are equi-distant from multiple regions  Voronoi Domain of a region = the “cell” of the Voronoi Diagram that contains the region  Details in the text 14 The voronoi diagram of a set of 10 points is public domain from:

Imaging Matching (ch. 13)  Matching iconic images  Matching graph-theoretic representations  Most important:  Eigenimages  Springs & Templates 15

Template Matching  Template ≈ a relatively small reference image for some feature we expect to see in our input image.  Typical usage: Move the template around the input image, looking for where it “matches” the best (has the highest correlation).  Rotation & scale can be problematic  Often require multiple passes if they can’t be ruled out a-priori  How “big” do we make each template?  Do we represent small, simple features  Or medium-size, more complex structures? 16

Eigenimages  Goal: Identify an image by comparing it to a database of other images  Problem: Pixel-by-pixel comparisons are two expensive to run across a large database  Solution: Use PCA 17

 Big Picture: Fitting a hyper-ellipsoid & then (typically) reducing dimensionality by flattening the shortest axes  Same as fitting an (N+1)-dimensional multivariate Gaussian, and then taking the level set corresponding to one standard deviation  Mathematically, PCA reduces the dimensionality of data by mapping it to the first n eigenvectors (principal components) of the data’s covariance matrix  The first principal component is the eigenvector with the largest eigenvalue and corresponds to the longest axis of the ellipsoid  The variance along an eigenvector is exactly the eigenvector’s eigenvalue  This is VERY important and VERY useful. Any questions? 18 PCA (K-L Expansion) x’ y’ b1b1 b2b2 x y

Eigenimages: Procedure  Run PCA on the training images  See the text for efficiency details  Store in the database:  The set of dominant Eigenvectors  = the principle components  = the Eigenimages  For each image, store its coefficients when projected onto the Eigenimages  Match a new image:  Project it onto the basis of the Eigenimages  Compare the resulting coefficients to those stored in the database. 19

20 Eigenimages Example Eigenimages / EigenfacesTraining Images PCA New Images: Project Onto The face database and the derived Eigenface examples are all from AT&T Laboratories Cambridge: & Which training image(s) does each face most resemble?

Matching Simple Features  Classification based on features  Ex: mean intensity, area, aspect ratio  Idea:  Combine a set of shape features into a single feature vector  Build a statistical model of this feature vector between and across object classes in a sequence of training shapes  Classification of a new shape = the object class from which the new shape’s feature vector most likely came. 21

One of two maximal cliques Graph Matching: Association Graphs  Match nodes of model to segmented patches in image  Maximal cliques represent the most likely correspondences  Clique = a totally connected subgraph  Problems: Over/under segmentation, how to develop appropriate rules, often > 1 maximal clique Model B C A Image 1A 2B 3C 3B 2C

 Idea: When matching simple templates, we usually expect a certain arrangement between them.  So, arrange templates using a graph structure.  The springs are allowed to deform, but only “so” much. 23 Graph Matching: Springs & Templates Eye Nose Left edge Right edge Top of head Mouth Fischler and Elschlager’s “Pictorial Structures” spring & template model for image matching from the early 1970s

 A match is based on minimizing a total cost.  Problem: Making sure missing a point doesn’t improve the score. 24 Graph Matching: Springs & Templates