Recognizing Deformable Shapes Salvador Ruiz Correa (CSE/EE576 Computer Vision I)

Slides:



Advertisements
Similar presentations
Distinctive Image Features from Scale-Invariant Keypoints
Advertisements

Feature extraction: Corners
Distinctive Image Features from Scale-Invariant Keypoints David Lowe.
Chapter 2: Marr’s theory of vision. Cognitive Science  José Luis Bermúdez / Cambridge University Press 2010 Overview Introduce Marr’s distinction between.
Road-Sign Detection and Recognition Based on Support Vector Machines Saturnino, Sergio et al. Yunjia Man ECG 782 Dr. Brendan.
RGB-D object recognition and localization with clutter and occlusions Federico Tombari, Samuele Salti, Luigi Di Stefano Computer Vision Lab – University.
Classification and Feature Selection for Craniosynostosis Shulin Yang, Linda G. Shapiro, Michael L. Cunningham, Matthew Speltz, Su-In Lee University of.
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Fitting: The Hough transform. Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not.
Computer Vision for Human-Computer InteractionResearch Group, Universität Karlsruhe (TH) cv:hci Dr. Edgar Seemann 1 Computer Vision: Histograms of Oriented.
Computer Vision Detecting the existence, pose and position of known objects within an image Michael Horne, Philip Sterne (Supervisor)
Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.
Adviser:Ming-Yuan Shieh Student:shun-te chuang SN:M
Intelligent Systems Lab. Extrinsic Self Calibration of a Camera and a 3D Laser Range Finder from Natural Scenes Davide Scaramuzza, Ahad Harati, and Roland.
A Versatile Depalletizer of Boxes Based on Range Imagery Dimitrios Katsoulas*, Lothar Bergen*, Lambis Tassakos** *University of Freiburg **Inos Automation-software.
Fitting: The Hough transform
1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.
A Study of Approaches for Object Recognition
Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.
UPM, Faculty of Computer Science & IT, A robust automated attendance system using face recognition techniques PhD proposal; May 2009 Gawed Nagi.
3-D Object Recognition From Shape Salvador Ruiz Correa Department of Electrical Engineering.
Technology Project: Shape-Based Retrieval of 3D Craniofacial Data PI: Linda Shapiro, Ph.D. Key Personnel: James Brinkley, M.D., Ph.D. Michael Cunningham,
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Object Recognition Using Distinctive Image Feature From Scale-Invariant Key point D. Lowe, IJCV 2004 Presenting – Anat Kaspi.
1 High-Level Computer Vision Detection of classes of objects (faces, motorbikes, trees, cheetahs) in images Detection of classes of objects (faces, motorbikes,
1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.
Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
Spatial Pyramid Pooling in Deep Convolutional
Oral Defense by Sunny Tang 15 Aug 2003
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
3D Models and Matching representations for 3D object models
AdvisorStudent Dr. Jia Li Shaojun Liu Dept. of Computer Science and Engineering, Oakland University 3D Shape Classification Using Conformal Mapping In.
Computer vision.
Recognizing Deformable Shapes Salvador Ruiz Correa Ph.D. Thesis, Electrical Engineering.
ENT 273 Object Recognition and Feature Detection Hema C.R.
Building local part models for category-level recognition C. Schmid, INRIA Grenoble Joint work with G. Dorko, S. Lazebnik, J. Ponce.
HOUGH TRANSFORM Presentation by Sumit Tandon
Intelligent Vision Systems ENT 496 Object Shape Identification and Representation Hema C.R. Lecture 7.
Generalized Hough Transform
Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao.
Recognizing Deformable Shapes Salvador Ruiz Correa UW Ph.D. Graduate Researcher at Children’s Hospital.
Object Recognition in Images Slides originally created by Bernd Heisele.
Features-based Object Recognition P. Moreels, P. Perona California Institute of Technology.
A Two-level Pose Estimation Framework Using Majority Voting of Gabor Wavelets and Bunch Graph Analysis J. Wu, J. M. Pedersen, D. Putthividhya, D. Norgaard,
Recognizing Deformable Shapes Salvador Ruiz Correa (CSE/EE576 Computer Vision I)
Fitting: The Hough transform
Limitations of Cotemporary Classification Algorithms Major limitations of classification algorithms like Adaboost, SVMs, or Naïve Bayes include, Requirement.
CSE 185 Introduction to Computer Vision Feature Matching.
Methods for 3D Shape Matching and Retrieval
Local features: detection and description
9.913 Pattern Recognition for Vision Class9 - Object Detection and Recognition Bernd Heisele.
Robotics Chapter 6 – Machine Vision Dr. Amit Goradia.
Image features and properties. Image content representation The simplest representation of an image pattern is to list image pixels, one after the other.
Instructor: Mircea Nicolescu Lecture 5 CS 485 / 685 Computer Vision.
Hough Transform CS 691 E Spring Outline Hough transform Homography Reading: FP Chapter 15.1 (text) Some slides from Lazebnik.
Lecture 07 13/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Recognizing Deformable Shapes
Recognition: Face Recognition
Paper Presentation: Shape and Matching
3D Models and Matching particular matching techniques
Common Classification Tasks
Local Feature Extraction Using Scale-Space Decomposition
Brief Review of Recognition + Context
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Scale-Space Representation for Matching of 3D Models
George Bebis and Wenjing Li Computer Vision Laboratory
Recognizing Deformable Shapes
Presented by Xu Miao April 20, 2005
Human-object interaction
Presentation transcript:

Recognizing Deformable Shapes Salvador Ruiz Correa (CSE/EE576 Computer Vision I)

Goal We are interested in developing algorithms for recognizing and classifying deformable object shapes from range data. We are interested in developing algorithms for recognizing and classifying deformable object shapes from range data. 3-D Output Surface Mesh 3-D Laser Scanner Input3-DObject This is a difficult problem that is relevant in several application fields. This is a difficult problem that is relevant in several application fields. Rangedata (Cloud of 3-D points) Post-processing

Applications Computer Vision: Computer Vision: - Scene analysis - Industrial Inspection - Robotics Medical Diagnosis: Medical Diagnosis: - Classification and - Detection of craniofacial deformations.

Basic Idea Generalize existing numeric surface representations for matching 3-D objects to the problem of identifying shape classes. Generalize existing numeric surface representations for matching 3-D objects to the problem of identifying shape classes.

Main Contribution An algorithmic framework based on symbolic shape descriptors that are robust to deformations as opposed to numeric descriptors that are often tied to specific shapes. An algorithmic framework based on symbolic shape descriptors that are robust to deformations as opposed to numeric descriptors that are often tied to specific shapes.

What Kind Of Deformations? Toy animals 3-D Faces Normal Mandibles Neurocranium Normal Abnormal Shape classes: significant amount of intra-class variability

Deformed Infants’ Skulls Sagittal Coronal Normal Sagittal Synostosis Bicoronal Synostosis Fused Sutures Metopic Occurs when sutures of the cranium fuse prematurely (synostosis).

More Craniofacial Deformations Facial Asymmetry Sagittal Synostosis Bicoronal Synostosis Unicoronal Synostosis Metopic Synostosis

Alignment-verification Alignment-verification 3-D Range Scene Recognized Models Objects Database Find correspondences using numeric signature information. Estimate candidate transformations. Verification process selects the transformation that produces the best alignment. Models

Alignment-Verification Limitations The approach does not extend well to the problem of identifying classes of similar shapes. In general: of identifying classes of similar shapes. In general: Numeric shape representations are not robust to deformations. Numeric shape representations are not robust to deformations. There are not exact correspondences between model and scene. There are not exact correspondences between model and scene. Objects in a shape class do not align. Objects in a shape class do not align.

Component-Based Methodology NumericSignatures Components SymbolicSignatures ArchitectureofClassifiers + Recognition And Classification Of Deformable Shapes Overcomes the limitations of the alignment-verification approach define Describespatialconfiguration

Recognition Problem (1) We are given a set of surface meshes {C 1,C 2,…,C n } which are random samples of two shape classes C We are given a set of surface meshes {C 1,C 2,…,C n } which are random samples of two shape classes C C1C1 C2C2 CkCk … CnCn …

Recognition Problem (2) The problem is to use the given meshes and labels to construct an algorithm that determines whether shape class members are present in a single view range scene. The problem is to use the given meshes and labels to construct an algorithm that determines whether shape class members are present in a single view range scene.

Classification Problem (1) We are given a set of surface meshes {C 1,C 2,…,C n } which are random samples of two shape classes C +1 and C -1, We are given a set of surface meshes {C 1,C 2,…,C n } which are random samples of two shape classes C +1 and C -1, where each surface mesh is labeled either by +1 or -1. where each surface mesh is labeled either by +1 or Normal Skulls C +1 Abnormal Skulls C -1 C1C1 C2C2 CkCk C k+1 C k+2 … CnCn …

Classification Problem (2) The problem is to use the given meshes and labels to construct an algorithm that predicts the label of a new surface mesh C new. The problem is to use the given meshes and labels to construct an algorithm that predicts the label of a new surface mesh C new. Is this skull normal (+1) or abnormal (-1)? C new

Classification Problem (3) We also consider the case of “missing” information: We also consider the case of “missing” information: 3-D Range Scene Single View Shape class of normal heads (+1) Shape class of abnormal heads (-1) Are these heads normal or abnormal? Clutter and Occlusion

Assumptions All shapes are represented as oriented surface meshes of fixed resolution. All shapes are represented as oriented surface meshes of fixed resolution. The vertices of the meshes in the training set are in full correspondence. The vertices of the meshes in the training set are in full correspondence. Finding full correspondences : hard problem yes … but it is approachable ( use morphable models technique: Blantz and Vetter, SIGGRAPH 99; C. R. Shelton, IJCV, 2000; Allen et al., SIGGRAPH 2003). Finding full correspondences : hard problem yes … but it is approachable ( use morphable models technique: Blantz and Vetter, SIGGRAPH 99; C. R. Shelton, IJCV, 2000; Allen et al., SIGGRAPH 2003).

Four Key Elements To Our Approach Architecture of Classifiers Numeric Signatures Components Symbolic Signatures Recognition And Classification Of Deformable Shapes

Numeric Signatures ArchitectureofClassifiers NumericSignatures Components SymbolicSignatures Encode Local Surface Geometry of an Object

The Spin Image Signature P X n   P is the selected vertex. X is a contributing point of the mesh.  is the perpendicular distance from X to P’s surface normal.  is the signed perpendicular distance from X to P’s tangent plane. tangent plane at P

Spin Image Construction A spin image is constructed - about a specified oriented point o of the object surface - with respect to a set of contributing points C, which is controlled by maximum distance and angle from o. It is stored as an array of accumulators S( ,  ) computed via: For each point c in C(o) 1. compute  and  for c. 2. increment S ( ,  ) o

Numeric Signatures: Spin Images Rich set of surface shape descriptors. Rich set of surface shape descriptors. Their spatial scale can be modified to include local and non-local surface features. Their spatial scale can be modified to include local and non-local surface features. Representation is robust to scene clutter and occlusions. Representation is robust to scene clutter and occlusions. P Spin images for point P 3-D faces

Components NumericSignatures Components SymbolicSignatures ArchitectureofClassifiers Equivalent Numeric Signatures: Encode Local Geometry of a Shape Class define

How To Extract Shape Class Components? …… ComponentDetector ComputeNumericSignatures Training Set SelectSeedPoints RegionGrowingAlgorithm Grown components around seeds

Labeled Surface Mesh Selected 8 seed points by hand Component Extraction Example RegionGrowing Grow one region at the time (get one detector per component) Detected components on a training sample

How To Combine Component Information? … Extracted components on test samples Note: Numeric signatures are invariant to mirror symmetry; our approach preserves such an invariance.

Symbolic Signatures NumericSignatures Components SymbolicSignatures Architecture of Classifiers Encode Geometrical Relationships Among Components

Symbolic Signature Symbolic Signature at P Labeled Surface Mesh Matrix storing component labels EncodeGeometricConfiguration Critical Point P

Symbolic Signature Construction Critical Point P a P b P a b Project labels to tangent plane at P tangent plane Coordinate system defined up to a rotation Normal

Symbolic Signatures Are Robust To Deformations P Relative position of components is stable across deformations: experimental evidence

Architecture of Classifiers Numeric Signatures Components Symbolic Signatures Architecture of Classifiers Learns Components And Their Geometric Relationships

Proposed Architecture (Classification Example) Input LabeledMesh ClassLabel(Abnormal) Verify spatial configuration of the components IdentifySymbolicSignatures IdentifyComponents Two classification stages SurfaceMesh

At Classification Time (1) Bank of ComponentDetectors Surface Mesh AssignsComponentLabels Labeled Surface Mesh Multi-wayclassifier Identify Components

Labeled Surface Mesh Bank of SymbolicSignaturesDetectors Symbolic pattern for components 1,2,4 At Classification Time (2) Symbolic pattern for components 5,6,8 AssignsSymbolicLabels Two detectors

Confidence Level Finding Critical Points On Test Samples 0 Critical Point 1 +1 Margin associated with the component detector classifiers

Architecture Implementation ALL our classifiers are (off-the-shelf) ν - Support Vector Machines ( ν - SVMs) (Schölkopf et al., 2000 and 2001). ALL our classifiers are (off-the-shelf) ν - Support Vector Machines ( ν - SVMs) (Schölkopf et al., 2000 and 2001). Component (and symbolic signature) detectors are one-class classifiers. Component (and symbolic signature) detectors are one-class classifiers. Component label assignment: performed with a multi-way classifier that uses pairwise classification scheme. Component label assignment: performed with a multi-way classifier that uses pairwise classification scheme. Gaussian kernel. Gaussian kernel.

Experimental Validation Recognition Tasks: 4 (T1 - T4) Classification Tasks: 3 (T5 – T7) No. Experiments: 5470 Setup RecognitionClassification Laser Rotary Table

Shape Classes

Enlarging Training Sets Using Virtual Samples Displacement Vectors Originals Morphs Twist (5deg) + Taper - Push + Spherify (10%) Push +Twist (10 deg) +Scale (1.2) Original Global Morphing Operators Morphs Physical Modeling (14) University of WashingtonElectrical Engineering

Task 1: Recognizing Single Objects (1) No. Shape classes: 9. No. Shape classes: 9. Training set size: 400 meshes. Training set size: 400 meshes. Testing set size: 200 meshes. Testing set size: 200 meshes. No. Experiments: No. Experiments: No. Component detectors:3. No. Component detectors:3. No. Symbolic signature detectors: 1. No. Symbolic signature detectors: 1. Numeric signature size: 40x40. Numeric signature size: 40x40. Symbolic signature size: 20x20. Symbolic signature size: 20x20. No clutter and occlusion. No clutter and occlusion.

Task 1: Recognizing Single Objects (2) Snowman: 93%. Snowman: 93%. Rabbit: 92%. Rabbit: 92%. Dog: 89%. Dog: 89%. Cat: 85.5%. Cat: 85.5%. Cow: 92%. Cow: 92%. Bear: 94%. Bear: 94%. Horse: 92.7%. Horse: 92.7%. Human head: 97.7%. Human face: 76%. Recognition rates (true positives) (No clutter, no occlusion, complete models)

Tasks 2-3: Recognition In Complex Scenes (1) No. Shape classes: 3. No. Shape classes: 3. Training set size: 400 meshes. Training set size: 400 meshes. Testing set size: 200 meshes. Testing set size: 200 meshes. No. Experiments: No. Experiments: No. Component detectors:3. No. Component detectors:3. No. Symbolic signature detectors: 1. No. Symbolic signature detectors: 1. Numeric signature size: 40x40. Numeric signature size: 40x40. Symbolic signature size: 20x20. Symbolic signature size: 20x20. T2 – low clutter and occlusion. T2 – low clutter and occlusion.

Task 2-3: Recognition in Complex Scenes (2) ShapeClassTruePositivesFalsePositivesTruePositivesFalsePositives Snowmen91%31%87.5%28% Rabbit90.2%27.6%84.3%24% Dog89.6%34.6%88.12%22.1% Task 2 Task 3

Task 2-3: Recognition in Complex Scenes (3)

Task 4: Recognizing Human Heads (1) No. Shape classes: 1. No. Shape classes: 1. Training set size: 400 meshes. Training set size: 400 meshes. Testing set size: 250 meshes. Testing set size: 250 meshes. No. Experiments: 710. No. Experiments: 710. No. Component detectors:8. No. Component detectors:8. No. Symbolic signature detectors: 2. No. Symbolic signature detectors: 2. Numeric signature size: 70x70. Numeric signature size: 70x70. Symbolic signature size: 12x12. Symbolic signature size: 12x12.

Task 4: Recognizing Human Heads (2) Recognition Rate % Clutter % Clutter < 40 % Occlusion < 40 % Occlusion FP rate: ~1%, (44,0.9) (40,0.88)

Task 4: Recognizing Human Heads (3)

Task 5: Classifying Normal vs. Abnormal Human Heads (1) No. Shape classes: 6. No. Shape classes: 6. Training set size: 400 meshes. Training set size: 400 meshes. Testing set size: 200 meshes. Testing set size: 200 meshes. No. Experiments: No. Experiments: No. Component detectors:3. No. Component detectors:3. No. Symbolic signature detectors: 1. No. Symbolic signature detectors: 1. Numeric signature size: 50x50. Numeric signature size: 50x50. Symbolic signature size: 12x12. Symbolic signature size: 12x12.

Task 5: Classifying Normal vs. Abnormal Human Heads (1) ShapeClasses Classification Accuracy % Normal vs. Abnormal 1 98 Normal vs. Abnormal Abnormal 1 vs Abnormal 1 vs Abnormal 1 vs Full models Normal (convex combinations of Normal and Abnormal 1) 65%-35%50%-50%25%-75% Abnormal Five Cases

Task 6: Classifying Normal vs. Abnormal Human Heads In Complex Scenes(1) No. Shape classes: 2. No. Shape classes: 2. Training set size: 400 meshes. Training set size: 400 meshes. Testing set size: 200 meshes. Testing set size: 200 meshes. No. Experiments: No. Experiments: No. Component detectors:3. No. Component detectors:3. No. Symbolic signature detectors: 1. No. Symbolic signature detectors: 1. Numeric signature size: 100x100. Numeric signature size: 100x100. Symbolic signature size: 12x12. Symbolic signature size: 12x12.

Task 6: Classifying Normal vs. Abnormal Human Heads In Complex Scenes(1) ShapeClasses Classification Accuracy % Normal vs. Abnormal 1 88 RR full models (no clutter and occlusion) ~ 96% (250 experiments per class) Clutter < 15% and occlusion < 50% Range scenes – single view

Task 7: Classifying Normal vs. Abnormal Neurocranium (1) No. Shape classes: 2. No. Shape classes: 2. Training set size: 400 meshes. Training set size: 400 meshes. Testing set size: 200 meshes. Testing set size: 200 meshes. No. Experiments: No. Experiments: No. Component detectors:3. No. Component detectors:3. No. Symbolic signature detectors: 1. No. Symbolic signature detectors: 1. Numeric signature size: 50x50. Numeric signature size: 50x50. Symbolic signature size: 15x15. Symbolic signature size: 15x15.

Task 7: Classifying Normal vs. Abnormal Neurocranium (2) ShapeClasses Classificatio n Accuracy % Normal vs. Abnormal 89 No clutter and occlusion 100 Experiments Abnormal (sagittal synostosis ) Superimposed models Normal

Main Contributions (1) A novel symbolic signature representation of deformable shapes that is robust to intra-class variability and missing information, as opposed to a numeric representation which is often tied to a specific shape. A novel symbolic signature representation of deformable shapes that is robust to intra-class variability and missing information, as opposed to a numeric representation which is often tied to a specific shape. A novel kernel function for quantifying symbolic signature similarities. A novel kernel function for quantifying symbolic signature similarities.

Main Contributions (2) A region growing algorithm for learning shape class components. A region growing algorithm for learning shape class components. A novel architecture of classifiers for abstracting the geometry of a shape class. A novel architecture of classifiers for abstracting the geometry of a shape class. A validation of our methodology in a set of large scale recognition and classification experiments aimed at applications in scene analysis and medical diagnosis. A validation of our methodology in a set of large scale recognition and classification experiments aimed at applications in scene analysis and medical diagnosis.

Main Contributions (3) Our approach: Our approach: - Is general can be applied to a variety of shape classes. - Is robust to clutter and occlusion - It Works in practice - Is a step forward in 3-D object recognition research.