Shape Classification Alex Yakubovich Elderlab Oct 7, 2011 John Wilder, Jacob Feldman, Manish Singh, Superordinate shape classification using natural shape statistics, Cognition, Volume 119, Issue 3, June 2011, Pages

Table of Contents Introduction Ground Truth (Psychophysics) Modeling – Feature Extraction – Classifier Design – Evaluation Analysis

Motivation Human object categories are hierarchical Superordinate Subordinate Basic

Superordinate Classification Superordinate classification is important – Knowing only rough category allows rapid response Nontrivial problem – Broad classes  large intra-class variability Humans do it well  rapid animal detection (Thorpe ‘02) – 20 ms: Animal or not? – 150 ms: which animal?

Parameterization of Shape Shape is the feature explored in this paper Many representations: “None have been quantitavely validated with respect to ecological shape categories” Fourier descriptors boundary moments shape grammarschain codes geons codons convex hull- deficiency shape matrices medial axis

Medial Axis Representation Proposed by Blum (1973) Encode shape using by its axis of symmetry (skeleton)

Medial Axis Representation Advantages – Psychophysical correlates (Kovacs & Julesz, ’98,’94) – Captures part structure of shapes  evidence that human representation relies on this (Barenholtz & Feldman ’03, Hoffman & Richards ’84…); – Amenable to computation – Stable - if computed using Bayesian estimator (Feldman & Sing ‘06)

Bayesian skeleton estimation (Feldman & Singh 2006) Problem: given a shape {x 1 …x N }, compute most probable skeleton Build a prior over the skeletons Assumptions. – Shape is generated by K axes. – In each axis, successive turning angles are independent – axes (branches) sprout with equal probability

Prior Model Axes (branches) Turning Angle Points on axis

Likelihood Model

Contributions Task: Classify salient contour as one of {animal, leaf}. Evaluate human performance Design and optimize Naïve Bayes Classifier, using the medial axis parameters as features Compare performance against ground truth.

Shape databases a)Brown LEMS lab animal database b)Smithsonian leaves database

Psychophysics When establishing ground truth, subjects shouldn’t use overt recognition To guarantee this, morph animal & leaf shapes into ‘blobs’ 100% animal 0% animal 250 animals, 250 leaves Select 2 randomly Sample equally (n=150) Align (match principal axes) Weighted average of matching points Proportion animal = {.3,.4,.5,.6,.7}

Psychophysics Independen t variables Dependent variable P(animal response) weights skeletal parameters Each subject sees 500 blobs – Random Pairing/weights Which is more “likely” to be animal or leaf?

Subject classification vs. Mixture proportion “subjects were both consistent and effective at recovering the true source of the morphed shape”

Psychophysics Remark: Evidence that mechanism for rapid classification (feed-forward) is distinct from slower processing (feedback) Repeated experiment with shorter stimulus durations, and mask following stimuli Same conclusion

Feature Selection Possible parameters of MAP skeleton: Par 1: # of branches Par 2: Max depth of skeleton Par 3: mean depth Par 6: mean length of axes / root

Feature Selection To avoid redundancy, only consider parameters whose distributions differ between the two classes – Wilcoxon rank-sum test (α=.01)

Feature Selection Par 1: # of branches Par 2: max depth of skeleton Par 3: mean depth Par 6: mean length of axes / root

Classifier Design Bayesian analysis – For shape parameter x i :

Classifier Design To pool information from k parameters, assume independence: Decision Rule:

Model Selection Reduce number of parameters using AIC: best model (according to AIC)={1,8} – X1 = # skeletal branches – X8 = total signed turning angle Agrees with BIC Max likelihood for given model # model parameters

Training The naïve Bayes classifier is trained on “unadulterated” shapes The best model reached 81% classification accuracy How will humans perform on such shapes? – Run another test group on shapes with weights = {0,1} – 88.8 ± 0.3 % accuracy

Evaluation Classifier is evaluated on database of morphed shapes Good agreement with ground truth “the human subjective judgment of “animalishness” correspond closely to the Bayesian estimate of the probability of the animal class”

Evaluation Good fit with few parameters  parameters well chosen Can we do better by using non skeletal-parameters? Options: – Aspect Ratio – Compactness (perimeter 2 /area) – Symmetry Hausdorff distance between two halves of shape (min. over all reflection axes) – Contour complexity measure (Feldman & Singh ‘05) Repeat earlier analysis

Evaluation Significant difference between animal/leaf classes for all 4 parameters (Wilcoxon rank- sum test)

Evaluation Train classifier over non-skeletal parameters Test: compare to human judgment Only complexity measure fits significantly, but the fit is worse than with skeletal parameters – R 2 = 0.55 < 0.71

Conclusion “subjects’ classification of shapes into our 2 natural categories can be well modeled by a Bayesian classifier with a very small number of shape skeletal parameters, in which the model’s assumptions are consistent with the empirical distribution in naturally-occurring shapes.” Classification (leafishness vs. animalishness) is done by applying stereotypes of shapes in each category Call for ‘naturalization’ of shape representations