1. 1 Zafer BarutcuogluPrinceton University Christopher DeCoro Hierarchical Shape Classification Using Bayesian Aggregation

2. 2 Shape Matching Given two shapes, quantify the difference between themGiven two shapes, quantify the difference between them –Useful for search and retrieval, image processing, etc. Common approach is that of shape descriptorsCommon approach is that of shape descriptors –Map arbitrary definition of shape into a representative vector –Define a distance measure (i.e Euclidean) to quantify similarity –Examples include: GEDT, SHD, REXT, etc. A common application is classificationA common application is classification –Given an example, and a set of classes, which class is most appropriate for that example? –Applicable to a large range of applications

3. 3 Hierarchical Classification Given a hierarchical set of classes,Given a hierarchical set of classes, And a set of labeled examples for those classesAnd a set of labeled examples for those classes Predict the hierarchically-consistent classification of a novel example, using the hierarchy to improve performance.Predict the hierarchically-consistent classification of a novel example, using the hierarchy to improve performance. Example courtesy of “The Princeton Shape Benchmark”, P. Shilane et. al (2004)

4. 4 MotivationMotivation Given these, how can we predict classes for novel shapes?Given these, how can we predict classes for novel shapes? Conventional algorithms don’t apply directly to hierarchiesConventional algorithms don’t apply directly to hierarchies –Binary classification –Multi-class (one-of-M) classification Using binary classification for each class can produce predictions which contradict with the hierarchyUsing binary classification for each class can produce predictions which contradict with the hierarchy Using multi-class classification over the leaf nodes loses information by ignoring the hierarchyUsing multi-class classification over the leaf nodes loses information by ignoring the hierarchy

5. 5 Other heirarchical classification methods, other domains TO ZAFER: I need something here about background information, other methods, your method, etc.TO ZAFER: I need something here about background information, other methods, your method, etc. Also, Szymon suggested a slide about conditional probabilities and bayes nets in general. Could you come up with something very simplified and direct that would fit with the rest of the presentation?Also, Szymon suggested a slide about conditional probabilities and bayes nets in general. Could you come up with something very simplified and direct that would fit with the rest of the presentation?

6. 6 Motivation (Example) Independent classifiers give an inconsistent predictionIndependent classifiers give an inconsistent prediction –Classified as bird, but not classified as flying creature Also cause incorrect resultsAlso cause incorrect results –Not classified as flying bird –Incorrectly classified as dragon

7. 7 Motivation (Example) We can correct this using our Bayesian Aggregation methodWe can correct this using our Bayesian Aggregation method –Remove inconsistency at flying creature Also improves results of classificationAlso improves results of classification –Stronger prediction of flying bird –No longer classifies as dragon

8. 8 Naïve Hierarchical Consistency biped NO human YES animal YES INDEPENDENTTOP-DOWN biped NO human YES animal YES BOTTOM-UP biped NO human YES animal YES Unfair distribution of responsibility and correction

9. 9 Our Method – Bayesian Aggregation Evaluate individual classifiers for each classEvaluate individual classifiers for each class –Inconsistent predictions allowed –Any classification algorithm can be used (e.g. kNN) –Parallel evaluation Bayesian aggregation of predictionsBayesian aggregation of predictions –Inconsistencies resolved globally

10. 10 Our Method - Implementation Shape descriptor: Spherical Harmonic Descriptor *Shape descriptor: Spherical Harmonic Descriptor * –Converts shape into 512-element vector –Compared using Euclidean distance Binary classifier: k-Nearest NeighborsBinary classifier: k-Nearest Neighbors –Finds the k nearest labeled training examples –Novel example assigned to most common class Simple to implement, yet flexibleSimple to implement, yet flexible * “Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors” M. Kazhdan, et. al (2003)

11. 11 superman bipedflying creature animal A Bayesian Framework y4y4 y2y2 y3y3 y1y1 g1g1 g3g3 g4g4 g2g2 Given predictions g 1...g N from kNN, find most likely true labels y 1...y N

12. 12 Classifier Output Likelihoods = α P(g 1...g N | y 1...y N ) P(y 1...y N ) P(y1...yN | g1...gN) = α P(g 1...g N | y 1...y N ) P(y 1...y N ) Conditional independence assumptionConditional independence assumption –Classifiers outputs depend only on their true labels –Given its true label, an output is conditionally independent of all other labels and outputs P(g 1...g N | y 1...y N ) =  i P(g i | y i )

13. 13 Estimating P(g i | y i ) Negative examples Positive examples #(g=0,y=0)#(g=1,y=0) #(g=0,y=1)#(g=1,y=1) Predicted negative Predicted positive The Confusion Matrix obtained using cross-validation e.g. P(g=0 | y=0) ≈ #(g=0,y=0) / [ #(g=0,y=0) + #(g=1,y=0) ]

14. 14 Hierarchical Class Priors = α P(g 1...g N | y 1...y N ) P(y 1...y N ) P(y1...yN | g1...gN) = α P(g 1...g N | y 1...y N ) P(y 1...y N ) Hierarchical dependency modelHierarchical dependency model –Class prior depends only on children P(y 1...y N ) =  i P(y i | y children(i) ) Enforces hierarchical consistencyEnforces hierarchical consistency –The probability of an inconsistent assignment is 0 –Bayesian inference will not allow inconsistency

15. 15 Conditional Probabilities P(y i | y children(i) )P(y i | y children(i) ) –Inferred from known labeled examples P(g i | y i )P(g i | y i ) –Inferred by validation on held-out data y4y4 y2y2 y3y3 y1y1 g1g1 g3g3 g4g4 g2g2 We can now apply Bayesian inference algorithmsWe can now apply Bayesian inference algorithms –Particular algorithm independent of our method –Results in globally consistent predictions –Uses information present in hierarchy to improve predictions

16. 16 Applying Bayesian Aggregation Training phase produces Bayes NetworkTraining phase produces Bayes Network –From hierarchy and training set, train classifiers –Use cross-validation to generate conditional probabilities –Use probabilities to create bayes net Test phase give probabilities for novel examplesTest phase give probabilities for novel examples –For a novel example, apply classifiers –Use classifier outputs and existing bayes net to infer probability of membership in each class HierarchyClassifiersBayes NetCross-validation ClassifiersBayes NetClass Probabilities Training Set Test Example

17. 17 Experimental Results 2-fold cross-validation on each class using kNN2-fold cross-validation on each class using kNN Area Under the ROC Curve (AUC) for evaluationArea Under the ROC Curve (AUC) for evaluation –Real-valued predictor can be thresholded arbitrarily –Probability that pos. example is predicted over a neg. example 169 of 170 classes were improved by our method169 of 170 classes were improved by our method –Average  AUC = +0.137 (+19% of old AUC) –Old AUC =.7004 (27 had AUC of 0.5, random guessing)

18. 18 AUC Scatter Plot

19. 19 AUC Changes 169 of 170 classes were improved by our method169 of 170 classes were improved by our method –Average  AUC = +0.137 (+19% of old AUC) –Old AUC =.7004 (27 had AUC of 0.5, random guessing)

20. 20 QuestionsQuestions