1. Constructing Category Hierarchies for Visual Recognition Marcin Marszaklek and Cordelia Schmid

2. Introduction Hierarchical classification scales well in the number of classes: – O(n^2): one-vs-one – O(n): one-vs-rest – O(log(n): classification tree Previous works to construct class hierarchies: – By hand [Zweig’07] – From external sources [Marszaklek’07] – From visual similarities: Exhaustive [Yuan’06] Top-down [Chen’04, Griffin’08] Botton-up [Zhigang’05, Griffin’08]

3. Motivation: Disjoint VS overlap Previous works: disjoint partitioning of classes (class separability) Increasingly difficult to disjoint partition for large number of classes. Propose: relaxed hierarchy – postpone uncertain classification decisions until the number of classes get reduced and learning good decision boundaries becomes tractable.

4. Method Building relaxed hierarchy Train top-down classifiers using hierarchy

5. Building top-down relaxed hierarchy Using balanced Normalized-cut, split the set of classes such that: Further relaxation: – Find the class on boundary – Define the split: (α: overlap ratio), given a partition

6. Building top-down relaxed hierarchy – conti.

7. Train/test top-down classifiers Training hierarchy: – For each node of DAG, samples of [L n \R n ] as positive sample, [R n \L n ] as negative samples; – Samples in classes X n =L n П R n not for training Testing: – Traversal DAG until a leaf is reached. – The decision is either directly the class label (leaves containing one class), or performing OAR classification on the remaining classes in current leaf.

8. Results: one-vs-rest High intra-class variability Confusion between mountain/touring bikes Low intra-class variability

9. Class hierarchies: caltech 256 relaxed hierarchy hand-crafted hierarchy Disjoint visual hierarchy Categories: animal, natural phenomena and man-made objects

10. Results Average per-class accuracy on Caltech-256

11. Results – conti. Complexity in the number of classes r: # relaxed training sample per class. Speed-for-accuracy trade-off

12. Learning and Using Taxonomies For Fast Visual Categorization Gregory Griffin and Pietro Perona

13. Motivation Given a testing sample: O( # category) O( log 2 (# category)) One-vs-rest strategy VS hierarchical strategy expensive inexpensive

14. Methods Building confusion matrix Building Taxonomies Re-train top-down classifiers

15. Building confusion matrix – Multi-class classification: one-vs-rest strategy – Classifier: Spatial Pyramid Matching; – Training data only and loo validation

16. Building Taxonomies Intuition: – Categories that are easily confused should be grouped together; – Decisions between easily-confused categories sholuld be taken later in the decision tree. Method: – Self-Tuning Spectral Clustering – Greedy, bottom-up grouping using mutual infor.

17. Re-train top-down classifiers Known the taxonomy tree of categories as a binary tree At each node, reformulating a binary-classifier Again, using Spatial Pyramid Matching + SVM F_{train} = 10%

18. Results Taxonomy tree for Caltech-256 Red: insects Yellow: birds Green: land mammals Blue: aquatic mammals

19. Trade-off between performance and speed Spectral clustering Greedy clustering A: ordinary one-vs-rest multi-classifier C: each testing image goes through the tree B: intermediate level N_{train} = 10; 5x speed up with 10% performance drop

20. Results Cascade performance / speed trade off as a function of # training example/class 20x speed up with 10% performance drop for N_{train}=50