Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85

Multi-Class Classification System 2 W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013

Out in the Real-World Detect New Category 3 Pool Table Bowling Pin Boxing glove Calculator Chess board ? ? ? ?

Open World Recognition 4 World with Knowns (K) & Unknowns Unknowns (UU) Detect as Unknown NU: Novel Unknowns Label Data LU: Labeled Unknowns Incremental Class Learning K: Known Scale

Open Set Learning Incremental Learning Scalable Learning Ristin+ (CVPR’14) Yeh+ (CVPR’08) Li+ CVPR’07) Mensink+ (PAMI’13) Related Work Jain+ (ECCV’14) Scheirer+ PAMI’13) Scheirer+(PAMI’1 4) Deng+ (NIPS’11) Marszalek+ (ECCV’08) Liu+ (CVPR’13) 5

Open Space in Classification ? ? ? ? ? ? ? ?? ?? ?? Open Set Recognition Empirical risk function Regularization constant Open space risk W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI Closed Space Open Space

NCM – Metric Learning 7 NCM Classifier with Metric Learning T Mensink, J Verbeek, F Perronin, G Csurka “Distance based Image Classification: Generalizing to New Classes at Near Zero Cost” IEEE TPAMI 2013 M Ristin, M Guillaumin, J Gall, L Van Gool “Incremental Learning of NCM Forests for Large-Scale Image Classification” CVPR 2014

Compact Abating Probability (CAP) Models W Scheirer, L Jain, T Boult “Probability Models for Open Set Recognition” IEEE TPAMI Class Mean

Theorem 1: Open Space Risk for Model Combination 9

Theorem 2: Open Space Risk for Transformed Spaces 10

Opening an Existing Algorithm: Nearest Non-Outlier (NNO) Algorithm 11 W = Linear Transformation (weight matrix from metric learning) Standard gamma function In volume of m-D ball Class mean for class i τ is threshold for open world Probability

Open World Evaluation Training phase Testing phase Parameter Learning Phase Incremental Learning Phase Closed Set Testing Unknown Categories Known Categories 12 Open Set Testing ? ?

Training for Open World 13 NCM - ML NNO Parameter Learning with initial set of categories Estimation of τ for open set learning to balance open space risk Optimize for Known vs Unknown Errors Incrementally add new categories

Experiments 14 Datasets ILSVRC’10: 1.2M training images, 1000 classes ILSVRC’12: 1.2M training images, 1000 classes Datasets ILSVRC’10: 1.2M training images, 1000 classes ILSVRC’12: 1.2M training images, 1000 classes Features Dense SIFT features, Quantized into 1000 Bag of Visual Words Publically available features LBP, HOG, Dense SIFT (for ILSVRC’12) Features Dense SIFT features, Quantized into 1000 Bag of Visual Words Publically available features LBP, HOG, Dense SIFT (for ILSVRC’12) Algorithms Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013] Nearest Non-Outlier Algorithm (NNO) [This Paper] 1vSet [Scheirer etal PAMI 2013] Linear SVM [Liblinear, Fan etal JMLR 2008] Algorithms Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013] Nearest Non-Outlier Algorithm (NNO) [This Paper] 1vSet [Scheirer etal PAMI 2013] Linear SVM [Liblinear, Fan etal JMLR 2008]

50 Initial Categories Increasing # of unknown categories during testing i.e. increasing openness of problem Incrementally adding categories during training Closed Set testing

Initial Categories Increasing # of unknown categories during testing i.e. increasing openness of problem Incrementally adding categories during training Closed Set testing 500 known unknown categories

Formalized Open World Recognition and showed how to “Open” an existing algorithm. NNO allows construction of scalable systems that can be updated incrementally Conclusion & Future Work 17 See us at Poster no 85 …!!!! Exploring sophisticated novelty detectors, open world detection, “opening” other baseline algorithms etc. Open World Deep Learning methods Happy to Collaborate…!!!