1. Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85

2. Multi-Class Classification System 2 W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013

3. Out in the Real-World Detect New Category 3 Pool Table Bowling Pin Boxing glove Calculator Chess board ? ? ? ?

4. Open World Recognition 4 World with Knowns (K) & Unknowns Unknowns (UU) Detect as Unknown NU: Novel Unknowns Label Data LU: Labeled Unknowns Incremental Class Learning K: Known Scale

5. Open Set Learning Incremental Learning Scalable Learning Ristin+ (CVPR’14) Yeh+ (CVPR’08) Li+ CVPR’07) Mensink+ (PAMI’13) Related Work Jain+ (ECCV’14) Scheirer+ PAMI’13) Scheirer+(PAMI’1 4) Deng+ (NIPS’11) Marszalek+ (ECCV’08) Liu+ (CVPR’13) 5

6. Open Space in Classification ? ? ? ? ? ? ? ?? ?? ?? Open Set Recognition Empirical risk function Regularization constant Open space risk W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013 6 Closed Space Open Space

7. NCM – Metric Learning 7 NCM Classifier with Metric Learning T Mensink, J Verbeek, F Perronin, G Csurka “Distance based Image Classification: Generalizing to New Classes at Near Zero Cost” IEEE TPAMI 2013 M Ristin, M Guillaumin, J Gall, L Van Gool “Incremental Learning of NCM Forests for Large-Scale Image Classification” CVPR 2014

8. Compact Abating Probability (CAP) Models W Scheirer, L Jain, T Boult “Probability Models for Open Set Recognition” IEEE TPAMI 2014 8 Class Mean

9. Theorem 1: Open Space Risk for Model Combination 9

10. Theorem 2: Open Space Risk for Transformed Spaces 10

11. Opening an Existing Algorithm: Nearest Non-Outlier (NNO) Algorithm 11 W = Linear Transformation (weight matrix from metric learning) Standard gamma function In volume of m-D ball Class mean for class i τ is threshold for open world Probability

12. Open World Evaluation Training phase Testing phase Parameter Learning Phase Incremental Learning Phase Closed Set Testing Unknown Categories Known Categories 12 Open Set Testing ? ?

13. Training for Open World 13 NCM - ML NNO Parameter Learning with initial set of categories Estimation of τ for open set learning to balance open space risk Optimize for Known vs Unknown Errors Incrementally add new categories

14. Experiments 14 Datasets ILSVRC’10: 1.2M training images, 1000 classes ILSVRC’12: 1.2M training images, 1000 classes Datasets ILSVRC’10: 1.2M training images, 1000 classes ILSVRC’12: 1.2M training images, 1000 classes Features Dense SIFT features, Quantized into 1000 Bag of Visual Words Publically available features LBP, HOG, Dense SIFT (for ILSVRC’12) Features Dense SIFT features, Quantized into 1000 Bag of Visual Words Publically available features LBP, HOG, Dense SIFT (for ILSVRC’12) Algorithms Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013] Nearest Non-Outlier Algorithm (NNO) [This Paper] 1vSet [Scheirer etal PAMI 2013] Linear SVM [Liblinear, Fan etal JMLR 2008] Algorithms Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013] Nearest Non-Outlier Algorithm (NNO) [This Paper] 1vSet [Scheirer etal PAMI 2013] Linear SVM [Liblinear, Fan etal JMLR 2008]

15. 50 Initial Categories Increasing # of unknown categories during testing i.e. increasing openness of problem Incrementally adding categories during training Closed Set testing

16. 16 200 Initial Categories Increasing # of unknown categories during testing i.e. increasing openness of problem Incrementally adding categories during training Closed Set testing 500 known + 500 unknown categories

17. Formalized Open World Recognition and showed how to “Open” an existing algorithm. NNO allows construction of scalable systems that can be updated incrementally Conclusion & Future Work 17 See us at Poster no 85 …!!!! Exploring sophisticated novelty detectors, open world detection, “opening” other baseline algorithms etc. Open World Deep Learning methods Happy to Collaborate…!!!