3 Small Comments Alex Berg Stony Brook University I work on recognition: features – action recognition – alignment – detection – attributes – hierarchical image classification & retrieval + machine learning for large scale recognition Collaborations: words & pictures – ImageNet – human visual search – neural coding of visual memory

0. Good computer vision tells us something about the structure of the visual world or about our descriptions of the visual world. 1.We should know what we are recognizing, and be able to prove it! 2.Large datasets can be gallant but doomed attempts to avoid hard representation problems. (Small datasets are worse.) 3.We should try to understand uncertainty in computer vision. Alex Berg – Stony Brook University 3 Small Comments

The structure of the visual world is complex. So are the ways we describe it. Recognition experiments probe mappings from samples of the visual world to descriptions. Alex Berg – Stony Brook University 0. Good computer vision tells us something about the structure of the visual world.

This is a PR problem! Advertise our successes and abilities, but avoid over-selling. Example Given training and evaluation of an X detector on my own dataset: IdealIf your dataset has Xs, then my algorithm can detect them. GreatIf people can detect Xs in your dataset, then my algorithm will detect them. GoodI can predict performance on your dataset based on a statistical characterization. Okay I can very roughly predict performance on your dataset based on a characterization. Now If I were to run my algorithm on your dataset, we could determine performance. Bad I don’t know how it works on my dataset! Alex Berg – Stony Brook University 1. We should know what we are recognizing, and be able to prove it!

Describable Visual Attributes for Face Verification and Image Search N. Kumar, A.C. Berg, P.N. Belhumeur, S.K. Nayar (T.PAMI 2011) Verification classifier

6 LFW Results In 2009

Describable Visual Attributes for Face Verification and Image Search N. Kumar, A.C. Berg, P.N. Belhumeur, S.K. Nayar Verification classifier

2. Large datasets can be gallant but doomed attempts to avoid hard representation problems. Alex Berg – Stony Brook University w/ Li Fei-Fei & Jia Stanford Efficient Additive Models for Detection & Classification w/ Subhransu UCB −> TTI-C Test Image Large Dataset There comes a point when it is necessary to go into more detail – this is the regime of mid-level vision. More data provides better joint statistics, but is enough only sometimes. There is some boost from looking at large data, but, to do well we still need to address hard (mid-level) representation problems. Might hope that matching/classifying a whole image / pattern / large scale “just works” for recognition. Large Scale Recognition Challenge going on now (part of PASCAL VOC) Image Classification + Object Detection! (Small datasets are worse!)

3. We should try to understand uncertainty in computer vision. Alex Berg – Stony Brook University 1.Need an explicit idea of what is possible/likely given observations. 2.We need this for low, mid, and high level vision. 3.This is a difficult representational challenge – mainly because of complex structure.

What does classifying more than 10,000 image categories tell us? J. Deng, A.C. Berg, K. Li, L. Fei-Fei (ECCV 2010) Correlation between CV classifier confusions and WordNet!

0. Good computer vision tells us something about the structure of the visual world or about our descriptions of the visual world. 1.We should know what we are recognizing, and be able to prove it! 2.Large datasets can be gallant but doomed attempts to avoid hard representation problems. (Small datasets are worse.) 3.We should try to understand uncertainty in computer vision. Alex Berg – Stony Brook University 3 Small Comments