1. IEEE 2015 Conference on Computer Vision and Pattern Recognition Active Learning for Structured Probabilistic Models with Histogram Approximation Qing SunAnkit LaddhaDhruv Batra

2. 2 Semantic segmentation Pose estimation Image classification No. Training images Accuracy

3. PASCAL VOC (~10,000 training images) 3

4. Leeds Sports Pose Dataset ( ~ 2000 annotated images) 4

5. 5  ImageNet: Hired more than 25,000 AMT workers  Human Hours ≈19 years  Cost: $$$

6. Active Learning CRF Train Unlabeled Data Query Labeled Data Pick top n images to annotate 6 Peaky  Low EntropyUniform  High Entropy Compute “Informative”–ness Intractable!

7. Conditional Random Field 7 Node potential / Local Rewards Edge potential / Distributed Prior y1y1 y2y2 … ynyn yiyi kx1 1 1 10 0 kxk 10 0 0 Gibbs distribution:

8. 8 Goal Active Learning in Structured Output Models Contribution Novel Variational Approach for Entropy Estimation Challenge: Entropy Computation Intractable

9. Approximate Entropy via Sampling Gibbs sampling 9 Other modes not explored

10. Approximate Entropy via Variational Estimation 10 Simple distribution Family Q q*q* KL Divergence p All distributions

11. Approximate Entropy via Variational Estimation 11 Simple distribution Family Q q*q* Good Approximation p Inefficient Entropy Computation Inefficient Entropy Computation KL Divergence

12. Approximate Entropy via Variational Estimation 12 Simple distribution Family Q q*q* p Poor Approximation Efficient Entropy Computation Efficient Entropy Computation KL Divergence

13. Idea 2: Histogram Approximation 13 Idea 2: Histogram Approximation Approximation Histogram Idea 1: Delta Approximation Approximation Delta

14. 14 Histogram Approximation 00.10.20.30.40.50.6 00.10.20.30.40.50.6 Approximation Histogram

15. What is the optimal histogram? 15 Theorem [Sun, Laddha and Batra,CVPR 2015] Optimal Histogram  Mass of Gibbs in Bins!

16. 16 What is the optimal histogram? 00.10.20.30.40.50.6 00.10.20.30.40.50.6

17. 110000101110000101 110000101110000101 110000101110000101 17 111001001111001001 010010100010010100 Mass of Gibbs in a Bin P P P #P-complete

18. Maximum in Bins = Diverse Solutions 18 [3] Meier et al. The More the Merrier: Parameter Learning for Graphical Models with Multiple MAPs, ICML workshop, 2013 [2] Dhruv Batra et al. Diverse M-Best Solutions in Markov Random Fields, ECCV 2012. Top viewFront view

19. PDivMAP 19  Hamming distance terms are absorbed into the node terms  Solve by reusing MAP solver! (graph cut, TRW-S, etc.)  Lagrangian Relaxation:  Primal: Hamming Distance CRF Score

20. Synthetic Experiment 20 Tree: Binary, 100 nodes, Potentials

21. 21 True: Predicted:

22. Segmentation Experiments CRF 22

23. Baselines 23 SamplingVariational methodsLocal EntropyMargin-based Gibbs Sampling Perturb-and-MAP [Papandreou et al. ICCV 2011] Mean FieldMarginals [Luo et al. NIPS 2013] Min-Marginals [Kohli et al. CVIU 2008] [Batra et al. CVPR 2010] Margin-based [Roth et al. ECML 2006]

24. iCoseg Dataset [5] http://chenlab.ece.cornell.edu/projects/touch-coseg/ [4] Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, Tsuhan Chen, Interactively Co-segmentation Topically Related Images with Intelligent Scribble Guidance,. International Journal of Computer Vision (IJCV) 2011 24

25. 25 iCoseg Dataset Same performance with only 11% annotations +1%

26. CMU Geometric Context Dataset Support Vertical Sky Class Left Center Right Porous Solid 26 [6] Derek Hoiem,Alexei A. Efros, Martial Hebert, Recovering Surface Layout from an Image. IJCV, 75(1): 151-172 (2007)

27. CMU Geometric Context Dataset 27 Same performance with only 14% annotations +3%

28. Summary  Variational histogram approximiation for Active learning in structure models.  Theoretically well motivated, easy to implement and outperforms baselines 28 Approximation Histogram 28

29. Thanks! 29 Qing SunAnkit LaddhaDhruv Batra machinelearning.ece.vt.edu Code is coming soon!