IEEE 2015 Conference on Computer Vision and Pattern Recognition Active Learning for Structured Probabilistic Models with Histogram Approximation Qing SunAnkit LaddhaDhruv Batra

2 Semantic segmentation Pose estimation Image classification No. Training images Accuracy

PASCAL VOC (~10,000 training images) 3

Leeds Sports Pose Dataset ( ~ 2000 annotated images) 4

5  ImageNet: Hired more than 25,000 AMT workers  Human Hours ≈19 years  Cost: $$$

Active Learning CRF Train Unlabeled Data Query Labeled Data Pick top n images to annotate 6 Peaky  Low EntropyUniform  High Entropy Compute “Informative”–ness Intractable!

Conditional Random Field 7 Node potential / Local Rewards Edge potential / Distributed Prior y1y1 y2y2 … ynyn yiyi kx kxk Gibbs distribution:

8 Goal Active Learning in Structured Output Models Contribution Novel Variational Approach for Entropy Estimation Challenge: Entropy Computation Intractable

Approximate Entropy via Sampling Gibbs sampling 9 Other modes not explored

Approximate Entropy via Variational Estimation 10 Simple distribution Family Q q*q* KL Divergence p All distributions

Approximate Entropy via Variational Estimation 11 Simple distribution Family Q q*q* Good Approximation p Inefficient Entropy Computation Inefficient Entropy Computation KL Divergence

Approximate Entropy via Variational Estimation 12 Simple distribution Family Q q*q* p Poor Approximation Efficient Entropy Computation Efficient Entropy Computation KL Divergence

Idea 2: Histogram Approximation 13 Idea 2: Histogram Approximation Approximation Histogram Idea 1: Delta Approximation Approximation Delta

14 Histogram Approximation Approximation Histogram

What is the optimal histogram? 15 Theorem [Sun, Laddha and Batra,CVPR 2015] Optimal Histogram  Mass of Gibbs in Bins!

16 What is the optimal histogram?

Mass of Gibbs in a Bin P P P #P-complete

Maximum in Bins = Diverse Solutions 18 [3] Meier et al. The More the Merrier: Parameter Learning for Graphical Models with Multiple MAPs, ICML workshop, 2013 [2] Dhruv Batra et al. Diverse M-Best Solutions in Markov Random Fields, ECCV Top viewFront view

PDivMAP 19  Hamming distance terms are absorbed into the node terms  Solve by reusing MAP solver! (graph cut, TRW-S, etc.)  Lagrangian Relaxation:  Primal: Hamming Distance CRF Score

Synthetic Experiment 20 Tree: Binary, 100 nodes, Potentials

21 True: Predicted:

Segmentation Experiments CRF 22

Baselines 23 SamplingVariational methodsLocal EntropyMargin-based Gibbs Sampling Perturb-and-MAP [Papandreou et al. ICCV 2011] Mean FieldMarginals [Luo et al. NIPS 2013] Min-Marginals [Kohli et al. CVIU 2008] [Batra et al. CVPR 2010] Margin-based [Roth et al. ECML 2006]

iCoseg Dataset [5] [4] Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, Tsuhan Chen, Interactively Co-segmentation Topically Related Images with Intelligent Scribble Guidance,. International Journal of Computer Vision (IJCV)

25 iCoseg Dataset Same performance with only 11% annotations +1%

CMU Geometric Context Dataset Support Vertical Sky Class Left Center Right Porous Solid 26 [6] Derek Hoiem,Alexei A. Efros, Martial Hebert, Recovering Surface Layout from an Image. IJCV, 75(1): (2007)

CMU Geometric Context Dataset 27 Same performance with only 14% annotations +3%

Summary  Variational histogram approximiation for Active learning in structure models.  Theoretically well motivated, easy to implement and outperforms baselines 28 Approximation Histogram 28

Thanks! 29 Qing SunAnkit LaddhaDhruv Batra machinelearning.ece.vt.edu Code is coming soon!