CVPR 2013 Diversity Tutorial Beyond MAP: Making Multiple Predictions: Diversity, DPPs and more. Dhruv Batra Virginia Tech Alex Kulesza Univ. of Michigan.

Slides:

Advertisements

Similar presentations

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.

Advertisements

Mean-Field Theory and Its Applications In Computer Vision1 1.

Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London Tutorial at GDR (Optimisation Discrète, Graph Cuts.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

Diverse M-Best Solutions in Markov Random Fields

The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.

Focused Inference with Local Primal-Dual Gaps

Introduction to Markov Random Fields and Graph Cuts Simon Prince

Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction Ľubor Ladický, Paul Sturgess, Christopher Russell, Sunando Sengupta, Yalin.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Learning with Inference for Discrete Graphical Models Nikos Komodakis Pawan Kumar Nikos Paragios Ramin Zabih (presenter)

ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

Learning with Inference for Discrete Graphical Models Nikos Komodakis Pawan Kumar Nikos Paragios Ramin Zabih (presenter)

1 Fast Primal-Dual Strategies for MRF Optimization (Fast PD) Robot Perception Lab Taha Hamedani Aug 2014.

Markov Nets Dhruv Batra, Recitation 10/30/2008.

Graphical models, belief propagation, and Markov random fields 1.

Discriminative and generative methods for bags of features

1 Graphical Models in Data Assimilation Problems Alexander Ihler UC Irvine Collaborators: Sergey Kirshner Andrew Robertson Padhraic Smyth.

Robust Higher Order Potentials For Enforcing Label Consistency

Statistical Recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and Kristen Grauman.

LARGE-SCALE NONPARAMETRIC IMAGE PARSING Joseph Tighe and Svetlana Lazebnik University of North Carolina at Chapel Hill CVPR 2011Workshop on Large-Scale.

Improved Moves for Truncated Convex Models M. Pawan Kumar Philip Torr.

2010/5/171 Overview of graph cuts. 2010/5/172 Outline Introduction S-t Graph cuts Extension to multi-label problems Compare simulated annealing and alpha-

Efficiently Solving Convex Relaxations M. Pawan Kumar University of Oxford for MAP Estimation Philip Torr Oxford Brookes University.

Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.

Generative learning methods for bags of features

The Layout Consistent Random Field for Recognizing and Segmenting Partially Occluded Objects By John Winn & Jamie Shotton CVPR 2006 presented by Tomasz.

Graph Cut based Inference with Co-occurrence Statistics Ľubor Ladický, Chris Russell, Pushmeet Kohli, Philip Torr.

Understanding Belief Propagation and its Applications Dan Yuan June 2004.

Linear Solution to Scale and Rotation Invariant Object Matching Professor: 王聖智教授 Student : 周節.

What Energy Functions Can be Minimized Using Graph Cuts? Shai Bagon Advanced Topics in Computer Vision June 2010.

Relaxations and Moves for MAP Estimation in MRFs M. Pawan Kumar STANFORDSTANFORD Vladimir KolmogorovPhilip TorrDaphne Koller.

Extensions of submodularity and their application in computer vision

What, Where & How Many? Combining Object Detectors and CRFs

Multiplicative Bounds for Metric Labeling M. Pawan Kumar École Centrale Paris École des Ponts ParisTech INRIA Saclay, Île-de-France Joint work with Phil.

Hamed Pirsiavash, Deva Ramanan, Charless Fowlkes

Probabilistic Inference Lecture 4 – Part 2 M. Pawan Kumar Slides available online

1 Efficient Stochastic Local Search for MPE Solving Frank Hutter The University of British Columbia (UBC), Vancouver, Canada Joint work with Holger Hoos.

Reconstructing Relief Surfaces George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla BMVC 2004.

Surface Stereo with Soft Segmentation Michael Bleyer 1, Carsten Rother 2, Pushmeet Kohli 2 1 Vienna University of Technology, Austria 2 Microsoft Research.

MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/24/10.

CVPR 2013 Diversity Tutorial Closing Remarks: What can we do with multiple diverse solutions? Dhruv Batra Virginia Tech.

Multiplicative Bounds for Metric Labeling M. Pawan Kumar École Centrale Paris Joint work with Phil Torr, Daphne Koller.

Fast Parallel and Adaptive Updates for Dual-Decomposition Solvers Ozgur Sumer, U. Chicago Umut Acar, MPI-SWS Alexander Ihler, UC Irvine Ramgopal Mettu,

An Efficient Message-Passing Algorithm for the M-Best MAP Problem Dhruv Batra (Currently) Research Assistant Professor TTI-Chicago (Spring 2013) Assistant.

Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London.

Associative Hierarchical CRFs for Object Class Image Segmentation Ľubor Ladický 1 1 Oxford Brookes University 2 Microsoft Research Cambridge Based on the.

Discrete Optimization Lecture 3 – Part 1 M. Pawan Kumar Slides available online

Dynamic Tree Block Coordinate Ascent Daniel Tarlow 1, Dhruv Batra 2 Pushmeet Kohli 3, Vladimir Kolmogorov 4 1: University of Toronto3: Microsoft Research.

Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.

Associative Hierarchical CRFs for Object Class Image Segmentation

Tractable Higher Order Models in Computer Vision (Part II) Slides from Carsten Rother, Sebastian Nowozin, Pusohmeet Khli Microsoft Research Cambridge Presented.

Dependency networks Sushmita Roy BMI/CS 576 Nov 25 th, 2014.

Probabilistic Graphical Models seminar 15/16 ( ) Haim Kaplan Tel Aviv University.

Pictorial Structures and Distance Transforms Computer Vision CS 543 / ECE 549 University of Illinois Ian Endres 03/31/11.

Discrete Optimization Lecture 2 – Part 2 M. Pawan Kumar Slides available online

Beyond Trees: MRF Inference via Outer-Planar Decomposition [(To appear) CVPR ‘10] Dhruv Batra Carnegie Mellon University Joint work: Andrew Gallagher (Eastman.

Towards Total Scene Understanding: Classiﬁcation, Annotation and Segmentation in an Automatic Framework N 工科所錢雅馨 2011/01/16 Li-Jia Li, Richard.

Markov Random Fields & Conditional Random Fields

Tightening LP Relaxations for MAP using Message-Passing David Sontag Joint work with Talya Meltzer, Amir Globerson, Tommi Jaakkola, and Yair Weiss.

Markov Networks: Theory and Applications Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208

IEEE 2015 Conference on Computer Vision and Pattern Recognition Active Learning for Structured Probabilistic Models with Histogram Approximation Qing SunAnkit.

Rounding-based Moves for Metric Labeling M. Pawan Kumar École Centrale Paris INRIA Saclay, Île-de-France.

Energy minimization Another global approach to improve quality of correspondences Assumption: disparities vary (mostly) smoothly Minimize energy function:

The M-Best Mode Problem

Alexander Shekhovtsov and Václav Hlaváč

Learning to Combine Bottom-Up and Top-Down Segmentation

Integration and Graphical Models

Diversity meets Deep Networks: Inference, Ensembles, and Applications

Presentation transcript:

CVPR 2013 Diversity Tutorial Beyond MAP: Making Multiple Predictions: Diversity, DPPs and more. Dhruv Batra Virginia Tech Alex Kulesza Univ. of Michigan Dennis Park UC Irvine Deva Ramanan UC Irvine

CVPR 2013 Diversity Tutorial Schedule TimeTopicPresenter 9:00 – 9:30Opening Remarks + Need for Multiple Diverse Solutions Dhruv 9:30 – 10:15Multiple Solutions via M-Best MAPDennis 10:15 – 10:45Coffee Break 10:45 – 11:30Multiple Solutions via Diverse M-BestDhruv 12:00 – 1:30Lunch (C) Dhruv Batra2

CVPR 2013 Diversity Tutorial TimeTopicPresenter 9:00 – 9:30Opening Remarks + Need for Multiple Diverse Solutions Dhruv 9:30 – 10:15Multiple Solutions via M-Best MAPDennis 10:15 – 10:45Coffee Break 10:45 – 11:30Multiple Solutions via Diverse M-BestDhruv 12:00 – 1:30Lunch 1:30 – 2:00Multiple Solutions via SamplingDeva 2:00 – 3:15Multiple Solutions via DPPsAlex 3:15 – 3:45Coffee Break 3:45 – 4:30DPPs (Continued)Alex 4:30 – 5:00Closing Remarks + What can we do with diverse solutions? Dhruv Schedule (C) Dhruv Batra3 1. Please interrupt & ask questions! 2. All slides available online.

CVPR 2013 Diversity Tutorial Local Ambiguity (C) Dhruv Batra4 slide credit: Fei-Fei Li, Rob Fergus & Antonio Torralba

CVPR 2013 Diversity Tutorial Graphical Models to the Rescue! (C) Dhruv Batra5 MAP Inference Most Likely Assignment y1y1 y2y2 … xnxn Person Table Plate

CVPR 2013 Diversity Tutorial Vision in 2000s (C) Dhruv Batra6

CVPR 2013 Diversity Tutorial Graphical Models in Vision (C) Dhruv Batra7 Segmentation Left imageRight imageDisparity map Stereo Geometric Labelling Denoising Motion Flow Object Recognition / Pose Estimation

CVPR 2013 Diversity Tutorial (C) Dhruv Batra8 Alpha-Expansion Simulated Annealing

CVPR 2013 Diversity Tutorial Dollar et al., BMVC

CVPR 2013 Diversity Tutorial (C) Dhruv Batra10

CVPR 2013 Diversity Tutorial Problems (C) Dhruv Batra11 Model-Class is Wrong! -- Approximation Error Human Body ≠ Tree Figure Courtesy: [Yang & Ramanan ICCV ‘11]

CVPR 2013 Diversity Tutorial Problems (C) Dhruv Batra12 Model-Class is Wrong! -- Approximation Error Not Enough Training Data! -- Estimation Error

CVPR 2013 Diversity Tutorial Problems (C) Dhruv Batra13 Model-Class is Wrong! -- Approximation Error Not Enough Training Data! -- Estimation Error MAP is NP-Hard -- Optimization Error

CVPR 2013 Diversity Tutorial Biggest Problem (C) Dhruv Batra14 Model-Class is Wrong! -- Approximation Error Not Enough Training Data! -- Estimation Error MAP is NP-Hard -- Optimization Error Inherent Ambiguity -- Bayes Error ? ? Old Lady looking left / Young woman looking away? Rotating clockwise / anti-clockwise? One instance / Two instances?

CVPR 2013 Diversity Tutorial Problems (C) Dhruv Batra15 Model-Class is Wrong! -- Approximation Error Not Enough Training Data! -- Estimation Error MAP is NP-Hard -- Optimization Error Inherent Ambiguity -- Bayes Error Make Multiple Predictions! Single Prediction = Uncertainty Mismanagement

CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra16 Flerova et al., 2011 Fromer et al., 2009 Yanover et al., 2003 (Diverse) M-Best MAP Dhruv 10:45 – 11:30 Dennis 9:30 – 10:15

CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra17 Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Sampling xxxxxxxxxxxxx Deva 1:30 – 2:00-pm

CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra18 Build a new model over sets that prefers diverse set Determinental Point Process Alex 2:00 – 3:15 3:45 – 4:30

CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra19 Build a new model over sets that prefers diverse set Determinental Point Process Alex 2:00 – 3:15 3:45 – 4:30 Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Sampling Flerova et al., 2011 Fromer et al., 2009 Yanover et al., 2003 (Diverse) M-Best MAP

CVPR 2013 Diversity Tutorial Multiple Predictions (C) Dhruv Batra20 Build a new model over sets that prefers diverse set Determinental Point Process Porway & Zhu, 2011 TU & Zhu, 2002 Rich History Sampling Flerova et al., 2011 Fromer et al., 2009 Yanover et al., 2003 (Diverse) M-Best MAP

CVPR 2013 Diversity Tutorial Schedule TimeTopicPresenter 9:00 – 9:30Opening Remarks + Need for Multiple Diverse Solutions Dhruv 9:30 – 10:15Multiple Solutions via M-Best MAPDennis 10:15 – 10:45Coffee Break 10:45 – 11:30Multiple Solutions via Diverse M-BestDhruv 12:00 – 1:30Lunch 1:30 – 2:00Multiple Solutions via SamplingDeva 2:00 – 3:15Multiple Solutions via DPPsAlex 3:15 – 3:45Coffee Break 3:45 – 4:30DPPs (Continued)Alex 4:30 – 5:00Closing Remarks + What can we do with diverse solutions? Dhruv (C) Dhruv Batra21 All slides available online.

CVPR 2013 Diversity Tutorial Notation and Review of CRFs (C) Dhruv Batra22

CVPR 2013 Diversity Tutorial Conditional Random Fields Discrete random variables Factorized Model 23(C) Dhruv Batra Node Energies / Local Costs Edge Energies / Distributed Prior X1X1 X2X2 … XnXn XiXi kx kxk

CVPR 2013 Diversity Tutorial MAP Inference In general NP-hard [Shimony ‘94] (C) Dhruv Batra24 Approximate Inference Heuristics: Loopy BP[Pearl, ‘88] Greedy: α-Expansion [Boykov ’01, Komodakis ‘05] LP Relaxations:[Schlesinger ‘76, Wainwright ’05, Sontag ’08, Batra ‘10] QP/SDP Relaxations: [Ravikumar ’06, Kumar ‘09]

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra25 kx1

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra26 kx

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra27 kx

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra28 kx

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra29 kx

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra30 kx k 2 x1

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra31 kx k 2 x1

CVPR 2013 Diversity Tutorial MAP Integer Program (C) Dhruv Batra32 Graphcuts, BP, Expansion, etc

CVPR 2013 Diversity Tutorial MAP Integer Program LP view (C) Dhruv Batra33 MAP Graphcuts, BP, Expansion, etc

CVPR 2013 Diversity Tutorial Schedule TimeTopicPresenter 9:00 – 9:30Opening Remarks + Need for Multiple Diverse Solutions Dhruv 9:30 – 10:15Multiple Solutions via M-Best MAPDennis 10:15 – 10:45Coffee Break 10:45 – 11:30Multiple Solutions via Diverse M-BestDhruv 12:00 – 1:30Lunch 1:30 – 2:00Multiple Solutions via SamplingDeva 2:00 – 3:15Multiple Solutions via DPPsAlex 3:15 – 3:45Coffee Break 3:45 – 4:30DPPs (Continued)Alex 4:30 – 5:00Closing Remarks + What can we do with diverse solutions? Dhruv (C) Dhruv Batra34 All slides available online.