Mean-Field Theory and Its Applications In Computer Vision1 1

Introduction 2 Problem formulation Mean-field based inference method Strategy for incorporating different costs

Labelling problem 3 Stereo Object detection Assign a label to each image pixel Object segmentation

Problem Formulation Find a Labelling that maximize the conditional probability 4

Inference 5 T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999 Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-1999 J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001 Message Passing Besag. On the Statistical Analysis of Dirty Pictures, JRSS, 1986 Boykov et al. Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001 Komodakis et al. Fast Approximate Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007 Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010 Move-Making Chekuri et al. Approximation Algorithms for Metric Labelling, SODA, 2001 M. Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995 M. Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003 RaviKumar et al. QP Relaxation for Metric Labelling, ICML 2006 Convex Relaxations K. Alahari et.al. Dynamic Hybrid Algorithms for MAP Inference, PAMI 2010 P. Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008 C. Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007 Other Algorithms

Inference 6 T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999 Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-99 J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001 Message Passing Variational message passing algorithm We focus on mean-field based inference We focus on mean-field based inference

Mean-field methods 7 Intractable inference with distribution Approximate distribution from tractable family Mean-fields methods (Jordan et.al., 1999)

Variational Inference 8 Minimize the KL-divergence between Q and P

Variational Inference 9 Minimize the KL-divergence between Q and P

Variational Inference 10 Minimize the KL-divergence between Q and P

Variational Inference 11 Minimize the KL-divergence between Q and P

Markov Random Field (MRF) 12 Graph: A simple MRF Product of potentials defined over cliques

Markov Random Field (MRF) 13 Graph: In general Un-normalized part

Energy minimization 14 Potential and energy

Variational Inference 15 Entropy of Q Expectation of cost under Q distribution

Naïve Mean Field 16 Family : assume all variables are independent

Variational Inference 17 Shannons entropy decomposes

Variational Inference 18 Stationary point solution Marginal update in mean-field Normalizing constant:

Variational Inference 19 Marginal for variable i taking label l

Variational Inference 20 Marginal for variable i taking label l An assignment of all variables in clique c

Variational Inference 21 Marginal for variable i taking label l An assignment of all variables in clique c An assignment of all variables apart from x_i

Variational Inference 22 Marginal for variable i taking label l An assignment of all variables in clique c An assignment of all variables apart from x_i Marginal distribution of all variables in c apart from x_i

Variational Inference 23 Marginal for variable i taking label l An assignment of all variables in clique c An assignment of all variables apart from x_i Marginal distribution of all variables in c apart from x_i Summation evaluates the expected value of cost over distribution Q given that x_i takes label l

Simple Illustration 24 Naïve mean-field approximation

Mean-field algorithm 25 Iterative algorithm Iterate till convergence Update marginals of each variable in each iteration

Q distribution 26

Max posterior marginal (MPM) 27 MPM with approximate distribution: Empirically achieves very high accuracy: MAP solution / most likely solution

Structured Mean Field 28 Naïve mean field can lead to poor solution Structured (higher order) mean-field

How to make a mean-field algorithm 29 Pick a model Unary, pairwise, higher order cliques Define a cost Potts, linear truncated, robust PN Calculate the marginal Calculate the expectation of cost defined

How to make a mean-field algorithm 30 Use this plug-in strategy in many different models Grid pairwise CRF Dense pairwise CRF Higher order model Co-occurrence model Latent variable model Product label space