Mean-Field Theory and Its Applications In Computer Vision2

Dense CRF construction Problem Formulation Grid CRF leads to over smoothing around boundaries Dense CRF is able to recover fine boundaries Grid CRF construction Dense CRF construction

Long Range Interaction Able to recover proper flow for objects Teddy arms recovered using Global interaction Optical flow Optical flow and stereo reconstruction image Local interaction Global interaction Ground truth

Very Expensive Step (O(n2)) Marginal Update Marginal Update for large neighbourhood: Very Expensive Step (O(n2))

Inference in Dense CRF Time complexity increases Neighbourhood size MCMC takes 36 hours on 50K variables Graph-cuts based algorithm takes hours

Inference in Dense CRF Time complexity increases Neighbourhood size MCMC takes 36 hours on 50K variables Graph-cuts based algorithm takes hours Not practical for vision applications

Inference in Dense CRF Time complexity increases Neighbourhood size MCMC takes 36 hours on 50K variables Graph-cuts based algorithm takes hours Filter-based Mean-field Inference takes 0.2 secs Possibility of development of many exciting vision applications

Efficient inference Assume Gaussian pairwise weight Label compatibility function

Efficient inference Assume Gaussian pairwise weight Mixture of Gaussians Spatial Bilateral

Bilateral filter output input output input reproduced from [Durand 02]

Marginal update Assume Gaussian pairwise weight

Very Expensive Step (O(n2)) How does it work Very Expensive Step (O(n2))

Message passing from all Xj to all Xi Accumulates weights from all other pixels except itself

Message passing from all Xj to all Xi Convert as Gaussian filtering step: Accumulate weights from all other pixels except itself

Message passing from all Xj to all Xi Convert as Gaussian filtering step: Accumulate weights from all other pixels except itself

Efficient filtering steps Now discuss how to do efficient filtering step