Mean-Field Theory and Its Applications In Computer Vision6 1

Inference In Product Label Space Many problem requires jointly estimating labels in product label space 2 Black Box Solver Left Camera Image Right Camera Image Object Class Segmentation Dense Stereo Reconstruction

Joint Object-Stereo Labelling Computation complexity very high Graph-cuts based method takes almost 50 secs for 320x200 image size We propose mean-field based inference method Our method takes 2 secs for the same task 3

Joint stereo-object inference Introduce two different set of variables 4 disparity variable object variable Messages exchanged between the variables

Joint stereo-object formulation 5 Unary Potential Weighted sum of object class, depth and joint potential Joint unary potential based on histograms of height

Joint stereo-object formulation 6 Pairwise Potential Object class and depth edges correlated We disregard the joint pairwise terms though Dense pairwise connection at both disparity variable and object variables

Joint stereo-object formulation 7 Higher Order Potential Use higher order terms only for object variables

Joint stereo-object updation 8 For object variables Message from disparity variables to object variables

Joint stereo-object updation 9 For object variables Filtering is done using permutohedral lattice based filtering strategy

Joint stereo-object updation 10 For disparity variables Message from object variables to disparity variables

Joint stereo-object updation 11 For disparity variables Filtering is done using domain transform based filtering strategy

Leuven dataset 12 Some of qualitative results

Leuven dataset 13 Some of qualitative results AlgorithmTime (s)Object (% correct)Stereo (% correct) GC + Range (1) GC + Range (2) GC + Range (3) Extended CostVol Dense + HO (PLBF) Dense + HO (DTBF) Dense + HO + CostVol + DTBF