Mean-Field Theory and Its Applications In Computer Vision3 1

Gaussian Pairwise Potential 2 Spatial Expensive message passing can be performed by cross-bilateral filtering Range

Cross bilateral filter 3 outputinput reproduced from [Durand 02] outputinput

Efficient Cross-Bilateral Filtering Based on permutohedral lattice (PLBF) 2 Embed the points on the permutohedral lattice Apply Gaussian Blurring 4

Efficient Cross-Bilateral Filtering Based on permutohedral lattice (PLBF) 2 Embed the points on the permutohedral lattice Apply Gaussian Blurring 5 Based on the domain-transform (DTBF) 3 Project the point to lower dimension Perform filtering in the transformed domain

Efficient Cross-Bilateral Filtering Based on permutohedral lattice (PLBF) 2 Embed the points on the permutohedral lattice Apply Gaussian Blurring 6 Based on the domain-transform (DTBF) 3 Project the point to lower dimension Perform filtering in the transformed domain Filtering in frequency domain Apply fast fourier transform convolution in (s) domain=multiplication in (f) domain

Barycentric Interpolation 7

Efficient Cross-Bilateral Filtering 8

Permutohedral Lattice based filtering For each pixel (x, y) 9 Downsample all the points (dependent on standard deviations)

Embed to the permutohedral lattice Embed each downsampled points to the lattice 10

Embed to the permutohedral lattice Embed each downsampled points to the lattice 11

Embed to the permutohedral lattice Embed each downsampled points to the lattice 12

Embed to the permutohedral lattice Embed each downsampled points to the lattice 13

Gaussian blurring Apply Gaussian blurring along axes 14

Gaussian blurring Apply Gaussian blurring along axes 15

Gaussian blurring Apply Gaussian blurring along axes 16

Splatting Upsample the points 17

Splatting Upsample the points 18

PLBF Final upsampled points 19

Domain Transform Filtering 20 Project points in low-dimension preserving the distance in the high dimension Projecting to the original space Filtering performed in low-dimension space

Distance in high-dimension space 21

Filtering in high-dimension space 22 Spatial Range Inefficient

Projection in low-dimension space 23 Project to low-dimension Maintain geodesic distance high-dimension space

Projection in low-dimension space 24 Project to low-dimension Maintain geodesic distance high-dimension space

Projection in low-dimension space 25 Project to low-dimension Maintain geodesic distance high-dimension space

Gaussian blurring in low-dimension 26 Apply Gaussian blurring in low-dimension space

Project 27 Project the blurred values in the original space

Project 28 Project the blurred values in the original space

PLBF Vs DTBF 29 Filter parameter: PLBF runtime is inversely proportional to the kernel size defined over space and range Use PLBF with the relatively large (~10) range Use DTBF with relatively smaller (~1-2) range Processing Time: Both linear in the number of pixels

Filtering in frequency domain 30

Convergence 31 Iteration vs. KL-divergence value In theory: (since parallel update) convergence is not guaranteed In practice: converges observe a convergence

MSRC-21 dataset colour images, 320x213 size, 21 object classes

MSRC-21 dataset colour images, 320x213 size, 21 object classes RuntimeStandard ground truthAccurate ground truth GlobalAverageGlobalAverage Unary Classifiers ± ±2.3 Grid CRF1 sec ± ±1.8 Robust Pn30 sec ± ±1.5 Dense CRF0.2 sec ± ±0.7

PascalVOC-10 dataset colour images, 320x213 size, 21 object classes

PascalVOC-10 dataset colour images, 320x213 size, 21 object classes RuntimeOverallAv. RecallAv. I/U Dense CRF0.67 sec

Long-range connections 36 Accuracy o n increasing the spatial and range standard deviations On MSRC-21 spatial – 61 pixels, range – 11

Long-range connections 37 On increasing the spatial and range standard deviations On MSRC-21 spatial – 61 pixels, range – 11

Long-range connections 38 Sometimes propagates misleading information

Mean-field Vs. Graph-cuts 39 Measure I/U score on PascalVOC-10 segmentation Increase standard deviation for mean-field Increase window size for graph-cuts method Both achieve almost similar accuracy

Mean-field Vs. Graph-cuts 40 Measure I/U score on PascalVOC-10 segmentation Increase standard deviation for mean-field Increase window size for graph-cuts method Time complexity very high, making infeasible to work with large neighbourhood system