Minimum Barrier Salient Object Detection at 80 FPS JIANMING ZHANG, STAN SCLAROFF, ZHE LIN, XIAOHUI SHEN, BRIAN PRICE, RADOMIR MECH IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 IEEE International Conference on Computer Vision (ICCV), 2015, Santiago, Chile 1

Salient Object Detection Generate a Saliency Map for segmenting significant objects in an image. InputSaliency Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Bottom-up Saliency Cues Contrast/RarityImage Boundary Connectivity Appearance Space Frequency salient IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Previous Works Rairity/Contrast Boundary Connectivity State-of- the-art SO [Zhu et al. CVPR’14] AMC [Jiang et al. ICCV’13] GS [Wei et al. ECCV’12] RC [Cheng et al. TPAMI’15] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Previous Works Rairity/Contrast Boundary Connectivity Using Super-pixel? State-of- the-art SO [Zhu et al. CVPR’14]Yes AMC [Jiang et al. ICCV’13]Yes GS [Wei et al. ECCV’12]Yes RC [Cheng et al. TPAMI’15]Yes IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Previous Works Rairity/Contrast Boundary Connectivity Using Super-pixel? State-of- the-art SO [Zhu et al. CVPR’14]Yes AMC [Jiang et al. ICCV’13]Yes GS [Wei et al. ECCV’12]Yes RC [Cheng et al. TPAMI’15]Yes Fast FT [Achanta et al CVPR’09]No HC [Cheng et al. CVPR’11]No IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Previous Works Rairity/Contrast Boundary Connectivity Using Super-pixel? State-of- the-art SO [Zhu et al. CVPR’14]Yes AMC [Jiang et al. ICCV’13]Yes GS [Wei et al. ECCV’12]Yes RC [Cheng et al. TPAMI’15]Yes Fast FT [Achanta et al CVPR’09]No HC [Cheng et al. CVPR’11]No ProposedNo IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Contributions FastMBD, a fast approximate Minimum Barrier Distance (MBD) [Strand et al. CVIU’13] transform algorithm, with error bound analysis. A salient object detection method based on FastMBD, which achieves state-of-the-art performance and is one order of magnitude faster! 77 FPS IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

System Overview Input L a b MBD-L MBD-a MBD-b MBD Sal-Map MBD Transform Post-processing Backgroundness IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Measuring Boundary Connectivity by Distance Transform Compute the distance for each pixel w.r.t. the image boundary Seed Set Shortest Path IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

MBD vs Geodesic Distance In what follows, we consider a single-channel image. MBD [Strand et al. CVIU’13]Geodesic + IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), =

MBD vs Geodesic Distance MBD is robust to small pixel value fluctuation = Geodesic + IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), = MBD

FastMBD A raster-scanning iterative algorithm Share the same spirit of the raster-scanning geodesic distance transform [Toivanen, Pattern Recognition Letters, 1996] Highly efficient in practice (2ms/scan for a 320X240 image!) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Algorithm For each visited pixel x : 1.Check each of the 4-connected neighbors IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), x y

Algorithm For each visited pixel x : 1.Check each of the 4-connected neighbors 2.Minimize the path cost 3.Update: ◦ D (x), cost of current assigned path ◦ U (x), highest value on assigned path ◦ L (x), lowest value on assigned path IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), D(y)D(y) U(y)U(y) L(y)L(y) D(x)D(x) U(x)U(x) L(x)L(x)

Some Analysis … FastMBD returns an upper-bound of the MBD for each pixel. FastMBD will eventually converge. Q: Is the converged solution of FastMBD equal to the exact MBD transform? A: No  [Falcao et al. TPAMI’04]. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

An Error Bound Result Definition: Maximum Local Difference is the maximum absolute pixel value difference between a pair of pixels that share an edge or a corner on an image. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), Measures maximal local discontinuity Under a mild condition, the approximation error of the converged solution of FastMBD is bounded by. The more continuous the image, the smaller the error.

Empirical Evaluation of FastMBD IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), PASCAL-S

Minimum Barrier Salient Object Detection 1.For each color channel, compute the MBD map by FastMBD 2.Post-process the averaged MBD map a)Morphological smoothing b)Multiplying a center-distance map c)Contrast enhancement IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), InputAveraged MBD MapOutput

Leveraging Backgroundness 1.For each color channel, compute the MBD map by FastMBD 2.Compute the Image Boudnary Contrast (IBC) Map 3.Post-process the averaged MBD map + IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), Averaged MBD MapIBC MapOutput

Computing the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), Image boundary region is mostly background. Compute mean color and color covariance Mahalanobis Distance to Mean Color

Computing the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Enhancement by the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), W/O IBC Map With IBC Map

Compared Methods MB+ (full system) MB (w/oIBC map) GD (Same as MB but uses Geodesic Distance) FT[Achantaet al. CVPR’09] HC [Chengetal. CVPR’11] ] SO [Zhu etal. CVPR’14 AMC [Jiang et al. ICCV’13] HS [Yan et al. CVPR’13] GS [Wei et al. ECCV’12] RC [Cheng et al. TPAMI’15] SIA [Cheng et al. ICCV’13] Experiments IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), Datasets #Images Difficulty MSRA10K DUTOmron ECSSD SPASCAL

Performance Precision-Recall Curve IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), Comparison with the State-of-the-art Comparison with Other Baselines

Performance Weighted F β [Margolin et al. CVPR’14] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Speed Excluding IO time Using a singe thread IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), FPS Mean Weighted Fb MB MB+

Qualitative Evaluation IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Conclusion Propose to use MBD to measure image boundary connectivity Present FastMBD, a fast approximation MBD transform algorithm Achieve state-of-the-art performance at a substantially reduced computational cost (~80 FPS) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), An executable program is available on our website

Performance Area Under the Curve (AUC) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Performance mean Average Error (mAE) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Performance Controlling Post-processing IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV),

Algorithm For each visited pixel x : 1.Check each of the 4-connected neighbors 2.Minimize the path cost 3.Update D (x), L (x), U (x) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), D(y)D(y) U(y)U(y) L(y)L(y) D(x)D(x) U(x)U(x) L(x)L(x)