Geodesic Saliency Using Background Priors

Slides:



Advertisements
Similar presentations
Weakly supervised learning of MRF models for image region labeling Jakob Verbeek LEAR team, INRIA Rhône-Alpes.
Advertisements

Ming-Ming Cheng 1 Ziming Zhang 2 Wen-Yan Lin 3 Philip H. S. Torr 1 1 Oxford University, 2 Boston University 3 Brookes Vision Group Training a generic objectness.
Shape Sharing for Object Segmentation
Scene Labeling Using Beam Search Under Mutex Constraints ID: O-2B-6 Anirban Roy and Sinisa Todorovic Oregon State University 1.
Leveraging Stereopsis for Saliency Analysis
Recovering Human Body Configurations: Combining Segmentation and Recognition Greg Mori, Xiaofeng Ren, and Jitentendra Malik (UC Berkeley) Alexei A. Efros.
- Recovering Human Body Configurations: Combining Segmentation and Recognition (CVPR’04) Greg Mori, Xiaofeng Ren, Alexei A. Efros and Jitendra Malik -
Computer Vision for Human-Computer InteractionResearch Group, Universität Karlsruhe (TH) cv:hci Dr. Edgar Seemann 1 Computer Vision: Histograms of Oriented.
Database-Based Hand Pose Estimation CSE 6367 – Computer Vision Vassilis Athitsos University of Texas at Arlington.
Face Alignment by Explicit Shape Regression
Face Alignment at 3000 FPS via Regressing Local Binary Features
Qualifying Exam: Contour Grouping Vida Movahedi Supervisor: James Elder Supervisory Committee: Minas Spetsakis, Jeff Edmonds York University Summer 2009.
Stephen J. Guy 1. Photomontage Photomontage GrabCut – Interactive Foreground Extraction 1.
Global spatial layout: spatial pyramid matching Spatial weighting the features Beyond bags of features: Adding spatial information.
Hierarchical Saliency Detection School of Electronic Information Engineering Tianjin University 1 Wang Bingren.
AAM based Face Tracking with Temporal Matching and Face Segmentation Dalong Du.
Computer Vision Group, University of BonnVision Laboratory, Stanford University Abstract This paper empirically compares nine image dissimilarity measures.
 Introduction  Principles of context – aware saliency  Detection of context – aware saliency  Result  Application  Conclusion.
Stas Goferman Lihi Zelnik-Manor Ayellet Tal. …
Hierarchical Region-Based Segmentation by Ratio-Contour Jun Wang April 28, 2004 Course Project of CSCE 790.
1 Image Recognition - I. Global appearance patterns Slides by K. Grauman, B. Leibe.
CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.
Visual Attention More information in visual field than we can process at a given moment Solutions Shifts of Visual Attention related to eye movements Some.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Christian Siagian Laurent Itti Univ. Southern California, CA, USA
CS292 Computational Vision and Language Visual Features - Colour and Texture.
Spatial Pyramid Pooling in Deep Convolutional
Introduction of Saliency Map
Handwritten Character Recognition using Hidden Markov Models Quantifying the marginal benefit of exploiting correlations between adjacent characters and.
Speaker: Chi-Yu Hsu Advisor: Prof. Jian-Jung Ding Leveraging Stereopsis for Saliency Analysis, CVPR 2012.
Computer Science Department, Duke UniversityPhD Defense TalkMay 4, 2005 Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis Nikos P.
Salient Object Detection by Composition
EADS DS / SDC LTIS Page 1 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen.
Visual Tracking Decomposition Junseok Kwon* and Kyoung Mu lee Computer Vision Lab. Dept. of EECS Seoul National University, Korea Homepage:
Texture. Texture is an innate property of all surfaces (clouds, trees, bricks, hair etc…). It refers to visual patterns of homogeneity and does not result.
Model comparison and challenges II Compositional bias of salient object detection benchmarking Xiaodi Hou K-Lab, Computation and Neural Systems California.
Lecture 29: Face Detection Revisited CS4670 / 5670: Computer Vision Noah Snavely.
Visual Object Recognition
MSRI workshop, January 2005 Object Recognition Collected databases of objects on uniform background (no occlusions, no clutter) Mostly focus on viewpoint.
Scene Completion Using Millions of Photographs James Hays, Alexei A. Efros Carnegie Mellon University ACM SIGGRAPH 2007.
Region-Based Saliency Detection and Its Application in Object Recognition IEEE TRANSACTIONS ON CIRCUITS AND SYSTEM FOR VIDEO TECHNOLOGY, VOL. 24 NO. 5,
Face Alignment at 3000fps via Regressing Local Binary Features CVPR14 Shaoqing Ren, Xudong Cao, Yichen Wei, Jian Sun Presented by Sung Sil Kim.
Category Independent Region Proposals Ian Endres and Derek Hoiem University of Illinois at Urbana-Champaign.
Non-Ideal Iris Segmentation Using Graph Cuts
AAM based Face Tracking with Temporal Matching and Face Segmentation Mingcai Zhou 1 、 Lin Liang 2 、 Jian Sun 2 、 Yangsheng Wang 1 1 Institute of Automation.
Colour and Texture. Extract 3-D information Using Vision Extract 3-D information for performing certain tasks such as manipulation, navigation, and recognition.
Stas Goferman Lihi Zelnik-Manor Ayellet Tal Technion.
Object Recognition by Integrating Multiple Image Segmentations Caroline Pantofaru, Cordelia Schmid, Martial Hebert ECCV 2008 E.
Speaker Min-Koo Kang March 26, 2013 Depth Enhancement Technique by Sensor Fusion: MRF-based approach.
 Mentor : Prof. Amitabha Mukerjee Learning to Detect Salient Objects Team Members - Avinash Koyya Diwakar Chauhan.
ICCV 2007 National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Half Quadratic Analysis for Mean Shift: with Extension.
Minimum Barrier Salient Object Detection at 80 FPS JIANMING ZHANG, STAN SCLAROFF, ZHE LIN, XIAOHUI SHEN, BRIAN PRICE, RADOMIR MECH IEEE INTERNATIONAL CONFERENCE.
Interactive Offline Tracking for Color Objects
Saliency detection with background model
Mean Shift Segmentation
Learning to Detect a Salient Object
Object detection as supervised classification
Enhanced-alignment Measure for Binary Foreground Map Evaluation
Week 8 Nicholas Baker.
Text Detection in Images and Video
Dingding Liu* Yingen Xiong† Linda Shapiro* Kari Pulli†
Saliency detection Donghun Yeo CV Lab..
Brief Review of Recognition + Context
KFC: Keypoints, Features and Correspondences
Grouping/Segmentation
Fourier Transform of Boundaries
Lecture 29: Face Detection Revisited
Saliency Optimization from Robust Background Detection
Learning complex visual concepts
Presentation transcript:

Geodesic Saliency Using Background Priors Yichen Wei, Fang Wen, Wangjiang Zhu, Jian Sun Visual Computing Group Microsoft Research Asia

Saliency detection is useful Find whatever attracts visual interest a built-in ability in human vision system Important computer vision tasks Image summarization, cropping… Object (instance) matching, retrieval… Object (class) detection, recognition… It’s relatively easy to find salient objects than other ones because they are …

What exactly is saliency? Subjective, ambiguous and task dependent traditionally, where a human looks recently, where the salient object is Categorization of methodology top down: integrate domain knowledge bottom up: biological observations / rules / priors

Saliency detection is challenging Subjective and ambiguous Hard evaluation (task-dependent) Few theories and principles Mostly built on image priors √ ? X

Almost all work uses contrast prior “Salient region-background contrast” is high local, global all those in statistics, information theory… contrast context contrast measure feature primitive intensity, color, orientation, texture… pixel, patch, window, region… implementation domain spatial, frequency pre-processing, post-processing parameters in all above aspects …

Putting our previous ‘salient window’ work in this terminology feature: color histogram primitive: window contrast context: global contrast measure: EMD domain: spatial pre-processing: segmentation Salient object detection by composition, Jie Feng, Yichen Wei, Litian Tao, Chao Zhang and Jian Sun, ICCV 2011

Contrast prior is insufficient Because saliency problem is highly ill-defined input true mask Itti et. al. PAMI 1998 Achanta et. al. CVPR 2009 Goferman et. al. CVPR 2010 Cheng et. al. CVPR 2011

?

𝐵 𝐹 𝐹 𝐵 The opposite question What is not foreground, or what is background? Spatial information matters arrangement, continuity… Exploit background priors boundary prior connectivity prior 𝐵 𝐹 𝐹 𝐵

Boundary and connectivity priors Salient objects do not touch image boundary Backgrounds are continuous and homogeneous

1. Boundary prior Salient objects do not touch image boundary a rule in photography more general than previous ‘image center bias’ exceptions, e.g., people cropped at image bottom

Evaluation of boundary prior Distribution of background pixel percentage only consider boundary pixels MSRA-1000 Berkeley-300

2. Connectivity prior Backgrounds are continuous and homogeneous common characteristics of natural images background patches are easily connected to each other connection is piecewise (e.g., sky and grass do not connect)

Geodesic saliency using background priors edge weight: appearance distance between adjacent patches background patch foreground patch Geodesic saliency: length of shortest path to image boundary 𝑠𝑎𝑙𝑖𝑒𝑛𝑐𝑦 𝑃 = 𝑚𝑖𝑛 𝑃 1 , 𝑃 2 ,…, 𝑃 𝑛 𝑖=1 𝑛−1 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒( 𝑃 𝑖 , 𝑃 𝑖+1 ) s.t. 𝑃 1 =𝑃, 𝑃 𝑛 𝑖𝑠 𝑜𝑛 𝑖𝑚𝑎𝑔𝑒 𝑏𝑜𝑢𝑛𝑑𝑎𝑟𝑦, 𝑃 𝑖 𝑖𝑠 𝑎𝑑𝑗𝑎𝑐𝑒𝑛𝑡 𝑡𝑜 𝑃 𝑖+1

Regular patches → superpixels better object boundary alignment and more accurate

Shortest paths and results

Comparison with other methods Itti et. al. PAMI 1998 Achanta et. al. CVPR 2009 Goferman et. al. CVPR 2010 Cheng et. al. CVPR 2011 input ours

Boundary prior could be too strict ? small cropping of object on the boundary causes large errors Image boundary needs more robust treatment

Refined geodesic saliency a virtual background node 𝐵 connected to boundary patches Geodesic saliency: length of shortest path to image boundary background node 𝐵 𝑠𝑎𝑙𝑖𝑒𝑛𝑐𝑦 𝑃 = 𝑚𝑖𝑛 𝑃 1 , 𝑃 2 ,…, 𝑃 𝑛 ,𝐵 𝑖=1 𝑛−1 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑃 𝑖 , 𝑃 𝑖+1 +𝑏𝑜𝑢𝑛𝑑𝑎𝑟𝑦 𝑤𝑒𝑖𝑔ℎ𝑡( 𝑃 𝑛 ,𝐵) 𝑠𝑎𝑙𝑖𝑒𝑛𝑐𝑦 𝑃 = 𝑚𝑖𝑛 𝑃 1 , 𝑃 2 ,…, 𝑃 𝑛 𝑖=1 𝑛−1 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒( 𝑃 𝑖 , 𝑃 𝑖+1 ) s.t. 𝑃 1 =𝑃, 𝑃 𝑛 𝑖𝑠 𝑜𝑛 𝑖𝑚𝑎𝑔𝑒 𝑏𝑜𝑢𝑛𝑑𝑎𝑟𝑦, 𝑃 𝑖 𝑖𝑠 𝑎𝑑𝑗𝑎𝑐𝑒𝑛𝑡 𝑡𝑜 𝑃 𝑖+1

Compute boundary weight 𝑏𝑜𝑢𝑛𝑑𝑎𝑟𝑦 𝑤𝑒𝑖𝑔ℎ𝑡 𝑃,𝐵 =𝑠𝑎𝑙𝑖𝑒𝑛𝑐𝑦 𝑜𝑓 𝑃 𝑜𝑛 𝑡ℎ𝑒 𝑏𝑜𝑢𝑛𝑑𝑎𝑟𝑦 ? boundary weight as a 1D saliency problem Goferman et. al. CVPR 2010 result with boundary weight result w/o boundary weight

Boundary weight improves results result w/o boundary weight result with boundary weight input boundary weight

“Small-weight-accumulation” problem if 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑃 𝑖 , 𝑃 𝑖+1 <𝑡, 𝑖𝑡 𝑖𝑠 𝑐𝑙𝑖𝑝𝑝𝑒𝑑 𝑡𝑜 0 𝑡: a small value indicating an insignificant distance with weight clipping

Weight clipping improves results w/o weight clipping with weight clipping

Advantages of geodesic saliency Effective combination of three priors moderate usage of contrast prior complementary to other algorithms Easy interpretation just one parameter: patch size (fixed as 1/40 image size) Super fast (2 ms, 400x400 image, regular patches)

Two salient object databases MSRA-1000, simple Berkeley-300, difficult one object large near center clean background one or multiple object different sizes different positions cluttered background

Running performance comparison methods time (ms) Our approach 2.0 FT (Achanta et. al. CVPR 2009) 8.5 LC (Zhai et. al. MM 2006) 9.6 HC (Cheng et. al. CVPR 2011) 10.1 SR (Hou et. al. CVPR 2007) 34 RC (Cheng et. al. CVPR 2011) 134.5 IT (Itti et. al. PAMI 1998) 483 GB (Harel et. al. NIPS 2006) 1557 CA (Goferman et. al. CVPR 2010) 59327

Performance evaluation on MSRA-1000 GS_GD: geodesic saliency using rectangular patches GS_SP: geodesic saliency using superpixels

Geodesic saliency is complementary to other algorithms Geodesic saliency relies on background priors previous methods mainly rely on contrast prior Combination improves both

Results on MSRA-1000 Image True Mask GS_GD GS_SP FT [9] CA [11] GB [22] RC [12]

Performance evaluation on Berkeley-300 GS_GD: geodesic saliency using rectangular patches GS_SP: geodesic saliency using superpixels

Results on Berkeley-300 Image True Mask GS_GD GS_SP FT [9] CA [11] GB [22] RC [12]

Failure examples

Summary of geodesic saliency Better usage of background priors State-of-the-art in both accuracy and efficiency Complementary to other methods