Model comparison and challenges II Compositional bias of salient object detection benchmarking Xiaodi Hou K-Lab, Computation and Neural Systems California.

Slides:



Advertisements
Similar presentations
Rich feature Hierarchies for Accurate object detection and semantic segmentation Ross Girshick, Jeff Donahue, Trevor Darrell, Jitandra Malik (UC Berkeley)
Advertisements

Sparsity and Saliency Xiaodi Hou K-Lab, Computation and Neural Systems California Institute of Technology for the Crash Course on Visual Saliency Modeling:
Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.
Analysis of Contour Motions Ce Liu William T. Freeman Edward H. Adelson Computer Science and Artificial Intelligence Laboratory Massachusetts Institute.
A generic model to compose vision modules for holistic scene understanding Adarsh Kowdle *, Congcong Li *, Ashutosh Saxena, and Tsuhan Chen Cornell University,
Ming-Ming Cheng 1 Ziming Zhang 2 Wen-Yan Lin 3 Philip H. S. Torr 1 1 Oxford University, 2 Boston University 3 Brookes Vision Group Training a generic objectness.
3 Small Comments Alex Berg Stony Brook University I work on recognition: features – action recognition – alignment – detection – attributes – hierarchical.
Leveraging Stereopsis for Saliency Analysis
Object class recognition using unsupervised scale-invariant learning Rob Fergus Pietro Perona Andrew Zisserman Oxford University California Institute of.
- Recovering Human Body Configurations: Combining Segmentation and Recognition (CVPR’04) Greg Mori, Xiaofeng Ren, Alexei A. Efros and Jitendra Malik -
Yuanlu Xu Human Re-identification: A Survey.
Foreground Modeling The Shape of Things that Came Nathan Jacobs Advisor: Robert Pless Computer Science Washington University in St. Louis.
Robust Object Tracking via Sparsity-based Collaborative Model
Object-centric spatial pooling for image classification Olga Russakovsky, Yuanqing Lin, Kai Yu, Li Fei-Fei ECCV 2012.
Hierarchical Saliency Detection School of Electronic Information Engineering Tianjin University 1 Wang Bingren.
Active Contour Models (Snakes)
Biased Normalized Cuts 1 Subhransu Maji and Jithndra Malik University of California, Berkeley IEEE Conference on Computer Vision and Pattern Recognition.
Control of Attention and Gaze in the Natural World.
Image Retrieval Using Eye Movements Fred Stentiford & Wole Oyekoya University College London.
Stas Goferman Lihi Zelnik-Manor Ayellet Tal. …
Statistical Recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and Kristen Grauman.
Fast, Multiscale Image Segmentation: From Pixels to Semantics Ronen Basri The Weizmann Institute of Science Joint work with Achi Brandt, Meirav Galun,
A Study of Approaches for Object Recognition
Saliency & attention (P) Lavanya Sharan April 4th, 2011.
Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.
Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.
Introduction of Saliency Map
Computer Vision Systems for the Blind and Visually Disabled. STATS 19 SEM Talk 3. Alan Yuille. UCLA. Dept. Statistics and Psychology.
Speaker: Chi-Yu Hsu Advisor: Prof. Jian-Jung Ding Leveraging Stereopsis for Saliency Analysis, CVPR 2012.
Computer Science Department, Duke UniversityPhD Defense TalkMay 4, 2005 Fast Extraction of Feature Salience Maps for Rapid Video Data Analysis Nikos P.
Salient Object Detection by Composition
Performance Evaluation of Grouping Algorithms Vida Movahedi Elder Lab - Centre for Vision Research York University Spring 2009.
Overcoming Dataset Bias: An Unsupervised Domain Adaptation Approach Boqing Gong University of Southern California Joint work with Fei Sha and Kristen Grauman.
Scott Helmer, David Meger, Pooja Viswanathan, Sancho McCann, Matthew Dockrey, Pooyan Fazli, Tristram Southey, Marius Muja, Michael Joya, Jim Little, David.
Vision System for Wing Beat Analysis of Bats in the Wild 1 Boston University Department of Computer Science 2 Boston University Department of Biology Mikhail.
Object Bank Presenter : Liu Changyu Advisor : Prof. Alex Hauptmann Interest : Multimedia Analysis April 4 th, 2013.
Computer Vision Why study Computer Vision? Images and movies are everywhere Fast-growing collection of useful applications –building representations.
Assessment of Computational Visual Attention Models on Medical Images Varun Jampani 1, Ujjwal 1, Jayanthi Sivaswamy 1 and Vivek Vaidya 2 1 CVIT, IIIT Hyderabad,
BING: Binarized Normed Gradients for Objectness Estimation at 300fps
Deformable Part Model Presenter : Liu Changyu Advisor : Prof. Alex Hauptmann Interest : Multimedia Analysis April 11 st, 2013.
BAGGING ALGORITHM, ONLINE BOOSTING AND VISION Se – Hoon Park.
Why is computer vision difficult?
MSRI workshop, January 2005 Object Recognition Collected databases of objects on uniform background (no occlusions, no clutter) Mostly focus on viewpoint.
UNBIASED LOOK AT DATASET BIAS Antonio Torralba Massachusetts Institute of Technology Alexei A. Efros Carnegie Mellon University CVPR 2011.
Scene Completion Using Millions of Photographs James Hays, Alexei A. Efros Carnegie Mellon University ACM SIGGRAPH 2007.
Geodesic Saliency Using Background Priors
Geodesic Flow Kernel for Unsupervised Domain Adaptation Boqing Gong University of Southern California Joint work with Yuan Shi, Fei Sha, and Kristen Grauman.
Segmentation of Vehicles in Traffic Video Tun-Yu Chiang Wilson Lau.
Recognition Using Visual Phrases
AAM based Face Tracking with Temporal Matching and Face Segmentation Mingcai Zhou 1 、 Lin Liang 2 、 Jian Sun 2 、 Yangsheng Wang 1 1 Institute of Automation.
Colour and Texture. Extract 3-D information Using Vision Extract 3-D information for performing certain tasks such as manipulation, navigation, and recognition.
Stas Goferman Lihi Zelnik-Manor Ayellet Tal Technion.
Spatio-temporal saliency model to predict eye movements in video free viewing Gipsa-lab, Grenoble Département Images et Signal CNRS, UMR 5216 S. Marat,
 Mentor : Prof. Amitabha Mukerjee Learning to Detect Salient Objects Team Members - Avinash Koyya Diwakar Chauhan.
Minimum Barrier Salient Object Detection at 80 FPS JIANMING ZHANG, STAN SCLAROFF, ZHE LIN, XIAOHUI SHEN, BRIAN PRICE, RADOMIR MECH IEEE INTERNATIONAL CONFERENCE.
ICCV 2009 Tilke Judd, Krista Ehinger, Fr´edo Durand, Antonio Torralba.
- photometric aspects of image formation gray level images
HFS: Hierarchical Feature Selection for Efficient Image Segmentation
Evaluating Techniques for Image Classification
Enhanced-alignment Measure for Binary Foreground Map Evaluation
Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science
Bringing Salient Object Detection to the Foreground
Saliency detection Donghun Yeo CV Lab..
Enhanced-alignment Measure for Binary Foreground Map Evaluation
Anomaly Detection in Crowded Scenes
Liyuan Li, Jerry Kah Eng Hoe, Xinguo Yu, Li Dong, and Xinqi Chu
Heterogeneous convolutional neural networks for visual recognition
Saliency Optimization from Robust Background Detection
SDSEN: Self-Refining Deep Symmetry Enhanced Network
Presentation transcript:

Model comparison and challenges II Compositional bias of salient object detection benchmarking Xiaodi Hou K-Lab, Computation and Neural Systems California Institute of Technology for the Crash Course on Visual Saliency Modeling: Behavioral Findings and Computational Models CVPR 2013

Schedule

On detecting salient objects Learning to Detect A Salient Object [Liu et. al., CVPR 07] Frequency-tuned Salient Region Detection [Achanta et. al., CVPR 09]

The progress! Some top performers: – [PCA] – [PCA] What makes a patch distinct [Margolin et. al., CVPR 13] – [SF] – [SF]Saliency filters [Perazzi et. al., CVPR 12]: F-Measure: 0.84 – [GC]/[GC-seg] – [GC]/[GC-seg]Global contrast-based salient region detection [Cheng et. al., CVPR 11] F-Measure: 0.75 – [FT] – [FT] Frequency Tuned Salient Region Detection [Achanta et. a.l., CVPR 09] : 0.65 by [Achanta et. al., CVPR 09]. Image from [Perazzi et. al., CVPR 2012]

The progress? Salient objects in PASCAL VOC? – 850 images from VOC 2013 validation set. – Intersection of main challenge and segmentation challenge. – Answers more questions: Where is your algorithm (in salient object detection)? Where is salient object detection (in computer vision). Where is salient object detection (in computer vision).

The progress FT: 0.28 GC: 0.39 SF: 0.35 PCA: 0.40 GC-seg: % performance drop!!

The arguments No!! These objects are not salient! images with salient objects Our algorithm works on images with salient objects only!

The paradox of salient object detection But hey, what is a “salient object”?

COMPOSITIONAL BIAS

Before we proceed… Google Image Search: “science” – Rutherford atomic model (9) – Test tubes (10) – Microscopes (4) – Double helix (3) – Old guys with crazy hair and glasses (3)

How to compose a biased salient object detection dataset Decide to build a new salient object dataset! So what is saliency? Searching for unambiguous examples of saliency… Found one! Add to my dataset! Job done! Let other people play with my dataset!

The compositional bias Compositional bias composition Compositional bias: Biases introduced during the composition of a dataset: – Exaggerating on stereotypical attributes. Limited variability in positive samples. Lack of negative samples at all. Unlike datasets in machine learning, where the dataset is the world, computer vision datasets are supposed to be a representation of the world [Torralba and Efros: Unbiased look at Dataset bias]

Compositional bias: the statistics Object number

Compositional bias: the statistics Object eccentricity

Compositional bias: the statistics Global foreground and background contrast

Compositional bias: the statistics Local foreground/background contrast (contour strength)

TOWARDS A BETTER SALIENT OBJECT DATASET

The new project salient object detection object detection Build a salient object detection dataset from a good object detection dataset (e.g. PASCAL VOC). Let the eye fixations pick up those salient objects!

Data collection (in process) SR Research EyeLink sec viewing time. “Free-viewing” instruction (will mention it later). 3 subjects (more subjects on the way).

What makes an object salient Unit conversion: – From fixation maps – To object fixation score sum of blurred fixation map intensity within the object mask.

Object size and saliency Large objects attract more fixations. Small objects receive denser fixations.

Object size and saliency

Objects, salient objects, and the most salient objects Salient objects: – Fixation score higher than mean (67.3% objects). Most salient objects: – Fixation score higher than mean*2 (27.8% objects). Image with fixationObject labelingSalient objectsMost salient object(s)

Salient objects and salient object detection Guess how does the algorithms perform on “salient objects” and “most salient objects”? On all objects: FT: 0.28 GC: 0.39 SF: 0.35 PC: 0.38

Testing on salient objects Salient objects on PASCAL VOC 60% performance drop!! FT: 0.22 GC: 0.35 SF: 0.31 PCA: 0.38 GC-seg: 0.39

Testing on most salient objects Most salient objects on PASCAL VOC FT: 0.10 GC: 0.20 SF: 0.15 PCA: 0.26 GC-seg: % performance drop!!

Something is wrong, seriously!

DISCUSSIONS

The role of saliency in a visual system Bad performance because of boundary detection? Bad performance because of unpredictability of human “free will”?

Saliency as an oracle Oracle selecting the best segment – CPMC: 78% from 154 segments – gPB: 61% from 1286 segments * coverage = intersect/union

Saliency and tasks Build a salient object detection dataset from an egocentric object dataset. Let the eye-fixation speaks Eye Tracker Forward-looking Camera Learning to recognize daily actions using gaze, [Fathi et. al. ECCV 12]

What makes an object salient? TaskObjectSaliency Object in egocentric actions Fixated object == Manipulated object?

THANKS

Acknowledgement Joint work with Yin Gatech. Special thanks to Nathan Faivre for his kind help on eye tracking.

Open discussions