A Database of Human Segmented Natural Images and Two Applications David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik UC Berkeley

Slides:

Advertisements

Similar presentations

Shape Matching and Object Recognition using Low Distortion Correspondence Alexander C. Berg, Tamara L. Berg, Jitendra Malik U.C. Berkeley.

Advertisements

Computer Vision Group UC Berkeley How should we combine high level and low level knowledge? Jitendra Malik UC Berkeley Recognition using regions is joint.

Texture. Limitation of pixel based processing Edge detection with different threshold.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Shape Sharing for Object Segmentation

Recovering Human Body Configurations: Combining Segmentation and Recognition Greg Mori, Xiaofeng Ren, and Jitentendra Malik (UC Berkeley) Alexei A. Efros.

Texture. Edge detectors find differences in overall intensity. Average intensity is only simplest difference. many slides from David Jacobs.

Proportion Priors for Image Sequence Segmentation Claudia Nieuwenhuis, etc. ICCV 2013 Oral.

Qualifying Exam: Contour Grouping Vida Movahedi Supervisor: James Elder Supervisory Committee: Minas Spetsakis, Jeff Edmonds York University Summer 2009.

Computer Vision Group University of California Berkeley Ecological Statistics of Good Continuation: Multi-scale Markov Models for Contours Xiaofeng Ren.

1 Contours and Junctions in Natural Images Jitendra Malik University of California at Berkeley (with Jianbo Shi, Thomas Leung, Serge Belongie, Charless.

Ghunhui Gu, Joseph J. Lim, Pablo Arbeláez, Jitendra Malik University of California at Berkeley Berkeley, CA

Biased Normalized Cuts 1 Subhransu Maji and Jithndra Malik University of California, Berkeley IEEE Conference on Computer Vision and Pattern Recognition.

Computer Vision Group, University of BonnVision Laboratory, Stanford University Abstract This paper empirically compares nine image dissimilarity measures.

Computer Vision Group University of California Berkeley Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint.

The Visual Recognition Machine Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

Lecture 6 Image Segmentation

Recognition using Regions CVPR Outline Introduction Overview of the Approach Experimental Results Conclusion.

Computer Vision Group University of California Berkeley 1 Learning Scale-Invariant Contour Completion Xiaofeng Ren, Charless Fowlkes and Jitendra Malik.

Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues David R. Martin Charless C. Fowlkes Jitendra Malik.

Understanding Gestalt Cues and Ecological Statistics using a Database of Human Segmented Images Charless Fowlkes, David Martin and Jitendra Malik Department.

Visual Grouping and Recognition Jitendra Malik U.C. Berkeley Jitendra Malik U.C. Berkeley.

CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.

Abstract We present a model of curvilinear grouping using piecewise linear representations of contours and a conditional random field to capture continuity.

Measuring the Ecological Statistics of Figure-Ground Charless Fowlkes, David Martin, Jitendra Malik.

1 Learning to Detect Natural Image Boundaries David Martin, Charless Fowlkes, Jitendra Malik Computer Science Division University of California at Berkeley.

CVR05 University of California Berkeley 1 Familiar Configuration Enables Figure/Ground Assignment in Natural Scenes Xiaofeng Ren, Charless Fowlkes, Jitendra.

Computer Vision Group University of California Berkeley Visual Grouping and Object Recognition Jitendra Malik * U.C. Berkeley * with S. Belongie, C. Fowlkes,

Berkeley Vision GroupNIPS Vancouver Learning to Detect Natural Image Boundaries Using Local Brightness,

CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.

Measuring the Ecological Statistics of Figure-Ground Charless Fowlkes, David Martin, Jitendra Malik.

Face Recognition Based on 3D Shape Estimation

1 The Ecological Statistics of Grouping by Similarity Charless Fowlkes, David Martin, Jitendra Malik Computer Science Division University of California.

Visual Grouping and Recognition David Martin UC Berkeley David Martin UC Berkeley.

Efficient Spatiotemporal Grouping Using the Nyström Method Charless Fowlkes, U.C. Berkeley Serge Belongie, U.C. San Diego Jitendra Malik, U.C. Berkeley.

WORD-PREDICTION AS A TOOL TO EVALUATE LOW-LEVEL VISION PROCESSES Prasad Gabbur, Kobus Barnard University of Arizona.

Computational Vision Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

Segmentation and Perceptual Grouping. The image of this cube contradicts the optical image.

On Measuring * the Ecological Validity of Local Figure-Ground Cues Charless Fowlkes, David Martin, Jitendra Malik Computer Science Division University.

1 How do ideas from perceptual organization relate to natural scenes?

Computer Vision Group University of California Berkeley Matching Shapes Serge Belongie *, Jitendra Malik and Jan Puzicha U.C. Berkeley * Present address:

1 Ecological Statistics and Perceptual Organization Charless Fowlkes work with David Martin and Jitendra Malik at University of California at Berkeley.

Computer Vision Group University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes and Jitendra Malik.

Computer Vision Group University of California Berkeley Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA Greg Mori and Jitendra Malik.

Autoencoders Mostafa Heidarpour

Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes and Jitendra Malik, U.C. Berkeley We present a model of edge and region grouping.

Heather Dunlop : Advanced Perception January 25, 2006

Performance Evaluation of Grouping Algorithms Vida Movahedi Elder Lab - Centre for Vision Research York University Spring 2009.

1 Patch Complexity, Finite Pixel Correlations and Optimal Denoising Anat Levin, Boaz Nadler, Fredo Durand and Bill Freeman Weizmann Institute, MIT CSAIL.

Recognition using Regions (Demo) Sudheendra V. Outline Generating multiple segmentations –Normalized cuts [Ren & Malik (2003)] Uniform regions –Watershed.

1 Contours and Junctions in Natural Images Jitendra Malik University of California at Berkeley (with Jianbo Shi, Thomas Leung, Serge Belongie, Charless.

Building local part models for category-level recognition C. Schmid, INRIA Grenoble Joint work with G. Dorko, S. Lazebnik, J. Ponce.

A Comparison Between Bayesian Networks and Generalized Linear Models in the Indoor/Outdoor Scene Classification Problem.

Lecture notes for Stat 231: Pattern Recognition and Machine Learning 1. Stat 231. A.L. Yuille. Fall 2004 Practical Issues with SVM. Handwritten Digits:

Visual Grouping and Recognition Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

Segmentation and Boundary Detection Using Multiscale Intensity Measurements Eitan Sharon, Meirav Galun, Ronen Basri, Achi Brandt Dept. of Computer Science.

Computer Vision Group University of California Berkeley On Visual Recognition Jitendra Malik UC Berkeley.

Layered Object Detection for Multi-Class Image Segmentation UC Irvine Yi Yang Sam Hallman Deva Ramanan Charless Fowlkes.

October Andrew C. Gallagher, Jiebo Luo, Wei Hao Improved Blue Sky Detection Using Polynomial Model Fit Andrew C. Gallagher, Jiebo Luo, Wei Hao Presented.

Digital Image Processing

Object Recognition by Integrating Multiple Image Segmentations Caroline Pantofaru, Cordelia Schmid, Martial Hebert ECCV 2008 E.

Computational Vision Jitendra Malik University of California, Berkeley.

Lecture 30: Segmentation CS4670 / 5670: Computer Vision Noah Snavely From Sandlot ScienceSandlot Science.

In: Pattern Analysis and Machine Intelligence, IEEE Transactions on, Vol. 30, Nr. 1 (2008), p Group of Adjacent Contour Segments for Object Detection.

Texture Analysis and Synthesis. Texture Texture: pattern that “looks the same” at all locationsTexture: pattern that “looks the same” at all locations.

Digit Recognition Using SIS Testbed Mengjie Mao. Overview Cycle 1: sequential component AAM training Cycle 2: sequential components Identifier 0 Ten perfect.

Can Computer Algorithms Guess Your Age and Gender?

Enhanced-alignment Measure for Binary Foreground Map Evaluation

Contours and Junctions in Natural Images

Learning to Combine Bottom-Up and Top-Down Segmentation

Presentation transcript:

A Database of Human Segmented Natural Images and Two Applications David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik UC Berkeley

David Martin - UC Berkeley - ICCV Motivation Berkeley Segmentation Dataset  Groundtruth for image segmentation of natural images App#1: A segmentation benchmark App#2: Ecological statistics

David Martin - UC Berkeley - ICCV Benchmark Example for Recognition MNIST handwritten digit dataset [LeCun, AT&T] Training set, test set, evaluation methodology, algorithm ranking

David Martin - UC Berkeley - ICCV The Image Dataset 1000 Corel images –Photographs of outdoor scenes –Texture is common –Large variety of subject matter –481 x 321 x 24b

David Martin - UC Berkeley - ICCV 20015

6

7

8

9 Establishing Groundtruth Def: Segmentation = Partition of image pixels into exclusive sets Manual segmentation by human subjects –Custom Java tool to facilitate task Currently: 1000 images, 5500 segmentations, 20 subjects Naïve subjects (UCB undergrads) given simple, non-technical instructions

David Martin - UC Berkeley - ICCV Directions to Image Segmentors You will be presented a photographic image Divide the image into some number of segments, where the segments represent “things” or “parts of things” in the scene The number of segments is up to you, as it depends on the image. Something between 2 and 30 is likely to be appropriate. It is important that all of the segments have approximately equal importance.

David Martin - UC Berkeley - ICCV

David Martin - UC Berkeley - ICCV

David Martin - UC Berkeley - ICCV The segmentations are not identical. But are they consistent??

David Martin - UC Berkeley - ICCV Perceptual organization forms a hierarchy image backgroundleft birdright bird grassbush headeye beakfar body headeye beak body Each subject picks a slice through this hierarchy.

David Martin - UC Berkeley - ICCV Quantifying inconsistency S1S1 S2S2 How much is S 1 a refinement of S 2 at pixel  ?

David Martin - UC Berkeley - ICCV Segmentation Error Measure One-way Local Refinement Error: Segmentation Error allows refinement in either direction at each pixel:

David Martin - UC Berkeley - ICCV Human segmentations are consistent Distribution of segmentation error over the dataset.

David Martin - UC Berkeley - ICCV ColorGrayInvNeg

David Martin - UC Berkeley - ICCV InvNeg

David Martin - UC Berkeley - ICCV ColorGrayInvNeg

David Martin - UC Berkeley - ICCV Gray vs. Color vs. InvNeg Segmentations SE (gray, gray) = SE (gray, color) = Color may affect attention, but doesn’t seem to affect perceptual organization SE (gray, gray) = SE (gray, invneg) = InvNeg interferes with high-level cues (2500 gray, 2500 color,200 invneg segmentations)

David Martin - UC Berkeley - ICCV Benchmark Methodology Separate training and test datasets with no images in common Generate computer segmentation(s) of each image in test set –Determine error of each computer segmentation using SE measure –Algorithm scored by mean SE Example: –SE (human, human) = 0.05 –SE (NCuts, human) = 0.22 –SE (different images) = 0.30

David Martin - UC Berkeley - ICCV Ecological Statistics of Image Segmentations Validating and quantifying Gestalt grouping factors [Brunswik 1953] Priors on region properties Recent work on natural image statistics: –Filter outputs [Ruderman 1994, Olshausen & Field 1996, Yuille et. al. 1999] –Object sizes [Alvarez, Gousseau, Morel 1999] –Shape [Zhu 1999] –Contours [August & Zucker 2000, Geisler et al. 2001]

David Martin - UC Berkeley - ICCV Relative power of cues Pairwise grouping cues –Proximity –Luminance similarity –Color similarity –Intervening contour –Texture similarity

David Martin - UC Berkeley - ICCV P (Same Segment | Proximity)

David Martin - UC Berkeley - ICCV P (Same Segment | Luminance)

David Martin - UC Berkeley - ICCV Bayes Risk for Proximity Cue

David Martin - UC Berkeley - ICCV Bayes Risk for Various Cues Conditioned on Proximity

David Martin - UC Berkeley - ICCV Mutual Information for Various Cues Conditioned on Proximity

David Martin - UC Berkeley - ICCV Priors on Region Properties Area Convexity

David Martin - UC Berkeley - ICCV Empirical Distribution of Region Area y = Kx -   = Compare with Alvarez, Gousseau, Morel 1999.

David Martin - UC Berkeley - ICCV Empirical Distribution of Region Convexity

David Martin - UC Berkeley - ICCV Conclusion Large new database of segmentations of natural images by humans A segmentation benchmark Ecological statistics –Relative power of grouping cues –Priors on region properties