SHAHAB iCV Research Group

Image Processing Prospective

IP/IU Wikipedia: image processing is any form of signal processing for which the input is an image. Any kind of image manipulation is referred as image processing.

IP/IU Vision is the process of discovering from images what is present in the world, and where it is. -- David Marr, Vision (1982) Humans can perceive and interpret images very fast and accurately.

Image Formation Scene Lens Shutter Area CCD 46 44 47 50 56 40 48 76 62 69 45 36 39 88 87 65 38 90 41 43 112 77 66 72 108 74 98 86 91 83 57

What Do We See? 3D world 2D image Figures © Stephen E. Palmer, 2002 This emphasizes that interpretation is inherently a difficult problem. It is ill-posed because there an infinite number of interpretations of a 3D scene. Figures © Stephen E. Palmer, 2002

What do we see? 3D world 2D image Painted backdrop It is impossible to know from pure visual processing if this scene is a backdrop or a real 3D scene. Painted backdrop Figures © Stephen E. Palmer, 2002

Brightness: Measurement vs. Perception Visual system tries to undo the measured brightness into the reflectance and illumination and estimate the reflectance that is inherent to the object. Do squares A and B have the same brightness?

Brightness: Measurement vs. Perception Squares A and B have the same measured brightness but a different perceived brightness!

Lengths: Measurement vs. Perception Müller-Lyer Illusion Our perception of geometric properties is affected by our interpretation. http://www.michaelbach.de/ot/sze_muelue/index.html

Which monster is larger? Shepard RN (1990) Mind Sights: Original Visual Illusions, Ambiguities, and other Anomalies, New York: WH Freeman and Company We can’t help but to integrate perspective cues into our interpretation of the image.

Find The Face In This “Coffee Beans” Image? We ourselves are susceptible to clutter as well. This is a problem where computer might do faster than human. http://www.michaelbach.de/ot/sze_muelue/index.html

Brain Fills In Occlusions

Segmentation and Grouping – Find The Dog This scene is really a collection of many random spots and dots but the brain can group them together to segment the dog from the background.

Is Face Processing Orientation Dependent?

What is context? Context is information relevant to the detection task but not directly due to the physical appearance of the object. Wolf and Bileschi, 2004 What do we mean by “context”? We can interpret the H or A based on context. Example from “Cognition in Action” Smyth Collins Morris Levy, 1994, LEA Publishers. In our case, we think of “the object” as a person’s face. Smyth et al., 1994

Role of Context in Image Understanding TENNIS BALL Inter-object semantics [Rabinovich 2007]

Examples Pedestrian detection Face detection Object recognition Geolocating images from content Event recognition using ground and aerial images Annotation of pictures

General Object Recognition Problem: Find many different types of general objects in images Feature: Histogram of Orientated Gradients (HOG) Dataset: PASCAL Classifier: SVM + Graphical Model Performance: Avg. Precision: horse 32%, person 42%, sofa 14%, car 33% On Inria Person Dataset: 86.9% avg. precision Image First feature selected by adaboost Second feature selected by adaboost

Event Recognition Using Ground and Aerial Images Problem: Find the event in an image from visual content and GPS coordinates Feature: Scale Invariant Feature Transform (SIFT) and Color Moments Dataset: Geo-tagged images from 1) Internet, and 2) Community collections Classifier: SVM + Boosted Trees Accuracy: theme-park 90%, forest 84%, beach 80%, city 58%, tennis 54% Image First feature selected by adaboost Second feature selected by adaboost

Annotation of Pictures Problem: Use visual content to annotate an image using a vocabulary Feature: Daubechies wavelet coefficients Dataset: Corel dataset + 54,700 images from Flickr Classifier: Generalized mixture modeling + D2 clustering Performance: Percentage of images correctly annotated by at least: 1 word 51%, 2 words 65%, 3 words 75% Image First feature selected by adaboost Second feature selected by adaboost

Thank You