1. SHAHAB
iCV Research Group

2. Image Processing Prospective

3. IP/IU
Wikipedia: image processing is any form of signal processing for which the input is an image.
Any kind of image manipulation is referred as image processing.

4. IP/IU
Vision is the process of discovering from images what is present in the world, and where it is.
-- David Marr, Vision (1982)
Humans can perceive and interpret images very fast and accurately.

5. Image Formation Scene Lens Shutter Area CCD 46 44 47 50 56 40 48 76 6269 45 36 39 88 87 65 38 90 41 43
112 77 66 72
108 74 98 86 91 83 57

6. What Do We See? 3D world 2D image Figures © Stephen E. Palmer, 2002
This emphasizes that interpretation is inherently a difficult problem. It is ill-posed because there an infinite number of interpretations of a 3D scene.
Figures © Stephen E. Palmer, 2002

7. What do we see? 3D world 2D image Painted backdrop
It is impossible to know from pure visual processing if this scene is a backdrop or a real 3D scene.
Painted
backdrop
Figures © Stephen E. Palmer, 2002

8. Brightness: Measurement vs. Perception
Visual system tries to undo the measured brightness into the reflectance and illumination and estimate the reflectance that is inherent to the object.
Do squares A and B have the same brightness?

9. Brightness: Measurement vs. Perception
Squares A and B have the same measured brightness but a different perceived brightness!

10. Lengths: Measurement vs. Perception
Müller-Lyer Illusion
Our perception of geometric properties is affected by our interpretation.

11. Which monster is larger?
Shepard RN (1990) Mind Sights: Original Visual Illusions, Ambiguities, and other Anomalies, New York: WH Freeman and Company
We can’t help but to integrate perspective cues into our interpretation of the image.

12. Find The Face In This “Coffee Beans” Image?
We ourselves are susceptible to clutter as well. This is a problem where computer might do faster than human.

13. Brain Fills In Occlusions

14. Segmentation and Grouping – Find The Dog
This scene is really a collection of many random spots and dots but the brain can group them together to segment the dog from the background.

15. Is Face Processing Orientation Dependent?

16. What is context?
Context is information relevant to the detection task but not directly due to the physical appearance of the object.
Wolf and Bileschi, 2004
What do we mean by “context”?
We can interpret the H or A based on context. Example from “Cognition in Action” Smyth Collins Morris Levy, 1994, LEA Publishers.
In our case, we think of “the object” as a person’s face.
Smyth et al., 1994

17. Role of Context in Image Understanding
TENNIS BALL
Inter-object semantics [Rabinovich 2007]

18. Examples Pedestrian detection Face detection Object recognition
Geolocating images from content
Event recognition using ground and aerial images
Annotation of pictures

19. General Object Recognition
Problem: Find many different types of general objects in images
Feature: Histogram of Orientated Gradients (HOG)
Dataset: PASCAL
Classifier: SVM + Graphical Model
Performance: Avg. Precision: horse 32%, person 42%, sofa 14%, car 33% On Inria Person Dataset: 86.9% avg. precision
Image
First feature selected by adaboost
Second feature selected by adaboost

20. Event Recognition Using Ground and Aerial Images
Problem: Find the event in an image from visual content and GPS coordinates
Feature: Scale Invariant Feature Transform (SIFT) and Color Moments
Dataset: Geo-tagged images from 1) Internet, and 2) Community collections
Classifier: SVM + Boosted Trees
Accuracy: theme-park 90%, forest 84%, beach 80%, city 58%, tennis 54%
Image
First feature selected by adaboost
Second feature selected by adaboost

21. Annotation of Pictures
Problem: Use visual content to annotate an image using a vocabulary
Feature: Daubechies wavelet coefficients
Dataset: Corel dataset + 54,700 images from Flickr
Classifier: Generalized mixture modeling + D2 clustering
Performance: Percentage of images correctly annotated by at least: 1 word 51%, 2 words 65%, 3 words 75%
Image
First feature selected by adaboost
Second feature selected by adaboost

22. Thank You