Introduction to Computer and Human Vision Shimon Ullman, Ronen Basri, Michal Irani Assistants: Tal Hassner Eli Shechtman

Misc...  Course website:  To be added to course mailing-list: send to  Other recommended courses (for credit): - Basic Topics - Statistical Machine Learning  Vision & Robotics Seminar (not for credit): Thursdays at 11:00-12:00 (Ziskind 1) send ask to be added to “seminar13” mailing list

 How is an image formed? (geometry and photometry)  What kind of operations can we apply to images?  What do images tell us about the world? (analysis & interpretation) Applications: - Manufacturing and inspection; QA - Robot navigation - Autonomous vehicles - Guiding tools for blind - Security and monitoring - Object/face recognition; OCR. - Medical Applications - Visualization; NVS - Visual communication - Digital libraries and video search - Video manipulation and editing

Tentative Schedule Lessons 7-9 (Michal): Motion and video analysis Lesson 10 (Ronen): Image Segmentation Lesson 11 (Ronen): Photometry Lesson 12 (Shimon): Object recognition Lessons (Shimon): Human Vision  3 programming exercises (MATLAB) -- CAN SUBMIT IN PAIRS  3-4 theoretical exercises -- MUST SUBMIT INDIVIDUALLY  EXAM Lessons 1-3 (Michal): Basic Image Processing Lessons 4-6 (Ronen): Stereo and Structure from Motion

Digital Images WorldCameraDigitizer Digital Image Image Formation: (i) What determines where the image of a 3D point appears on the 2D image? (ii) What determines how bright that image point is? (iii) How is a digital image represented? (iv) Some simple operations on 2D images? today

Digital Images WorldCameraDigitizer Digital Image PIXEL Typically: 0 = black 255 = white (picture element)

x = Grayscale Image y =

Three types of images: –Gray-scale images I(x,y)  [0..255] –Binary images I(x,y)  {0, 1} –Color images I R (x,y) I G (x,y) I B (x,y)

Color Image

Effects of down-sampling (reducing number of pixels) 4 x 4 16 x x 64 8 x 8 32 x x 128

2 gray levels (1 bit/pixel) BINARY IMAGE 4 gray levels (2 bits/pixel) 8 gray levels (3 bits/pixel) 16 gray levels (4 bits/pixel) 256 gray levels (8 bits/pixel) Effects of reducing number of gray levels

Occurrence (# of pixels) Gray Level The Image Histogram H(k) = #pixels with gray-level k Normalized histogram: H norm (k)=H(k)/N (N = # pixels in the image) Histogram = The gray-level distribution: Continuous probability density function:

The Image Histogram (Cont.) P I (k) k k P I (k) k 0.1

P I (k) k P I (k) k 0.1 Histogram Stretching

kk kk Histogram Equalization

OriginalEqualized Histogram Equalization

Histogram Equalization OriginalEqualized

k AaAa k AbAb Histogram Specification Transforms an image so that its histogram matches that of another image (e.g., for comparing two images of the same scene acquired under different lighting condition)

Image Enhancement 1) Gray value (histogram) Domain 2) Spatial Domain 3) Frequency Domain - Histogram stretching - Histogram equalization - Histogram specification - Gamma correction etc... noisy image (salt & pepper noise)

Spatial Operations g(x,y) = 1/M  f(n,m) (n,m) in  S S = neighborhood of pixel (x,y) M = number of pixels in neighborhood S e.g., Examples of neighborhoods S: 3 x 3 5 x 5 Replace center pixel with average/median level: (averaging mask; weighted mask; median filter…)

Salt & Pepper Noise 3 X 3 Average5 X 5 Average 7 X 7 AverageMedian Noise Cleaning

Salt & Pepper Noise 3 X 3 Average5 X 5 Average 7 X 7 AverageMedian Noise Cleaning

Other spatial filters Are strong brightness variations always noise…?

Edge Detection

Edge Types Line Edge gray value x edge gray value x edge Step Edge

Edge Detection by Differentiation gray value x 1D image f(x) 1st derivative f'(x) threshold |f'(x)| Edge Pixels: |f'(x)| > Threshold

Original image x derivative Gradient magnitude y derivative

Vertical edges Horizontal edges Edge Detection Image

Edge Detection Image

Image Sharpening Blurry ImageLaplacianSharpened Image Also Laplacian; Zero-crossings; Edge sharpening; etc….

The End...  Exercise#1: Noise Cleaning -- on course website (+ Matlab tutorial) DUE: Nov. 10 (in 2 weeks)  Course mailing list: Send to  Vision & Robotics Seminar: send ask to be added to “seminar13” mailing list

Panoramic Mosaic Image Original video clip Generated Mosaic image

Original Outliers Original Synthesized Video Removal

Image Segmentation

Photometric Stereo