1. Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha
Stanford CS223B Computer Vision, Winter Lecture 2 Lenses, Filters, Features
Professor Sebastian Thrun
CAs: Dan Maynes-Aminzade and Mitul Saha
[with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…]

2. Today’s Goals Thin Lens Aberrations Features 101
Linear Filters and Edge Detection

3. Pinhole Camera (last Wednesday)
“shards of diamond” – those little pieces of beautiful truth scattered about.
-- Brunelleschi, XVth Century
Marc Pollefeys comp256, Lect 2

4. Snell’s Law
Snell’s law
n1 sin a1 = n2 sin a2

5. Thin Lens: Definition f optical axis focus
Spherical lense surface: Parallel rays are refracted to single point

6. Thin Lens: Projection f z optical axis Image plane
Spherical lense surface: Parallel rays are refracted to single point

7. Thin Lens: Projection f f z optical axis Image plane
Spherical lense surface: Parallel rays are refracted to single point

8. Thin Lens: Properties
Any ray entering a thin lens parallel to the optical axis must go through the focus on other side
Any ray entering through the focus on one side will be parallel to the optical axis on the other side

9. Thin Lens: Model Q P O Fr Fl p R Z f f z

10. Transformation

11. A Transformation

12. The Thin Lens Law Q P O Fr Fl p R Z f f z

13. The Thin Lens Law

14. Limits of the Thin Lens Model
3 assumptions :
all rays from a point are focused onto 1 image point
Remember thin lens small angle assumption
2. all image points in a single plane
3. magnification is constant
Deviations from this ideal are aberrations

15. Today’s Goals Thin Lens Aberrations Features 101
Linear Filters and Edge Detection

16. Aberrations 2 types : geometrical : geometry of the lense,
small for paraxial rays
chromatic : refractive index function of
wavelength
Marc Pollefeys

17. Geometrical Aberrations
spherical aberration
astigmatism
distortion
coma
aberrations are reduced by combining lenses

18. Astigmatism
Different focal length for inclined rays
Marc Pollefeys

19. Astigmatism
Different focal length for inclined rays
Marc Pollefeys

20. Spherical Aberration rays parallel to the axis do not converge
outer portions of the lens yield smaller
focal lenghts

21. Can be corrected! (if parameters are know)
Distortion
magnification/focal length different
for different angles of inclination
pincushion
(tele-photo)
barrel
(wide-angle)
Can be corrected! (if parameters are know)
Marc Pollefeys

22. Coma
point off the axis depicted as comet shaped blob
Marc Pollefeys

23. Chromatic Aberration rays of different wavelengths focused
in different planes
cannot be removed completely
Marc Pollefeys

24. Vignetting
Effect: Darkens pixels near the image boundary

25. CCD vs. CMOS Mature technology Specific technology Recent technology
High production cost
High power consumption
Higher fill rate
Blooming
Sequential readout
Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration with other components
Marc Pollefeys

26. Today’s Goals Thin Lens Aberrations Features 101
Linear Filters and Edge Detection

27. Today’s Question What is a feature? What is an image filter?
How can we find corners?
How can we find edges?
(How can we find cars in images?)

28. What is a Feature?
Local, meaningful, detectable parts of the image

29. Features in Computer Vision
What is a feature?
Location of sudden change
Why use features?
Information content high
Invariant to change of view point, illumination
Reduces computational burden

30. (One Type of) Computer Vision
Feature 1
Feature 2 :
Feature N
Image 2
Image 1
Feature 1
Feature 2 :
Feature N
Computer
Vision
Algorithm

31. Where Features Are Used
Calibration
Image Segmentation
Correspondence in multiple images (stereo, structure from motion)
Object detection, classification

32. What Makes For Good Features?
Invariance
View point (scale, orientation, translation)
Lighting condition
Object deformations
Partial occlusion
Other Characteristics
Uniqueness
Sufficiently many
Tuned to the task

33. Today’s Goals Features 101 Linear Filters and Edge Detection
Canny Edge Detector

34. What Causes an Edge? Depth discontinuity
Surface orientation discontinuity
Reflectance discontinuity (i.e., change in surface material properties)
Illumination discontinuity (e.g., shadow)
Slide credit: Christopher Rasmussen

35. Quiz: How Can We Find Edges?

36. Edge Finding 101 im = imread('bridge.jpg'); image(im); figure(2);
bw = double(rgb2gray(im));
image(bw);
gradkernel = [-1 1];
dx = abs(conv2(bw, gradkernel, 'same'));
image(dx);
colorbar; colormap gray
[dx,dy] = gradient(bw);
gradmag = sqrt(dx.^2 + dy.^2);
image(gradmag);
matlab
colorbar
colormap(gray(255))
colormap(default)

37. Edge Finding 101
Example of a linear Filter

38. Today’s Goals Thin Lens Aberrations Features 101
Linear Filters and Edge Detection

39. What is Image Filtering?
Modify the pixels in an image based on some function of a local neighborhood of the pixels 10 5 3 4 1 7 7
Some function

40. Linear Filtering Linear case is simplest and most useful
Replace each pixel with a linear combination of its neighbors.
The prescription for the linear combination is called the convolution kernel. 10 5 3 4 1 7
0.5
1.0 7
kernel

41. Linear Filter = Convolution
g11
g11 I(i-1,j-1)
I(.)
g12
+ g12 I(i-1,j)
I(.)
g13
+ g13 I(i-1,j+1) +
I(.)
g21
g21 I(i,j-1)
g22
+ g22 I(i,j)
I(.)
I(.)
g23
+ g23 I(i,j+1) +
I(.)
g31
g31 I(i+1,j-1)
I(.)
g32
+ g32 I(i+1,j)
I(.)
g33
+ g33 I(i+1,j+1)
f (i,j) =

42. Linear Filter = Convolution

43. Filtering Examples

44. Filtering Examples

45. Filtering Examples

46. Image Smoothing With Gaussian
figure(3);
sigma = 3;
width = 3 * sigma;
support = -width : width;
gauss2D = exp( - (support / sigma).^2 / 2);
gauss2D = gauss2D / sum(gauss2D);
smooth = conv2(conv2(bw, gauss2D, 'same'), gauss2D', 'same');
image(smooth);
colormap(gray(255));
gauss3D = gauss2D' * gauss2D;
tic ; smooth = conv2(bw,gauss3D, 'same'); toc

47. Smoothing With Gaussian
Averaging
Gaussian
Slide credit: Marc Pollefeys

48. Smoothing Reduces Noise
The effects of smoothing
Each row shows smoothing
with gaussians of different
width; each column shows
different realizations of
an image of gaussian noise.
Slide credit: Marc Pollefeys

49. Example of Blurring
Image
Blurred Image - =

50. Edge Detection With Smoothed Images
figure(4);
[dx,dy] = gradient(smooth);
gradmag = sqrt(dx.^2 + dy.^2);
gmax = max(max(gradmag));
imshow(gradmag);
colormap(gray(gmax));

51. Scale Increased smoothing: Eliminates noise edges.
Makes edges smoother and thicker.
Removes fine detail.

52. The Edge Normal

53. Displaying the Edge Normal
figure(5);
hold on;
image(smooth);
colormap(gray(255));
[m,n] = size(gradmag);
edges = (gradmag > 0.3 * gmax);
inds = find(edges);
[posx,posy] = meshgrid(1:n,1:m); posx2=posx(inds); posy2=posy(inds);
gm2= gradmag(inds);
sintheta = dx(inds) ./ gm2;
costheta = - dy(inds) ./ gm2;
quiver(posx2,posy2, gm2 .* sintheta / 10, -gm2 .* costheta / 10,0);
hold off;

54. Separable Kernels

55. Combining Kernels / Convolutions

56. Effect of Smoothing Radius
1 pixel
3 pixels
7 pixels

57. Robert’s Cross Operator
1 0
0 -1
0 1
-1 0 +
S =
[ I(x, y) - I(x+1, y+1) ]2 + [ I(x, y+1) - I(x+1, y) ]2 or
S =
| I(x, y) - I(x+1, y+1) | + | I(x, y+1) - I(x+1, y) |

58. Sobel Operator -1 -2 -1 0 0 0 1 2 1 -1 0 1 -2 0 2 S1= S2 =
S1=
S2 =
Edge Magnitude =
Edge Direction =
S1 + S1 2
tan-1 S1 S2

59. The Sobel Kernel, Explained
Sobel kernel is separable!
Averaging done parallel to edge

60. Sobel Edge Detector
figure(6)
edge(bw, 'sobel')

61. Robinson Compass Masks

62. Claim Your Own Kernel!
Frei & Chen

63. Comparison (by Allan Hanson)
Analysis based on a step edge inclined at an angle q (relative to y-axis) through center of window.
Robinson/Sobel: true edge contrast less than 1.6% different from that computed by the operator.
Error in edge direction
Robinson/Sobel: less than 1.5 degrees error
Prewitt: less than 7.5 degrees error
Summary
Typically, 3 x 3 gradient operators perform better than 2 x 2.
Prewitt2 and Sobel perform better than any of the other 3x3 gradient estimation operators.
In low signal to noise ratio situations, gradient estimation operators of size larger than 3 x 3 have improved performance.
In large masks, weighting by distance from the central pixel is beneficial.