1. 3D Imaging
Midterm Review

2. Pinhole cameras Abstract camera model - box with a small hole in it
Pinhole cameras work in practice
The point to make here is that each point on the image plane sees light from only
one direction, the one that passes through the pinhole.

3. The equation of projection
Cartesian coordinates:
We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f)
Ignore the third coordinate, and get
Prove at home

4. The camera matrix Homogenous coordinates for 3D point are (X,Y,Z,T)
Homogenous coordinates for point in image are (U,V,W)

6. Properties of “thin” lens (i.e., ideal lens)
Light rays passing through the center are not deviated.
Light rays passing through a point far away from the center are deviated more.
focal point f 6

7. Thin lens equation (cont’d)
Combining the equations(do at home): v u f
And the set of all such points forms a plane parallel to the image (plane of focus). 1 u v f + =
image 7

8. Thin lens equation (cont’d)
The thin lens equation implies that only points at distance u from the lens are “in focus” (i.e., focal point lies on image plane).
Other points project to a “blur circle” or “circle of confusion”
in the image (i.e., blurring occurs).
“circle of
confusion” 8

9. Depth of field
f / 5.6
f / 32
Changing the aperture size or focal length affects depth of field

10. Basic models of reflection
Specular: light bounces off at the incident angle
E.g., mirror
Diffuse: light scatters in all directions
E.g., brick, cloth, rough wood
specular reflection
incoming light Θ Θ
diffuse reflection
incoming light

11. Bidirectional Reflectance Distribution Function (BDRF)
Model of local reflection that tells how bright a surface appears when viewed from one direction when light falls on it from another
surface normal

12. The Retina

13. Two types of light-sensitive receptors
Cones
cone-shaped
less sensitive
operate in high light
color vision
Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision
© Stephen E. Palmer, 2002

14. Color Vision

15. The raster image (pixel matrix)
0.92
0.93
0.94
0.97
0.62
0.37
0.85
0.99
0.95
0.89
0.82
0.56
0.31
0.75
0.81
0.91
0.72
0.51
0.55
0.42
0.57
0.41
0.49
0.96
0.88
0.46
0.87
0.90
0.71
0.80
0.79
0.60
0.58
0.50
0.61
0.45
0.33
0.86
0.84
0.74
0.39
0.73
0.67
0.54
0.48
0.69
0.66
0.43
0.77
0.78

16. Color Images: Multi-chip
Bulky and expensive
wavelength
dependent

17. Color Images: Bayer Grid
Estimate RGB at ‘G’ cells from neighboring values
Slide by Steve Seitz

18. HSV Color Space

19. Moravec corner detector
Change of intensity for the shift [u,v]:
Intensity
Window function
Shifted intensity
Four shifts: (u,v) = (1,0), (1,1), (0,1), (-1, 1)
Look for local maxima in min{E}

20. Harris Corner Detector

21. Image filtering 1 90 90
ones, divide by 9

22. Practice with linear filters- 2 1
Original
Sharpening filter
Accentuates differences with local average
Source: D. Lowe

23. Other filters -1 1 -2 2
Sobel
Vertical Edge
(absolute value)

24. Important filter: Gaussian
Spatially-weighted average
5 x 5,  = 1

25. Smoothing with Gaussian filter

26. Smoothing with box filter

27. A sum of sines Our building block:
Add enough of them to get any signal f(x) you want!

28. Fourier analysis in images
Intensity Image
Fourier Image

29. Gaussian

30. Box Filter

31. Aliasing problem
1D example (sinewave):
Source: S. Marschner

32. Aliasing in video

33. Aliasing in graphics

34. Subsampling without pre-filtering
1/2
1/4 (2x zoom)
1/8 (4x zoom)

35. Subsampling with Gaussian pre-filtering

36. Template matching Goal: find in image
Main challenge: What is a good similarity or distance measure between two patches?
Correlation
Zero-mean correlation
Sum Square Difference
Normalized Cross Correlation

37. Normalized Cross Correlation
mean template
mean image patch
Invariant to mean and scale of intensity

38. Matching with filters (Normalized Cross Correlation)
True detections
Input
Thresholded Image
Normalized X-Correlation

39. Reducing salt-and-pepper noise by Gaussian smoothing
3x3
5x5
7x7
What’s wrong with these results?

40. Alternative idea: Median filtering
A median filter operates over a window by selecting the median intensity in the window
Better at salt’n’pepper noise
Not convolution: try a region with 1’s and a 2, and then 1’s and a 3
Is median filtering linear?

41. Salt-and-pepper noise
Median filter
Salt-and-pepper noise
Median filtered

42. Effects of noise Where is the edge?
Consider a single row or column of the image
Plotting intensity as a function of position gives a signal
Where is the edge?
How to fix?

43. Solution: smooth firstg
f * g
To find edges, look for peaks in
Source: S. Seitz

44. Derivative theorem of convolution
Differentiation is convolution, and convolution is associative: : f

45. Derivative of Gaussian filter

46. Final Canny Edges

47. Estimating Camera Parameters
Alper Yilmaz, CAP5415, Fall 2004

48. Ames Room

49. Julesz: had huge impact because it showed that recognition not needed for stereo.

50. Epipolar Constraint

51. Basic Stereo Derivations
Disparity:

52. We can always achieve this geometry with image rectification
Image Reprojection
reproject image planes onto common plane parallel to line between optical centers
(Seitz)

53. Using these constraints we can use matching for stereo
Improvement: match windows
For each pixel in the left image
For each epipolar line
compare with every pixel on same epipolar line in right image
pick pixel with minimum match cost
This will never work, so:

54. Stereo matching as energy minimizationD
W1(i )
W2(i+D(i ))
D(i )
Energy functions of this form can be minimized using graph cuts
Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001

55. Active stereo with structured light
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured Light and Multi-pass Dynamic Programming. 3DPVT 2002

56. Random Dot

57. Time to Collision v Can be directly measured from image L L f t t=0 Do
l(t) f t
t=0 Do
And time to collision:
D(t)
Can be found, without knowing L or Do or v !!

58. 2D Motion Field

59. 2D Optical Flow
Apparent motion of image brightness pattern

60. 2D Motion Field and 2D Optical Flow
Motion field: projection of 3D motion vectors on image plane
Optical flow field: apparent motion of brightness patterns
We equate motion field with optical flow field

61. Brightness Constancy Equation
Taking derivative wrt time:

62. Normal Motion/Aperture Problem

63. Barber Pole Illusion
A: u and v are unknown, 1 equation

64. Full 3D Rotation
Any rotation can be expressed as combination of three rotations about three axes.
Rows (and columns) of R are orthonormal vectors.
R has determinant 1 (not -1).

65. Velocity Model in 2D
Perspective projection