1. Lecture 2 Imaging Geometry and OpenCV
Slides by:
David A. Forsyth
Clark F. Olson
Jean Ponce
Linda G. Shapiro

2. Pinhole Cameras Abstract camera model - box with a small hole in it
Pinhole cameras work in practice

3. Distant objects are smaller

4. Parallel lines meet Common to draw film plane
in front of the focal point.
Moving the film plane merely
scales the image.

5. The equation of projection
Cartesian coordinates:
We have, by similar triangles, that:
(X, Y, Z) ~ (f X/Z, f Y/Z, f)
f is called the focal length.
[X, Y, Z]
[fX/Z, fY/Z]

6. The reason for lenses
We won’t worry much about lenses in this class.

7. Lens distortion
“Barrel distortion” of rectangular grid is common for inexpensive lenses.
Precision lenses can be expensive.
Zoom lenses often show severe distortion.
Fish-eye lenses also have severe distortion.

8. Image capture Images are not continuous
Typically captured with a CCD camera (charge-coupled-device)
The amount of light striking each location on a grid is integrated over some time period
Rows are read out one at a time
For color images, successive
pixels usually correspond to
different colors
High quality color cameras use
a beam splitter and 3 separate
CCD chips
APS (active pixel sensor) is a
cheaper (lower quality) technology

9. Resolution
Resolution often (but not always) refers to the number of pixels in the image.
Lower resolution has fewer pixels.
Interestingly, faces of people you know can usually be recognized at 64x64 (or less) pixels.
Squint and look at the lowest resolution image.

10. Some of my research
Time permitting, the following slides give a very brief overview of some of my previous and current research.

11. Visual Terrain Mapping for Mars Exploration

12. Pose Sampling for Efficient Model-Based Recognition
Careful sampling of the viewpoints reduces
the complexity from cubic to linear in the
number of craters.

13. Robust Registration of Aerial Image Sequences
Registration results
Goal: Provide registration between images and maps for persistent aerial surveillance

14. Simple and Efficient Projective Clustering
Projective clustering problems have the following properties:
Many dimensions - d
Many points – n
Clusters do not form in the full d-dimensional space
No labeled exemplars

15. Keypoint recognition
A popular object recognition technique uses descriptive “keypoints” that have been extracted from images.
We are investigating getting better use out of the color information in the image when creating keypoint descriptors.

16. Image classification
Current techniques for image classification cluster image keypoints into “visual words” similar to text retrieval methods. We are studying the use of projective clustering to improve performance.

17. Programming in OpenCV
In OpenCV, images are represented as matrices (as in linear algebra).
Mat image = imread("photo.jpg"); // Most generic declaration
The image could have a number of underlying data types for each pixel:
uchar – unsigned byte (greyscale image)
Vec3b – vector of 3 bytes (color image)
Point2f – point in two dimensions, float
many others…

18. Creating images Images can be created using a number of methods:
using namespace cv; // all my code assumes this
Mat image; // creates 0x0 image
Mat image = … // uses copy constructor
Mat image(rows, cols, type); // type is CV_8U, for example
Mat image(rows, cols, type, scalarValue);
Example: Mat allBlue(360, 480, CV_8UC3, Scalar(255, 0, 0));
Mat_<Vec3b> colorImage = imread(“color.jpg”);
// Can be convenient, but now limited to Vec3b images (matrices)
// Also, must declare as a similar parameter type when passed

19. Copying images
Be careful to remember that most image copy and pass by value methods do NOT perform a deep copy. image2 = image1; // shallow copy void someMethod(Mat imageParam); // shallow copy If you want a deep copy, then use clone (or copyTo): image2 = image1.clone(); // deep copy image1.copyTo(image2); // deep copy

20. Memory management Memory management is handled by the Mat class.
This is different from the IplImage class in OpenCV 1
This works correctly even if multiple images share the same data
A reference count is kept for each image
The data is deallocated only when the reference count goes to zero
However, this can allow privacy leaks unless you are careful

21. Backwards compatibility
OpenCV 2 is backwards compatible with OpenCV 1. IplImage *iplIm = cvLoadImage("photo.jpg"); // Do some work with image here cvReleaseImage(&iplIm); // necessary to prevent memory leak Can convert to Mat simply: Mat converted(iplIm); // do not release image until finished

22. Image manipulation
OpenCV provides many methods to manipulate entire images:
Filtering: blur, smooth, median, gradient, laplacian
Transformations: resize, affine, perspective, color space, threshold, flood fill
Segmentation: grabCut, watershed
Feature detection: edges, corners, lines, circles, template matching, SIFT