1. CS-424 Gregory Dudek Today’s Lecture Computational Vision –Images –Image formation in brief (+reading) –Image processing: filtering Linear filters Non-linear operations Signatures Edges –Image interpretation Edge extraction Grouping –Scene recovery –Recognition Color code: Done Today Next class (or not at all)

2. CS-424 Gregory Dudek Image formation Key processes: 1. Light comes from source 2. Hits objects in scene 3. Is reflected towards A) other objects [return to step 2] B) the camera lens 4. Passes through lens being focussed on imaging surface 5. Interacts with transducers on surface to produce electrical signals In the eye, the lens is that of the eye, the imaging surface is the retina, and the transducers are rods and cones. In a camera, the imaging surface is a CCD (charge-coupled device) made up of transistors (or related devices). [several other types of video camera technology are commonly available, including newer “cheap and cheerful” CMOS cameras, and older “opticons”.]

3. CS-424 Gregory Dudek Scene “Reconstruction” Solve the inverse problem: find the scene that produced the image. Things to account for: –The camera –The geometry of the scene –The interaction of light with objects in the scene. Camera model can be: –Perspective [ loosely: x & y -> fx/z fy/z ] Orthogonal & orthographic are special cases –Paraperspective (AKA weak perspective)

4. CS-424 Gregory Dudek Recovery Typically simplified models of illumination and reflectance are employed. –One light source. –All objects have the same matte reflectance. Sometimes: don’t worry about occlusion. Sometimes: don’t worry about shading. Recovery of geometry is known as scene reconstruction. We will return to this later...

5. CS-424 Gregory Dudek The Sobel Edge operator Filter for horizontal and vertical edges, combine.

6. CS-424 Gregory Dudek Edge extraction Edges: places where the intensity changes: hence there is a large derivative. Each edge has an amplitude and orientation that can be expressed as a combination of orthogonal components in the x and y directions.

7. CS-424 Gregory Dudek Sobel detector specifics Sobel edge detector –Convolve image with a pair of operators, S and S’ S*I and S’*I –The edge map was the Pythagorean sum of the two convolutions E = (S * I) 2 +(S' * I) 2 The Sobel kernel, as an example, can be broken down into two parts: 1. A smoothing operation (discussed last class) reduces effect of noise, sets scale (i.e. size) observe: d/dt F(w) = d/dt k e -iwt = -ikw e -iwt 2. A numerical differentiation df/dt ≈ f(t) - f(t-1)

8. CS-424 Gregory Dudek Other edge operators Older edge detectors neglected the smoothing step : –Larry Roberts developed one of the first (2x2 Robert’s operator[s]). –The Prewitt operator was based on assuming we could fit a smooth surface to the data and then differentiate. –Hueckel operator: use least-squares line fitting More recent work involves smoothing with a two-dimension Gaussian: Gaussian provides localization in space, as well as frequency. Rather than explicitly compute (1D form): we can use the fact that I is constant to improve efficiency: Can be precomputed

9. CS-424 Gregory Dudek Edges: modern methods Instead to looking for peaks in the derivative, look for zero-crossings in the second derivative. A recent operator developed by Canny uses multiple scales (sizes) to improve edge detection, based on optimizing the notion of what an edge is. Wanted: –Detection: detect an edge iff it’s there –Uniqueness: detect each edge just once –Localization: detect it at the right place –Consider an analytic step edge and derive an optimal edge detector. Track edges along contours and suppress (remove) neighbors.

10. CS-424 Gregory Dudek Edges An UNREALISTICALLY simple example: –Edges from the Canny Operator.

11. CS-424 Gregory Dudek Input image Consider: extract the edge elements + grouping them into contours.

12. CS-424 Gregory Dudek Sobel detector output

13. CS-424 Gregory Dudek Canny operator output Note the effect of non-maximum suppression. Note how difficult naive grouping would be.

14. CS-424 Gregory Dudek Human Edge Detection Overheads