Low-Level Vision

Low Level Vision--outline Problem to be solved Example of one computation—lines How simple computations yield more complex information

Problems to be solved Problem 1: Indeterminacies Problem 2: The input to resolve these indeterminacies is impoverished

Indeterminacies Many of the qualities of objects that we would like to know about trade off with other qualities. shape/orientation reflectance/light source/shadow size/distance

Shape/Orientation

Reflectance/Light Source/Shadow This joke turns on the assumption that you will see a shadow, not a difference in reflectance of the object (moon) across its face.

Size/Distance

Problems So problem 1 is that the types of information that we want trade off with one another Problem 2 is that the initial information the visual system has is extremely impoverished

This is the input You end up with the # of objects, their sizes, shapes, distances, textures, motions.

How do you get from one to the other? Researchers divide this question into two parts: Low-level vision: we assume that we can’t get much information out of this array of intensity values. There must be algorithms that summarize this info. High-level vision: taking the output of the low-level processes and transforming it to get objects & their properties.

Simple computation You saw this before.... Can you tell what this is?

Crucial summary--find edges An edge is a sudden discontinuity in intensity

Why edges? Edges frequently correspond to the boundaries of objects; a map of edges is a good start to identifying objects. Edges are invariant to lighting conditions.

How to find edges? Computationally easy to find discontinuities Compare means of adjacent columns, rows, diagonals

What about textures? Why don’t you see a million objects when you see a hat with many “edges” (Herringbone pattern)?

Assess at more than one scale Assess neighboring columns: yields five edges

Assess every three columns (i.e., take the mean) yields one edge Assess at more than one scale

Biological evidence

Retina

Ganglion cells: center-surround

On-off can combine to form line detectors

Or an edge detector

Hubel & Wiesel’s experiments

Biological evidence It does seem that some of the cells relatively early in the visual processing stream care about edges.

All of this was about lines. Now how do you get distance, shapes, etc.

Shape/orientation indeterminacy Perkin’s laws--conjunctions of lines assumed to correspond to different 3D shapes.

Perkins’ Laws Possibly built into early visual processing. Pop-out with perkins’ laws type angles, but not with other angles.

That’s it for lines Focus on other assumptions

Light source/reflectance/shadow What’s this?

Assumption 1: surfaces are uniformly colored. (That’s why shading gives the impression of 3 dimensions. Shading is assumed to be due to hills & valleys.

Light/reflectance/shadow Shadow: light is assumed to be coming from above.

Reflectance/Light Source/Shadow

How is constancy figured out? Obviously, absolute constancy is not calculated

Local contrast Assumption 3: the brightest thing around is white; the darkest thing around is black.

Distance/size Isn’t it the case that we frequently just know the size an object should be? This is familiar size and it’s actually not that powerful a cue.

Familiar size When you remove the cue of height in the picture plane the person looks tiny.

Cues to distance Convergence--not very effective More effective are a range of cues that can be evaluated in a picture plane, and so are often called pictorial cues.

Occlusion

Texture Gradient

Linear perspective

Height in picture plane

Atmospheric Perspective

Stereopsis Very important cue. This is NOT a pictorial cue. Based on the fact that the two eyes get slightly different views of the world.

Stereopsis Difference between what the left eye and right eye sees is called retinal disparity. Farther object, less difference

Stereopsis Problem: how do you match up the views of the two retinas if objects are similar? This is called the correspondence problem. (Consider that highlights differ because of different reflections and there are geometric distortions due to seeing things from a different angle.)

Stereopsis Solutions to the correspondence problem 1. Uniqueness constraint: an object in the left eye can be matched to only one item in the right eye. 2. Epipolar line constraint: because the eyes don’t move independently in vertical dimension, there is a limited number of places that an object on the left retina can be on the right retina.

Where will the boat be in the right retina? It can’t be just anywhere. It must be somewhere on the horizon line

More generally....

Summary Problems –Indeterminacies –Impoverished Input Lines –Computations –Biology Solutions –Shape/orientation (Perkins’s laws) –reflectance/light source/shadow (uniform color, local contrast) –Size/distance (familiar size, convergence, occlusion, texture gradient, linear perspective, relative height, atmospheric perspective, stereopsis).