1 Perception and VR MONT 104S, Fall 2008 Lecture 6 Seeing Motion

2 Tips for Scientific Writing 1.Stay focused: Writing should address question. Good: People in the distance appeared to be small. Their size helped me to judge their distance. Not-so-good: The scenery was spectacular. The lakes were small so I could not see any details. 2.Provide specific examples as evidence. 3.Write it all out: Good: When walking downhill, things that were more distant were lower in the visual field. This is a situation in which height in the visual field would not be a useful cue to distance. Not-so-good: I misjudged the size of the rocks. 4.Answer each part of the question: When size works, when size doesn't work, when height works, when height doesn't work. 5.Be as clear as you can be. Simple and clear is better than flowery and obtuse.

3 Causes of Image Motion Image motion can result from numerous causes: A moving object in the scene Eye movements Motion of the observer

4 Uses of Image Motion Image motion on the retina can be used to compute a variety of scene properties. Among them are: Image segmentation (dividing up the scene into individual objects or surfaces) 3D structure of an object (structure from motion) Depth (motion parallax) Time to collision Heading direction Moving object direction Speed of eye movements (for smooth pursuit)

5 Two stages of Motion processing Visual motion processing is thought to occur in two stages: 1)Extract the 2D image velocity field. 2)Use the 2D velocity field to compute properties of the scene (as listed in the previous slide).

6 Motion Detection by Neurons Problem: A single photoreceptor (or retinal ganglion cell) cannot detect motion unambiguously. A spot of light moving across its receptive field will cause a temporary increase in light followed by a decrease. The photoreceptor cannot distinguish between motion or changes in ambient lighting.

7 A Neural Circuit for Motion Barlow and Levick proposed a model to compare the response at one location with a delayed response at a neighboring position. Prefers right motion 1.Each neuron has a receptive field that is shifted to the left with respect to the receptive field of the previous neuron. 2.Each neuron causes the neuron with a RF to its left to be inhibited a short time after the first neuron is stimulated. 3.The neurons in the circuit will not respond to leftward motion because of this inhibition. 4.They will still respond to rightward motion.

8 Mapping of Visual Areas This map shows a flattened cortex with the known visual areas mapped onto it. There are a large number of distinct visual areas (probably at least 20). Each area appears to have a specific function. The areas show a roughly hierarchical organization (although most areas have reciprocal connections).

9 Two Major Processing Streams There appear to be 2 major processing streams (although there are cross connections between them): 1.The Dorsal Stream: Includes areas MT, MST, VIP, 7a, etc. Processes motion, stereo, spatial relationships The "where" pathway. 2.The Ventral Stream: Includes areas V4, IT, etc. Processes color, form, objects. The "what" pathway.

10 Dorsal and Ventral Streams

11 Motion Processing in V1 In V1, some simple cells and complex cells are tuned to direction of motion. I.e. they respond most strongly to motion in a given direction and their response falls off as the motion deviates from that direction. 180 o 120 o 240 o Firing Rate Direction of Motion Direction Tuning Polar Plot (tuning for zero deg) Tuning for 180 deg

12 Motion Processing in MT MT (The Middle Temporal Area) is thought to be important for processing motion information. Characteristics of MT neurons: 1)Cells tuned for direction of motion (more broadly tuned than V1 cells. 2)Cells tuned for speed. 3)Large receptive field sizes. (Some are 100x bigger than V1 receptive fields). They range from 1-2deg in diameter in the foveal region and increase in the periphery.

13 Speed Selectivity McKee and Nakayama have shown that people are very good at discriminating two different speeds. The Weber fraction gives a measure of how big a change in speed is necessary to distinguish two different speeds. It is fairly constant over a broad range of speeds:  V/V =.05 MT may be the area that first computes speed.

14 The Aperture Problem If our view is limited to an edge seen through an aperture, we can only find the component of motion perpendicular to the edge. The aperture problem is a fundamental problem when one is trying to measure image velocity using local detectors. This is true in biological vision (neurons have local receptive fields). In the "barber pole" illusion, the edges of the aperture affect perception. Aperture Edge Perpendicular velocity component

15 Motion Adaptation MT cells do not respond if opposite directions of motion (e.g. left and right) presented at the same time. This is called motion opponency. The detectors for opposite directions balance each other. The waterfall effect: If you view one direction of motion for a long time, the detectors for that direction become fatigued. If you then look at a stationary surface, it will appear to move in the opposite direction. This works for expansion and contraction as well. Demo:

16 2D Motion is just the Beginning 2D image motion contains information about: Relative depth of surfaces 3D motion of objects 3D structure of objects Direction of observer motion Among other things. Many of these tasks require local comparisons of neighboring motions.

17 Motion Parallax For an observer moving in a straight line, the images of objects that are nearby move faster than the images of objects that are far away. This is the result of perspective projection. The difference in image speed, known as motion parallax, is a strong cue for the relative depth of objects. A moving observer can find changes in depth in the scene by finding changes in image speed. object 1 object 2

18 MT cells have inhibitory surround + -Many MT cells have an inhibitory surround. Motion in the surround inhibits the response to motion in the center. The inhibitory surround may be involved in: Figure-ground segmentation based on motion. Motion parallax Heading judgments (judging where one is going).

19 Structure From Motion Relative motion can give information about the 3D structure of objects. Structure from motion originally studied rigorously by Wallach and O'Connell (1953). They studied wire-frame objects and examined peoples ability to judge the structure of the objects when moving. The ability to see a 3D structure from a moving 2D image is known as the Kinetic Depth Effect. Demo:

20 Detecting Biological Motion Johansson attached points of light to the joints of people and filmed them walking. He showed that when the lights are in motion, people can recognize the actions being performed. Demo: Relative motion of the points of light is certainly important for recognizing the actions.

21 Using Motion to See Where You're Going When one moves in a straight line, the images on the retina move in a radial pattern. The center of the pattern coincides with the direction you're going. When we move on a curved path, the computation is more difficult (and involves relative motion), but we can still judge our path of motion.