1 Computational Vision CSCI 363, Fall 2012 Lecture 24 Computing Motion

2 Measuring Motion from 2 points Recall intersection of constraints: v p1 v p2 v p1 vxvx vyvy V p1p1 p2p2 We can solve for V x and V y :

3 The Problem of Noise Problem: Measurements are often inaccurate because: 1)Noise in the image measurements 2)Inaccuracies due to the discrete (pixelated) image. Solution: Make multiple measurements and find V that best matches all the measurements. The effects of noise will tend to average out.

4 Finding the best fit Because of noise, there will be no single V that satisfies all the local measurements. We want to find a V that fits the measurements best, i.e. it gives the least error when compared with predictions from measured quantities. Predicted perpendicular velocity Measured perpendicular velocity

5 Add up the errors For a given V = (V x, V y ), we can find the total error by summing all the error terms for all the measured perpendicular velocities. Goal: Find V that gives the minimum error, E.

6 Finding the Minimum To find the minimum, we find the point where the derivative is zero. Rearranging, we have:

7 The Solution The previous equation has the form: where Solving, we obtain:

8 Recall: Motion Energy Motion energy filters are constructed with 2 gabor filters, one of which uses a sine and the other uses a cosine (a "quadrature pair"). If you square the outputs of the gabors and sum, the result is motion energy.

9 Reverse Phi If a pattern of white and black lines is moved rightward in small steps, people see rightward motion. If the contrast is reversed with each step (white becomes black and vice versa), people see leftward motion. (The Reverse Phi Effect) Demo: Energy White energy => rightward motion. Dark energy => leftward motion. Move pattern in steps. Reverse Phi: Move pattern in steps while reversing contrast

10 Fluted Square Wave When the fluted square wave is shifted to the right in 90 deg steps, it appears to move left! A square wave that is shifted to the right in 90 deg steps, appears to move right. 90 deg step to right A square wave with the fundamental frequency component removed is a fluted square wave. The highest amplitude component is 3f.

11 Fluted Square Wave For a fluted square wave, the highest amplitude component is 3f. When the square wave (frequency f) moves 90 deg to the right, the 3f component is being shifted 270 deg to the right, which appears as 90 deg to the left. Why? When a square wave that is shifted to the right in 90 deg steps, its fundamental frequency moves right in 90 deg steps.

12 Energy Response to a fluted square wave x t Energy Square wave Fluted Square wave White = Right Black = Left

13 Moving Plaid Demo Demo of a moving plaid grating: + = Demo: (Search on "plaid", choose coherence.mp4)

14 Motion Energy for 2D images For a 2D image, we use a 3-D gabor filter: Selects frequency range within an ovoid in spatio-temporal frequency space: sfx sfy tf

15 Velocity lies on a Plane For a 1D image, all measurements of the same velocity lie along a line in SF-TF space (because v = TF/SF) For a 2D image, all measurements of the same velocity lie on a plane in SF-TF space. sf tf sfx sfy tf Find the plane by making multiple measurements and finding best fit.

16 Extra-striate visual areas Folded Cortex Flattened Cortex

17 Dorsal and Ventral Streams

18 Evidence for two processing streams Evidence for separate streams of processing comes from three areas: 1)Lesion studies. Lesions in the ventral areas cause selective deficits in color and orientation discrimination abilities. They can also cause deficits in object or face recognition. Lesions in the dorsal areas cause selective deficits in judgments of motion (e.g. speed or direction). Can also cause deficits in localization of objects. 2) Psychophysics: Hard to see motion at "isoluminance". 3)Connection patterns: Parvocellular->4C  ->Superficial cortical layers (color and form) Magnocellular->4C  ->4B-> MT

19 Motion Processing in V1 In V1, some simple cells and complex cells are tuned to direction of motion. I.e. they respond most strongly to motion in a given direction and their response falls off as the motion deviates from that direction. 180 o 120 o 240 o Firing Rate Direction of Motion Direction Tuning Polar Plot (tuning for zero deg) Tuning for 180 deg

20 V1 neurons tuned to temporal frequency V1 neurons appear to be tuned to temporal frequency. Their preferred speed depends on the spatial frequency of the pattern. Firing Rate Temporal Frequency v =  t /  x Neurons in V1 behave like motion energy filters.