1 Computational Vision CSCI 363, Fall 2012 Lecture 31 Heading Models

2 Motion over a Ground Plane

3 Image motion for Translation plus Rotation

4 Models for Computing Observer Motion 1. Error minimization: Minimize the equation 2. Template models: Use a group of "templates" that correspond to the flow fields that would be created by a given set of translation and rotation parameters. Find the template that matches the flow field best. 3.Motion Parallax models: Make use of the fact that image velocities due to translation are dependent on Z. Image velocities due to rotation are not.

5 Image Velocities

6 Neurons vs. Pure Math

7 Motion-subtraction by neurons

8 Computing Heading

9 Layer 2 is a Template

10 Motion toward a 3D cloud

11 Translation + Rotation

12 Operator Responses

13 Model Heading Estimates Model Response

14 Moving Objects Demo

15 Heading Bias (deg)

16 Model vs. Experiment

17 Radial Optic Flow Field Scene Focus of Expansion (FOE)

18 Lateral Flow Field

19 Illusory Center of Expansion (Duffy and Wurtz, 1993) Scene Focus of Expansion (FOE) Perceived Focus of Expansion Demo

20 Difference Vectors for Illusion Center of Difference Vectors

21 Model Response to Illusion Estimated Center Lateral Dot Speed Model Calculated

22 Model vs. Human Response Estimated Center Lateral Dot Speed Model Calculated Average

23 Conclusions 1.A model based on motion subtraction done by neurons in MT can accurately compute observer heading in the presence of rotations. 2.The model shows biases in the presence of moving objects that are similar to the biases shown by humans. 3.The model responds to an illusory stimulus in the same way that people do. 4.The fact that the model responds in the same way as humans with stimuli for which it was not designed provides evidence that the human brain uses a mechanism similar to that of the model to compute heading.

24 How does the brain process heading? It is not known how the brain computes observer heading, but there are numerous models and hypotheses. One of the simplest ideas is based on template models: Neurons in the brain are tuned to patterns of velocity input that would result from certain observer motions. Support for this idea: Tanaka, Saito and others found cells in the dorsal part of the Medial Superior Temporal area (MSTd) that respond well to radial, circular or planar motion patterns. Since then, people have assumed that MSTd is involved in heading computation.

25 Visual Pathway

26 Types of Responses in MSTd (from Duffy & Wurtz, 1991)

27 Combinations of Patterns Duffy and Wurtz (1991) tested cell responses to planar, circular and radial patterns. They found some cells responded only to one type of pattern (e.g. only to circular). Others responded to two or three types of patterns (e.g. both planar and circular). Single ComponentDouble ComponentTriple Component RadialPlano-RadialPlano-Circulo-Radial CircularPlano-Circular Planar They did not suggest a model of how these might be involved in heading detection. They also showed there is not a simple way that MST receptive fields are made from inputs from MT cell receptive fields.

28 Spiral Patterns Graziano et al. (1994) showed that MSTd cells respond to spiral patterns of motion:

29 Does MST compute heading? Prediction: If MST is involved in heading computation, one would expect to find cells tuned to a particular position for the center of expansion. Duffy and Wurtz (1995) tested this prediction.

30 Do MSTd cells use eye- movement information? Psychophysical experiments showed that humans can make use of eye movement information to compute heading. Some MSTd cells have responses that are modulated by eye movements. Do eye movements affect the responses of MSTd cells to compensate for rotation? This was tested in an experiment by Bradley et al (1996). They recorded from MSTd cells while showing flow fields that consisted of an expansion plus a rotation. The rotation was generated by real or simulated eye movements.

31 Real eye movement condition

32 Simulated eye movement condition

33 Results No eye movement Eye movement in preferred direction. Eye movement in anti-preferred direction. This cell seems to take into account eye movements. The effect was not consistent among all cells tested.