1. Sensory-motor integration Bijan Pesaran May 1 st, 2008

3. Visual response in PMv centered on hand Graziano and Gross (1994)

4. Visual response in PMv centered on hand Graziano and Gross (1994)

5. Duhamel et al (1997) Invariant visual response in VIP

6. Duhamel et al (1997)

7. PRR neurons are eye-centered Batista et al (1999)

8. Updating of response in PRR

9. Mechanisms for encoding multiple variables Centered tuning curve

10. Eye position influences visual response of 7a neurons Andersen et al (1985)

11. Mechanisms for encoding multiple variables Centered tuning curve Multiplicative encoding - gain field Tuning curveGain field

12. Visual receptive fields in VIP partially shift with eye position Avillac et al (2005) Eye positions

13. Mechanisms for encoding multiple variables Centered tuning curve Multiplicative encoding - gain field Partial shift encoding – intermediate reference frame Tuning curveGain field

14. Eye position encoding Widespread phenomenon throughout visual and motor cortices Peaked response Eye left Eye middle Eye right Response Target position

15. Eye position encoding Difficulty addressing flanking responses Peaked response Flanking response Eye left Eye middle Eye right Response Target position

17. Gain model

18. Shifting model

19. Geometry of visually-guided reaching Eye-centered Hand-centered Relative position vectors Nonlinear 3D geometry and muscle dynamics H T E T-E E-H H-E T-H

20. Geometry of visually-guided reaching Nonlinear 3D geometry and muscle dynamics H T E T-E E-H H-E T-H MIP PMd Batista et al (1999) Eye-centered Graziano et al (1997) Hand-centered PMv What is code? T-E T-H Pesaran et al (2006)

21. Decompose the 3-D response matrix

22. Distinguish between different reference frames

23. Define multiple effector reference frames

24. Multiple reference frames Direct visuomotor transformation –Supports Area 5 data (Buneo et al 2002) Direct Transformation Serial Transformation Head-on-body position Eye-in-head position Eye- centered target location Head- centered target location Body- centered target location Hand- centered target location Body-centered hand position + - Motor output Eye- centered target location Hand- centered target location Eye-centered hand location + - Nonlinear 3D geometry and muscle dynamics Motor output Nonlinear 3D geometry and muscle dynamics

25. MIP PMd Nonlinear muscle dynamics Encoding relative position Nonlinear 3D geometry and muscle dynamics Batista et al 1999 Graziano et al 1997 H T E T-E E-H H-E T-H PMv Coordinating the hand and the eye Eye-centered Hand-centered Relative hand- eye position Vectors in abstract space

26. Design reference frame task 0 12 Start Cue Delay Go Hold Time (sec)

27. Design reference frame task 0 12 Start Cue Delay Go Hold Time (sec)

28. Design reference frame task Four initial eye positions

29. Design reference frame task Four initial eye and hand positions

30. Design reference frame task Four initial eye, hand and target positions 111 PMd neurons and 48 MIP neurons

32. Example PMd neuron

34. Example MIP neuron

37. Sensory-motor transformations Head-on-body position Eye-in-head position Eye- centered target location Head- centered target location Body- centered target location Hand- centered target location Body-centered hand position + - Serial Transformation + - + -

38. Sensory-motor transformations Eye-centered hand location Eye-centered target location Hand-centered target location - + Direct Transformation Buneo et al. (2002)

39. Sensory-motor transformations Eye-centered target location Hand-centered target location - + Relative position Transformations Eye-centered target location Hand-centered target location - + Relative location of hand and eye Eye-centered hand location Hand-centered eye location

40. Relative position particularly useful for coordination Relative hand-eye position could be important for hand-eye coordination –Can map hand and eye onto each other –Computationally efficient No need to code for all combinations of locations H T E T-E E-H H-E T-H

41. Relative position particularly useful for coordination Relative hand-eye position could be important for hand-eye coordination –Can map hand and eye onto each other –Computationally efficient No need to code for all combinations of locations H T E T-E E-H H-E T-H

42. Relative position particularly useful for coordination Relative hand-eye position could be important for hand-eye coordination –Can map hand and eye onto each other –Computationally efficient No need to code for all combinations of locations H T E T-E E-H H-E T-H