1. Cursus Doelgericht Handelen (BPSN33) R.H. Cuijpers, J.B.J. Smeets and E. Brenner (2004). On the relation between object shape and grasping kinematics. J Neurophysiol, 91: 2598- 2606. R.H. Cuijpers, E. Brenner and J.B.J. Smeets (2006). Grasping reveals visual misjudgements of shape. Exp Brain Res 175:32-44

2. Topics 1 st hour: Control Variables in Grasping –Opposing views on visuomotor control –Research question 2 nd hour: Grasping elliptical cylinders –Real cylinders Which positions? How to get there? –Virtual cylinders Constant haptic feedback Veridical haptic feedback –If time permits: Modeling grip planning –Conclusions

3. Control variables in grasping Many levels of description: Activity motor neurons Muscle activity (EMG) Posture (Joint angles) Kinetics (Forces, torques) Kinematics (Position, speed etc.) Task level Degrees of Freedom (DoF) low high

4. Control variables in grasping How does the brain ‘plan’/compute the desired motor neuron output? If movements are planned in task space: –little computational power needed for planning stage But … –Need to solve DoF-problem (Motor primitives) –Cannot control everything (Stereotypic movements) –Need low-level on-line control (e.g. stiffness control)

5. Control variables in grasping What is/are the correct level(s) of description for movement planning and visuomotor control? Method of research in visuomotor control: Manipulate visual information / haptic feedback / proprioceptive feedback Measure effect on motor output Variables that have an effect are ‘controlled’ Variables that have no effect are redundant Haptic = by touch Proprioceptor = sensory receptor in muscles, tendons or joints

6. Opposing views on visuomotor control Fingertip positions and object size Milner & Goodale: perception vs. action Franz et al: common source model Smeets & Brenner: position vs. size Fingertip positions and object orientation Glover & Dixon: planning vs. on-line control Smeets & Brenner: position vs. orientation

7. perception vs. action Goodale (1993); Milner, Goodale (1993) RV: lesions in occipito-parietal cortex (dorsal). DF: damage in ventrolateral occipital areas due to CO poisoning.

8. perception vs. action Dorsal pathway for guiding movements (should be veridical) Ventral pathway for perception (perception of shape, colour etc.)

9. perception vs. action Agliotti, De Souza, Goodale (1995): Grip aperture NOT influenced by size- illusion. Due to separate processing of information for perception and action.

10. Common source model Franz et al (2000): equal effects of illusion

11. Position vs. size Brenner, Smeets (1996): Size-illusion does not affect grip aperture, but does affect the initial lifting force. Explanation: not size information is used but position information. They are inconsistent.

12. Planning vs. on-line control Glover & Dixon (2001) Relative effect of illusion decreases with time  Illusion mainly affects planning

13. Position vs. orientation Smeets et al. (2002) Assumption: illusion affects orientation, not position Also explains data of Glover and Dixon

14. Research Question: How is shape information used for grasping? The visually perceived shape is deformed Shape (ventral) determines where it is best to grasp an object (dorsal) –Grip locations not veridical Shape information could be used during planning (ventral) or on-line control (dorsal) –Grip errors arise early or late in the movement

15. Grasping elliptical cylinders: real cylinders

16. Experimental design seven 10cm tall cylinders elliptical circumference with fixed 5cm axis variable axis: 2, 3, 4, 5, 6, 7 and 8 cm

17. Experimental Design

18. Experimental design Optotrak recorded traces of fingertips 2 distances x 7 shapes x 6 orientations = 84 trials 3 repetitions 10 subjects

19. Experimental Design

20. Example

21. Which positions? Geometry: grasping is stable at principle axes

22. Which positions? Principle axes preferred. But systematic errors…

23. Which positions? Systematic "errors" depending on orientation.

24. Scaling grip orientation  0.7 except for aspect ratios close to 1,  0.5 Which positions? Scaling grip orientation = slope + 1

25. Comfortable grip Prediction: Slope a = w-1 Offset b = -(w-1)  0 Suppose: grip orientation = mixture between cylinder orientation + comfortable grip

26. Thus … Subjects grasp principle axes, but make systematic errors Cannot be explained by comfort of posture Additional effect of deformation of perceived shape

27. How to get there?

29. High correlation despite errors! Sudden drop at end: Grip aperture automatically corrected Correlation much higher for max. grip aperture than final grip aperture Gradual increase: grip errors were planned that way

30. Thus … Systematic errors already present in the planning of the movement Maximum Grip Aperture reflects planned size rather than true size

31. Grasping virtual cylinders

32. Experimental design

33. Experimental Design

34. Experimental design

35. Constant haptic feedback: –Real cylinder is always circular –Virtual cylinders: 15 aspect ratios, 3 orientations Veridical haptic feedback: –Virtual and real cylinders are the same, 7 aspect ratios and 2 orientations

36. Constant haptic feedback Only half of the subjects scale their grip orientation If they do, the scaling of grip orientation is similar to real objects (0.42)

37. Constant haptic feedback Subjects hardly scale their max. grip aperture Scaling of max. grip aperture is much smaller than for real objects (0.14 instead of 0.57)

38. Thus Inconsistent haptic feedback reduces scaling gains Possible cause: All subjects scale their grip aperture based on the felt size Scaling of grip orientation based on seen orientation for only half of the subjects, and the felt orientation for the other half

39. Veridical haptic feedback Similar pattern of grip orientations for all subjects Scaling of grip orientation (0.58) close to those for real objects (0.60)

40. Veridical haptic feedback All subjects adjust their maximum grip aperture Scaling of max. grip aperture (0.39) much higher and closer to real objects (0.57)

41. Thus With consistent haptic feedback Scalings of grip orientation and grip aperture close to those for real cylinders Less variability between subjects

42. Comparison of experiments Real Cylinders Consistent Feedback Inconsistent Feedback

43. Thus Natural grasping of virtual cylinders requires veridical haptic feedback Grip orientation and grip aperture can be scaled independently

44. Modeling grip planning

45. Physical constraints –Grip force through centre of mass –Grip force perpendicular to surface –Optimal grip along major or minor axis Biomechanical constraints –For a given cylinder location there is a most comfortable grip –Evident when grasping circular cylinder

46. Modeling grip planning Assumptions: –The planned grip orientation is a weighted average of the optimal and the comfortable grip orientation –The weights follow from the expected cost functions for comfort and mechanical stability

47. Modeling grip planning If Then(required)

48. Modeling grip planning Perceptual errors change the perceived cylinder orientation The comfortable posture may also be uncertain

49. Modeling grip planning If distributions are Gaussian with zero mean, we get: For the circular cylinder w=0, so that:

50. Modeling grip planning Each grip axis may be grasped in different modes: Model predicts probability of each mode

51. Modeling grip planning The model describes the relative costs for grip comfort and mechanical stability It predicts the relative probability of choosing the major or minor axis We can incorporate biases in the perceived cylinder orientation We can extend to more general shapes

52. Conclusions Subjects plan their grasps to suboptimal locations based on the perceived shape and the anticipated (dis)comfort Upon touching the surface the errors are corrected Haptic feedback is necessary for natural grasping With our model we can identify relative contributions of comfort, stability and perceptual errors

53. Conclusions Visual shape information (slant, curvature) is used for planning suitable grip locations (position information) –Perceptual bias –Bias due to comfort of posture No substantial on-line corrections  On-line control uses position information When inconsistent, haptic and visual shape information is combined differently for the planning of grip aperture and grip orientation

54. The end

55. Stable grip of an ellipse