1. Hand trajectories reveal hidden cognitive states Ken Nakayama a Joo-Hyun Song b Matthew Finkbeiner c Alfonso Caramazza a a Harvard University b Smith-Kettlewell Eye Research Institute c Macquarie University

2. today’s talk Suggest there are new opportunities to study a wide range of internal cognitive states from the detailed study of motor behavior (Spivey et al., for language processing)

3. Remarkable power of psychophysical and behavioral methods Some examples Wertheimer, 1912 Hecht, Shlaer and Pirenne, 1943 Julesz 1960s Sperling 1960s

4. Early processing Later processing Cognition/decision Motor plan input

5. Early processing Later processing Cognition/decision Motor plan input fMRI EEG/MEG TMS

6. Early processing Later processing Cognition/decision Motor plan input Reaching Measure RT And accuracy PLUS Full details of trajectory Reveals otherwise hidden states

7. . Hand movement recording: 120Hz with Polhemus Fastrak Polhemus system: Sample rate (120 Hz) Recording 3-D movement of finger tip Visually-guided pointing Song and Nakayama (VSS 05,06)

8. I. Early hand trajectories reflect an underlying number representation –Song and Nakayama, Cognition (in press) II. pointing trajectories reveal influence of unseen words –Finkbeiner, Song, Nakayama, and Caramazza, Visual Cognition (in press) Two recent studies

9. Mental number line

10. IF X>5 respond “greater” Else respond “lesser” if digital number line BUT, RT and error rates systematically decrease as the numerical distance between two numbers increases. Moyer and Landauer (1967) Supporting “ Analog ” number line X In numeric comparison task3 vs. 5, 13 vs. 5 Same response to 3 vs. 5 &13 vs. 5 Numeric distance effect

11. Characteristics of number representations are mainly examined with discrete responses such as reaction time and accuracy. In the current study, to map invisible internal cognitive processes of numeric comparison in spatial domain over time, we measured trajectories.

13. X-Y trajectories Systematic shift of initial trajectories towards a hypothetical position on a number line intermediate between the numeral 1 and 5 positions

14. Mean trajectory deviation ≈ 480 ms after target onset ≈ 150 ms after the movement onset

15. Endpoint deviation

16. Continuous internal processes for number representations Early detection of mental numeric line than previous tasks Kinematic measurements: More reliable and robust with small numbers of subjects

17. I. Early hand trajectories reflect an underlying number representation –Song and Nakayama, Cognition (in press) II. Pointing trajectories reveal influence of unseen words –Finkbeiner, Song, Nakayama, and Caramazza, Visual Cognition (in press)

18. linkage between invisible text and actions? Goodale and Milner Dorsal Vs Ventral

19. spinach Point to color of the word ######## boy KMDKS spinach 500 ms 30ms 10 ms Until Response ######## red KMDKS spinach 500 30 10 2000 ######## green KMDKS spinach 500 30 10 2000 neutral incongruent congruent

20. ######## boy KMDKS spinach 500 ms 30ms 10 ms Until Response ######## red KMDKS spinach 500 30 10 2000 ######## green KMDKS spinach 500 30 10 2000 neutral incongruent congruent

21. 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 100 msec after movement onset Time after movement initiation Horizontal movement (in) incongruent congruent neutral

22. Replication with varying prime duration

23. Despite the unavailability of the invisible stimulus to visual awareness, the congruity of the prime stimulus with the target stimulus had a significant effect on participants’ pointing trajectories. Masked unseen word processing extends down to include the formulation of overt motor responses

24. Early processing Later processing Cognition/decision Motor plan input Reaching Full details of trajectory LEAKAGE

25. Suggest there are new opportunities to study a wide range of internal cognitive states from the detailed study of motor behavior