Tool-based transformations Dr. Tom Froese IIMAS-UNAM

Toward a new cognitive science In cognitive science we are seeing a paradigm shift. Gone is the idea that “mind = digital computer”. Instead, it has been found that mind is essentially: Embodied Embedded Extended Experiential And there is growing evidence that mind is also: Technological

The performance artist Stelarc

Mind-body problem Matter Mind Living being ??? Descartes (1596-1650) Matter Mind Living being ??? Physical body Traditional dualism: the body is a physical mechanism like any other in the world. The relationship between consciousness and matter remains a complete mystery. How to explain mental transformations caused by tool use? Froese (2011)

The mind-body-body problem Matter Mind Living being ??? Physical body Lived body Husserl (1859-1938) Descartes neglected embodiment. The body is not just mechanism, it is our lived flesh. We are our body. Mind is embodied. I experience the world by means of my phenomenally lived body (Leib). Froese (2011)

Life-mind continuity Matter Mind Life Living body Lived body Thompson (2007) Phenomenology: mind is embodied in a living body. The relationship between consciousness and matter can be understood through the phenomenon of life. Froese (2011)

Froese and Di Paolo (2011)

Dynamical approach to cognition The brain is embodied in a biological body. The body is situated in an environment. Behavior is an emergent property of a brain-body-environment system. What happens if we place an interface between the body and its environment? Beer (2000)

Traditional interface design Cognitivist view of tools: Subject and tool are independent The tool is just another object in the world Sensorimotor interaction loops are ignored Symbolic information is given precedence over direct perceptual experience Froese, McGann, Bigge, Spiers and Seth (2012a)

EDSAC Computer, 1949, Cambridge Uni.

Enactive interface design Embodied view of tools: Subject and tool are interdependent The tool is an interface Interfaces mediate sensorimotor interaction Continuous interaction shapes user experience Paradigmatic example: sensory substitution interfaces Tactile-visual substitution system (TVSS) Tools for studying the embodied mind Froese et al. (2012a)

“Artificial embodiment” If mind is rooted in a living body, then this means serious trouble for replicating our minds artificially. Bad news for traditional AI and robotics! (Froese and Ziemke 2009) If mind is embodied in life, then this means profound new opportunities for transforming our minds artificially. Good news for human-computer interaction (HCI)! Clark (2003). Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence If mind is embodied, and using technology transforms our embodiment, then technology can transform our mind! Useful scientific methodology! (Froese et al. 2012b)

Effects of tool-use Psychologists have been busy documenting the various ways in which the use of tools affects the users. Changes in body schema (living embodiment) Changes in perceived space Changes in body image (lived embodiment) Body schema / body image? (Gallagher 1986) Usually, but not always, such tool use based changes happen to various extents at the same time. Effect of modulation, not control.

Changes in body schema There is well-known evidence that use of elongated tools can lead to changes of the neural organization related to the body schema. For example, distant object manipulation with a rake leads to an extension of the neural body schema for the arm (Iriki et al. 1996). Associated visual receptive fields are also extended. Tool-based modulation of sensorimotor loop. This only occurs after intentional usage of the tool, not during passive holding. Maravita and Iriki (2004)

Maravita and Iriki (2004)

Iriki et al. (2001)

Changes in perceptual neglect of space Berti and Frassinetti (2000) tested a patient who showed perceptual neglect in near space during a similar line bisection task. The patient showed no neglect in far space when using a laser pointer to do the task. The patient also showed neglect in far space when using a long stick to bisect the line.

Changes in perceived space When participants are asked to indicate the middle of a line using a laser pointer, there is a systematic rightward deviation as the line is placed further and further away. Longo and Lourenco (2006) found that using an elongated stick in the same task eliminated this bias. Tool use may be extending the range of what is seen as ‘near space’ according to reachability.

Changes in perceived space Witt et al. (2005) manipulated reachability with an elongated tool and asked participants for estimates of a target object’s distance. They showed that tool-users perceived shorter distances to target objects that were beyond normal reach, but within reach with the tool. They also found that manipulating reachability only influenced perceived distance when the perceiver actually intended to reach the object.

Active reaching condition Witt et al. (2005)

Passive holding condition Witt et al. (2005)

Changes in body image Cardinali et al. (2009) found that using a mechanical grabber that extends the arm alters the kinematics of subsequent free-hand grasping (body schema). Furthermore, this effect is related to an increase of the perceived length of the arm (body image)

Tool-use modifies movement kinematics and somatosensory morphology. Cardinali et al. (2009)

Changes are task dependent Farnè, Iriki and Làdavas (2005) showed that these kinds of changes critically depend upon active tool use. They are not found after passive exposure to a hand-tool body configuration. They demonstrated that the elongation extent of near hand space after tool-use is related to the operationally effective length of the tool, and not simply its absolute length.

Changes in body ownership The rubber hand illusion (Botvinick & Cohen 1998)

Changes in body ownership The body swapping illusion (Petkova & Ehrsson 2008)

Body swapping illusion Petkova and Ehrsson (2008)

Substitutional reality system Suzuki, Wakisaka and Fujii (2012)

Suzuki, Wakisaka and Fujii (2012)

Conclusions If the hypothesis of life-mind continuity is correct, then the possibilities of replicating human intelligence in robots and other technological systems is limited – no life, no mind! But our embodied mind presents us with open-ended possibilities for self-transformation – by designing new technological interfaces that shape our existence. The goal should be to extend human capacities with technological interfaces, and not to externalize our existing capacities to make machines more autonomous.

Homework Try out the rubber hand illusion! Finish reading: Froese, T., McGann, M., Bigge, W., Spiers, A., & Seth, A. K. (2012a). The Enactive Torch: A new tool for the science of perception. IEEE Transactions on Haptics, 5(4), 365-375.

References Beer, R. D. (2000). Dynamical approaches to cognitive science. Trends in Cognitive Sciences, 4(3), 91-99. Berti, A., & Frassinetti, F. (2000). When far becomes near: Remapping of space by tool use. Journal of Cognitive Neuroscience, 12(3), 415-420. Botvinick, M., & Cohen, J. (1998). Rubber hands 'feel' touch that eyes see. Nature, 391, 756. Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., & Farnè, A. (2009). Tool-use induces morphological updating of the body schema. Current Biology, 19(12), R478-R479. Clark, A. (2003). Natural-Born Cyborgs: Minds, Technologies, and the Future of Human Intelligence. New York, NY: Oxford University Press. Ehrsson, H. H. (2007). The experimental induction of out-of-body experiences. Science, 317, 1048. Farnè, A., Iriki, A., & Làdavas, E. (2005). Shaping multisensory action-space with tools: Evidence from patients with cross-modal extinction. Neuropsychologia, 43, 238-248 Froese, T. (2011). From second-order cybernetics to enactive cognitive science: Varela's turn from epistemology to phenomenology. Systems Research and Behavioral Science, 28, 631-645. Froese, T., & Di Paolo, E. A. (2011). The enactive approach: Theoretical sketches from cell to society. Pragmatics & Cognition, 19(1), 1-36. Froese, T., McGann, M., Bigge, W., Spiers, A., & Seth, A. K. (2012a). The Enactive Torch: A new tool for the science of perception. IEEE Transactions on Haptics, 5(4), 365-375. Froese, T., Suzuki, K., Ogai, Y., & Ikegami, T. (2012b). Using human-computer interfaces to investigate 'mind- as-it-could-be' from the first-person perspective. Cognitive Computation, 4(3), 365-382. Gallagher, S. (1986). Body image and body schema: A conceptual clarification. The Journal of Mind and Behavior, 7(4), 541-554.

References Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Coding of modified body schema during tool use by macaque postcentral neurones. NeuroReport, 7(4), 2325-2330. Iriki, A., Tanaka, M., Obayashi, S., & Iwamura, Y. (2001). Self-images in the video monitor coded by monkey intraparietal neurons. Neuroscience Research, 40, 163- 173 Longo, M. R., & Lourenco, S. F. (2006). On the nature of near space: Effects of tool use and the transition to far space. Neuropsychologia, 44, 977-981. Petkova, V. I., & Ehrsson, H. H. (2008). If I were you: Perceptual illusion of body swapping. PLoS ONE, 3(12), e3832. doi: 10.1371/journal.pone.0003832 Suzuki, K., Wakisaka, S., & Fujii, N. (2012). Substitutional reality system: A novel experimental platform for experiencing alternative reality. Scientific Reports, 2(459). doi: 10.1038/srep00459 Thompson, E. (2007). Mind in Life: Biology, Phenomenology, and the Sciences of Mind. Cambridge, MA: Harvard University Press. Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends in Cognitive Sciences, 8(2), 79-86. Witt, J. K., Proffitt, D. R., & Epstein, W. (2005). Tool use affects perceived distance, but only when you intend to use it. Journal of Experimental Psychology: Human Perception and Performance, 31(5), 880-888.