dap Boxed Ego devices that alter perception Artificial autoscopy for fun and work Alvaro CASSINELLI and Masatoshi ISHIKAWA Meta-Perception Group

Autoscopy, hyper-stereo (miniaturized “avatar”) & time delay Boxed ego Computers have made possible to reproduce and reinforce these experimentations done in the 70’s Autoscopy, hyper-stereo (miniaturized “avatar”) & time delay

I. Altered self-perception II. The sense of self: a modality of perception? III. Manipulating the sense of Self IV. Conclusion

I. Altered self-perception (body scheme, and “ego-location”) Pathological conditions (Alice in Wonderland syndrome, out-of-body experience, autoscopy, heautoscopy…) Experiments in cognitive neurosciences (inverted goggles, rubber hand, TMS, etc) Medially mediated perception (artistic experimentation with mirrors, video, art installations) Human-computer interfaces (augmented perception, sensory substitution systems, teleprescence) Will concentrate on body-self perception, but this is necessarily correlated to the sense of agency and being someone, at least if we embrace the sensory-motor theory of perception. At the cognitive level, a fundamental distinction can be made between two different higher-order body representations which have been called body schema and body image (Paillard, 1999). Body schema refers to a representation of the positions of body parts in space, which is updated during body movement. This typically does not enter into awareness, and is primarily used for spatial organization of action. The body schema is therefore a central representation of the body’s spatial properties, that includes the length of limb segments, their hierarchical arrangement, the configuration of the segments in space and the shape of the body surface. Body image refers to a conscious visual representation of the way the body appears from the outside, typically in a canonical position. The scientific concept corresponds roughly to the everyday use of the term. This chapter is not primarily concerned with body image, since there is little evidence of any special connection between disorders of body image (e.g., in anorexia) and movement control. We will not discuss pure disorders of body image, but we will discuss the many cases where abnormalities of body schema lead to altered body image. The body scheme represents the position and configuration of the body as a volumetric object in space. Crucially, the body scheme integrates tactile information from the body surface with proprioceptive information about the configuration of the limbs in space (Head & Holmes, 1911). This integration means that a stimulus on the body can be localized in external space. For example, combining a tactile sensation on the left hand, with information about the joint angles of my left arm allows me to program a rapid movement of my right arm to swat the fly. Thus, tactile sensations are obligatorily transformed from body surface locations to locations in external space, suggesting that body scheme representations dominate primary representations in normal human behaviour (Yamamoto & Kitazawa, 2001). For example, a visual stimulus on the right side facilitates processing of a subsequent tactile stimulus on whichever hand is adjacent to the visual event. In a normal posture, the right hand shows this facilitation, but if the hands are crossed then the left hand shows a comparable benefit (Spence, Pavani, & Driver, 2000). Ictal Body Scheme Disturbance Induced by Looking Through a Small Opening Itsuo Kawai* Shin Fujii† *Department of Psychiatry, Faculty of Medicine, Kyoto University, Shogoin, Sakyoku, Kyoto 606, Japan †Department of Internal Medicine, Shin-Osaka Hospital, Osaka, Japan Copyright 1979 International League Against Epilepsy KEYWORDS Looking through a small opening • Ictal body scheme disturbance • Right parietal focus Summary: The case of a patient is reported in which seizures consisting of a feeling that the left arm was absent were induced by looking through a small opening (e.g. camera view finder). The EEG showed right parietal sharp wave discharges between and during such seizures. This observation is interpreted as an example of a reflex epilepsy triggered by the specific stimulus of looking through a small opening, and involving the right parietal lobe. Sensed presence and spatial distortions of body size can be induced by TMS

Out of body experience: from phantom limb to phantom body? Phantom limbs can occur in non-amputated people OBEs have a distinctive experiential character a specific brain area involved (temporo-parietal junction)

Artificial OBE in a controlled environment: cognitive neurosciences … this is research on cognitive neurosciences! As a passing remark, note the similarity with video installations of the 70’s… O. Blanke research at the Laboratory of Cognitive Neuroscience, EPFL , Switzerland, (2007)

Artificially relocated body parts and telexistence Haptic telexistence (Tachi Lab) Uses: robotic teleoperation in dangerous places, tele-surgery, etc. "Telesar" (Tachi Lab) Eliciting a true sense of tele-existence may be key to performance here

Explorations of the self in the media arts: the magic of the mirror…

…beyond the mirror Dan Graham’s «Present Continuous Past(s)», (1974) Dan Graham: TIME is represented in SPACE Brunce Nauman: PHYSICAL motion necessary to the PERCEPTION OF TIME. The closer you get, the smaller you are. Heidegger: http://en.wikipedia.org/wiki/Being_and_Time Bruce Nauman, «Live-Taped Video Corridor» (1970) ...point to a sensory-motor theory of the perception of “being” in time and space!

More SMC dislocations… Avatar Machine, by Marc Owen (2008) "... a system which replicates the aesthetics and visuals of third person gaming” Joachim Sauter “deviewer” (1991)

II. But can the sense of self can be considered a modality of perception? (in the sense of the sensory-motor theory of perception) Perception of what? The body scheme, the body image Phenomenology: there is something that it is to “be there” But which sensory-motor knowledge is involved? Hypothesis: Probably (enactive) knowledge in all “classic” modalities of perception (visual, tactile, propioceptive and auditory), but not enacted the perceived object, but rather to the fact that there is a perceiver! Phenomenologically speaking, the felt self-presence (the “qualia” of being) is not an inference, but rather the knowledge you have that certain movements will affect sensorial input in a way compatible with the fact that you have a human body. Noa: Feeling of presence of (say, of a bottle) is not a conjecture or inference: the feeling you have is the knowledge that movements of the hand open up and reveal new aspect of bottle surface, as dictated by the SMC. Broadened concept of “sensory-attentional contingencies” (Nusbaum et al. 2001) This SMC-based hypothesis can accounts for dislocated self-perception a phenomena. Good grounding for practical design of devices that alter perception of the self.

An example practical application: autoscopic display Enhancement of training mirrors… Augmented awareness on public places (personal orientation and elimination of blind spots) Supermirrors: medical applications REM: the mirror may not be so innocent!!! Try to look at yourself in a mirror at 5 AM… Nissan system The problem of responsibility in “disembodied” display interfaces: marc owen avatar

Conclusion “Sense of self” may result from an innate/learned capacity to enact sensory-motor knowledge and to recognize on these patters special egocentric features. Therefore it can be altered by artificially manipulating: - the structure of sensory-motor contingencies for each modality of perception and - the inter-modality contingencies (time correlations and delays!) Sense of Self may suffer from “perceptual illusions” such as “filling-in” body schema, attentional “self-less”… Devices that alter the perception of the self may enable a lot of experiments and have a lot of practical applications Experiments: what about a multi-located sense of self? (space odysey computer) Altered perception: TV, phone… we don’t need to go as far as telexitence systems! One consequence of requiring sensorimotor knowledge for successful perception is that disruptions to the normal way that movement affects sensory stimulation can result in failing to perceive. “Altered” is a matter of degree: the self has changed enormously with technology, and will continue to change (tranhumanism?)

Questions?

Sensed observed object and sensed observer object: two sides of the same perceptual experience? Perceptual presence of an observed object is part of the experience of seeing/sensing even a partly occluded object, steaming from the fact that we know that we have access, at any moment, to the rest of the object thanks to the exercise of SMC knowledge. Perceptual presence of the observer (i.e, the sense of self) is part of the experience of seeing/sensing (even with a , steaming from the fact that we know that we have access, at any moment, to the rest of the sensation thanks to the exercise of SMC knowledge. Dislocated and fragmented sense of self Noë claims that experimental subjects who are made to wear goggles that invert the light entering the eye (so that, e.g., light that normally enters the left side of the eye now enters the right and vice versa) are initially "experientially blind” (althought not sensory blind – but this can be a matter of degree, not a real disctinction! What’s really “sensation”? Perhaps it’s a basic “perception”. Otherwise, we are left with a theory that still has some qualia biases: there would be “atomic” qualia, but qualia all the same…