Cognitive Computing 2012 Embodiment and Enactivism A SENSORIMOTOR ACCOUNT OF VISUAL CONSCIOUSNESS (OREGAN & NOE) Professor Mark Bishop - Images from J. K. ORegan, 2001.

01/04/2014(c) Bishop: Consciousness and Cognition2 The explanatory gap Traditional representational theories of vision leads to a fundamental problem, the explanatory gap: They must either employ a homunculus oraccount for the gap between the physics of a conscious system and the raw feel of experience. But just how could any neural structure be responsible for [give rise to; generate] first person lived experience?

The phenomenology of different sensory modalities Why is the experience of red not like the sound of a bell? Is it because there is something characteristic about the nerve pathways from vision to brain contra ear to brain? Or is it because there is something characteristic about the visual cortex of the brain different from the auditory cortex? 01/04/2014(c) Bishop: Consciousness and Cognition3

01/04/2014(c) Bishop: Consciousness and Cognition4 The problem of representation: change blindness (1) Kevin ORegans seminal work on human change blindness research suggests that ourinternal representationsof the world are very sparse; they contain little or no detail.

01/04/2014(c) Bishop: Consciousness and Cognition5 Change blindness (2) Consider the retinal blindspot where there are no photoreceptors yet we perceive a continuous - and rich - visual field. Similarly colour vision is weak outside the fovea centralis.

01/04/2014(c) Bishop: Consciousness and Cognition6 Change blindness (3) Even if we stare at the area of change in an image, the probability of detecting it is less than 60%.

01/04/2014(c) Bishop: Consciousness and Cognition7 Change blindness (4) The further away a change takes place from the area that we are attending, the less likely we are to notice it even if it quite large..

01/04/2014(c) Bishop: Consciousness and Cognition8 Change blindness (5)

01/04/2014(c) Bishop: Consciousness and Cognition9 Change blindness (6)

01/04/2014(c) Bishop: Consciousness and Cognition10 Change blindness (7) Even if we stare at the area of change in an image we can continue to miss the change. Inattentional Blindness Changes can be slow.

01/04/2014(c) Bishop: Consciousness and Cognition11 A sensorimotor account of vision Experience is not something we passively feel but something we actively do; seeing is a verb! Analogy is of driving a car. This automatically accounts for the difference between the different sensory modalities as each employ different sensorimotor skills. Contrast this sensorimotor account of the phenomenology of perception with: Muellers specific nerve energies (sensory pathways); Cortical areas of activation; In either case why should the phenomenology of perception [of the senses; of the different sensory modalities] be different?

01/04/2014(c) Bishop: Consciousness and Cognition12 The need for high fidelity internal representations in vision? No high fidelity internal representations need be created for functional vision; [Pace Brookes] In this view there is no need for detailed internal representations as the brain can uses the outside world as its own external memory. I.e. Knowledge of the world becomes accessible via our sensorimotor co-ordinated interactions with it. Consider the harmonica in the bag experiment.

01/04/2014(c) Bishop: Consciousness and Cognition13 Harmonica in the bag experiment Childrens game in which someone puts a household object like a cork, or a potato, or a harmonica into a bag, Subject puts hand into the bag and tries to figure out what the object is..

01/04/2014(c) Bishop: Consciousness and Cognition14 Feeling the harmonica At first you have no idea what the object is, but then suddenly you have a kind of aha! experience. Suddenly you feel youre no longer touching bits of texture on the ends of your fingers, but you're holding a whole object: it's a harmonica. And its ALL there at once, even though youre in fact only touching a few parts of it. Its not just that you know its a whole harmonica, you actually feel its a whole harmonica. The reason you have the feeling of touching the whole harmonica is that you have mastery of the appropriate sensorimotor contingencies: You KNOW that IF you were to move your fingers this way, then you would get THIS feeling, and if you moved them that way, you would get THAT feeling.

01/04/2014(c) Bishop: Consciousness and Cognition15 Exploring the world No need for a rich internal representation of the world, we just need to learn algorithms for sensorimotor coordination to explore it. For example, in vision these algorithms would control the eye saccades as we attend visual features of the world. This process results in the grand illusion that we see the whole visual field albeit we actually only attend to a small part of it at any one time.

01/04/2014(c) Bishop: Consciousness and Cognition16 On really seeing Consider our perception of the image opposite: Instead of regularly exploring the image to build up a representation, we simply check to see it confirms our belief that it depicts a man and woman eating. We believe we maintain a rich visual representation of the world, because each time we enquire of some visual feature, our eye immediately saccades to attend to it.

01/04/2014(c) Bishop: Consciousness and Cognition17 Seeing with our eye muscles Seeing only occurs when exorcising our sensorimotor mastery over a scene Image manipulation by moving our eyes. Only the aspects attended to by our eyes are actually perceived. In a recent flight simulator experiment two airline pilots out of eight continued to attempt to land their aircraft even though the runway had a small light aircraft parked across it.

01/04/2014(c) Bishop: Consciousness and Cognition18 A problem If seeing is the process of accessing external memory, why doesn't the phenomenology of seeing correspond to the phenomenology of memory? We visually perceive dreams even though there are no sensorimotor contingencies at play however the dream sight is different to a consistent, rich, real vision of the world. Similarly, seeing my mother is different [and feels different] to recalling a memory of her. This is because (in the 2001 work) ORegan and Noe suggest that perception in the real world (cf. memories) also has the twin dimensions of: Bodiliness. Grabiness.

01/04/2014(c) Bishop: Consciousness and Cognition19 Bodiliness and Grabiness Bodiliness In visual experience the process of exploring the environment is closely tied to a multitude of minute bodily actions which together form a direct linkage between the outside world and the body. Bodiliness is not present in even the most vivid dreams or hallucination. Grabiness Our visual system is hard-wired such that sudden events in the world grab our attention, forcing us to examine them. Grabiness gives us the illusion we have tabs on what is going on.

ORegans 2011 version of SMT In ORegans latest version of SMT (cf. Why red doesnt sound like a bell) he introduces two additional dimensions to characterise the quality of experience. These new dimensions are: (i) The RICHNESS of the real world and (ii) The [partial] insubordinateness of the real world. Together with an additional requirement for cognitive agent awareness [presumably to rule out consciousperception in relatively dumb industrial style robots]: (a) the agent meta-awareness of having a perception and (b) the agent self: the agent/world distinction. 01/04/2014(c) Bishop: Consciousness and Cognition20

01/04/2014(c) Bishop: Consciousness and Cognition21 Memory and vision Memory is the process of accessing information stored in the brain. Vision is the process of accessing information, (stored), in the world. E.g. A list of Latin verbs scores low on grabiness or bodiliness hence we have a different phenomenal experience of them than something visual. BodilinessGrabinessRaw Feel Memory--. Wealthiness+-+ Driving++-++ Seeing NB. The act of driving per-se has no grabiness, however driving involves seeing (which clearly does)..

01/04/2014(c) Bishop: Consciousness and Cognition22 The hard problem of qualia What is it about a red bit of paper that makes us experience red? Redness is defined by the processes that are carried out by the sensorimotor system when the changes of incoming reflectance spectra are typical of a red thing. Hence redness is not just the spectra of a point on a red surface; but the relative changes we observe when looking across the scene.

01/04/2014(c) Bishop: Consciousness and Cognition23 The qualities of sensory modalities Johannes Muller Different pathways have different, nerve energies. O'Regan & Noe Car driving is different from bike riding, is different from juggling, is different from swimming etc. as they all involve mastery of different sensorimotor skills.

01/04/2014(c) Bishop: Consciousness and Cognition24 Two views of vision Representation Vs ActionThe explanatory gap

01/04/2014(c) Bishop: Consciousness and Cognition25 Conclusion (1) 'Enactive' cognitive science was defined (by Varela, Thompson and Rosch, in The Embodied Mind, 1991) as a new theoretical perspective, influenced by the Husserlian phenomenology, Heidegger and Gibsonian ecological approaches to perception. The sensorimotor approach of ORegan [and Noe] is one enactive approach to cognition … … but one whereby seeing is not simply an on-going coupled process between organism and environment, but a particular kind of enacted knowledge where the environment is an external memory and where particular bodiliness and grabiness interactionscolour the seeing..

01/04/2014(c) Bishop: Consciousness and Cognition26 Conclusion (2) The sensorimotor theory attempts to explain the feltness, of mental states. The different sensory modalities; the intermodal gap. The different shades of perception - the ineffable red of a rose rather than the ineffable pink; the intramodal gap. However in additional commentary on sensorimotor theory Hurley and Noe – Neural Plasticity and consciousness, (2003) – seem to set aside the question of the absolute [explanatory] gap … dividing [the problem] in the hope of conquering.