1 Consciousness and Cognition Janusz A. Starzyk Cognitive Architectures

Motivated Learning  Various pains and external signals compete for attention.  Attention switching results from competition.  Cognitive perception is aided by winner of competition.  Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent.  Machine creates abstract goals based on the primitive pain signals.  It receives internal rewards for satisfying its goals (both primitive and abstract).  ML applies to EI working in a hostile environment.

Reinforcement Learning Motivated Learning  Single value function  Various objectives  Measurable rewards  Predictable  Objectives set by designer  Maximizes the reward  Potentially unstable  Learning effort increases with complexity  Always active  Multiple value functions  One for each goal  Internal rewards  Unpredictable  Sets its own objectives  Solves minimax problem  Always stable  Learns better in complex environment than RL  Acts when needed

Primitive Goal Creation -+ Pain Dry soil Primitive level open tank sit on garbage refill faucet w. can water Dual pain  Reinforcing a proper action

Abstract Goal Hierarchy  Abstract goals are created to reduce abstract pains and to satisfy the primitive goals  A hierarchy of abstract goals is created to satisfy the lower level goals Activation Stimulation Inhibition Reinforcement Echo Need Expectation -+ + Dry soil Primitive Level Level I Level II faucet - w. can open water + Sensory pathway (perception, sense) Motor pathway (action, reaction) Level III tank - refill

Drought Reservoir Irrigate Thirsty Water Drink Water Primitive Needs Dirty Wash in Water Abstract Needs Primitive needs

Drought Reservoir Public Money Irrigate Spend Money to Build Thirsty Water Drink Water Spend Money to Buy Primitive Needs Well Draw own Water Dirty Wash in Water Abstract Needs Abstract needs

Drought Reservoir Public Money Tourists' Attractions Irrigate Spend Money to Build Build Ecotourism Thirsty Water Drink Water Spend Money to Buy Primitive Needs Build Water Recreation Wealthy Taxpayers Rise Taxes Well Draw own Water Dirty Wash in Water Well Building Dig a Well Abstract Needs Ground Water Water Supply Abstract needs

Drought Reservoir Public Money Tourists' Attractions Irrigate Spend Money to Build Build Ecotourism Thirsty Water Drink Water Spend Money to Buy Primitive Needs Build Water Recreation Policy Develop Infrastructure Wealthy Taxpayers Rise Taxes Well Draw own Water Dirty Wash in Water Well Building Dig a Well Abstract Needs Employment Opportunities Ground Water Water Supply Receive Salary Resource Management and Planning Management Regulate Use Planning Abstract needs

Definition of Machine Consciousness Consciousness is attention driven cognitive perception motivations, thoughts, plans and action monitoring. A machine is conscious IFF besides ability to perceive, act, learn and remember, it has a central executive mechanism that controls all the processes (conscious or subconscious) of the machine; Photo:

 Intelligence  Central executive  Attention and attention switching  Mental saccades  Cognitive perception  Cognitive action control Photo: Consciousness: functional requirements

Computational Model of Machine Consciousness Semantic memory Sensory processors Data encoders/ decoders Sensory units Motor skills Motor processors Data encoders/ decoders Motor units Emotions, rewards, and sub-cortical processing Attention switching Action monitoring Motivation and goal processor Planning and thinking Episodic memory Queuing and organization of episodes Episodic Memory & Learning Central Executive Sensory-motor Inspiration: human brain Photo (brain):

13 Sensory and Motor Hierarchies  Sensory and motor systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies.

Sensory- Motor Block Semantic memory Sensory processors Data encoders/ decoders Sensory units Motor skills Motor processors Data encoders/ decoders Motor units Emotions, rewards, and sub-cortical processing Sensory-motor  sensory processors integrated with semantic memory  motor processors integrated with motor skills  sub-cortical processors integrated with emotions and rewards

Central Executive  Platform for the emergence, control, and manifestation of consciousness  Controls its conscious and subconscious processes  Is driven by  attention switching  learning mechanism  creation and selection of motivations and goals ahsmail.uwaterloo.ca/kin356/cexec/cexec.htm

Attention switching Action monitoring Motivation and goal processor Planning and thinking Central Executive  Tasks o cognitive perception o attention o attention switching o motivation o goal creation and selection o thoughts o planning o learning, etc. Central Executive

 Interacts with other units for o performing its tasks o gathering data o giving directions to other units  No clearly identified decision center  Decisions are influenced by o competing signals representing motivations, pains, desires, plans, and interrupt signals need not be cognitive or consciously realized o competition can be interrupted by attention switching signal Attention switching Action monitoring Motivation and goal processor Planning and thinking Central Executive

 Attention  is a selective process of cognitive perception, action and other cognitive experiences like thoughts, action planning, expectations, dreams  Attention switching  is needed to have a cognitive experience  leads to sequences of cognitive experiences Comic: Attention Switching !!!

 Dynamic process resulting from competition between representations related to motivations sensory inputs internal thoughts including spurious signals (like noise). blog.gigoo.org/.../

Attention Switching !!! Thus, while paying attention is a conscious experience, switching attention does not have to be. May be a result of : deliberate cognitive experience (and thus fully conscious signal) subconscious process (stimulated by internal or external signals)

Formulate episode Saccade control Changing perception Changing environment Advancement of a goal? YesNo Action control Changing motivation Write to episodic memory Loop 1 Loop 2 Attention spotlight Associative memory From virtual game Simplified Cognitive Machine

Input image A B C D A B C D A B C D WhatWhere Visual Saccades

Mental Saccades  This in turn activates memory traces in the global workspace area that will be used for mental searches (mental saccades).  Selected part of the image resulting from an eye saccade.  Perceived input activates object recognition and associated areas of semantic and episodic memory.

Mental saccades in a conscious machine Perceptual saccadesChanging perception Changing environment Associative memory No Action control Loop 5 Loop 2 Perceptual saccadesChanging perception Changing environment Associative memory No Action control Advancement of a goal? Yes Learning Advancement of a goal? Advancement of a goal? Yes Learning Attention spotlight Mental saccades Continue search? Yes Loop 1 Attention spotlight Mental saccades Continue search? Continue search? Yes Loop 1 Plan action? No Yes Action? Yes No Changing motivation Loop 3 Loop 4 Plan action? No Yes Action? Yes No Changing motivation Loop 3 Loop 4 Loop 5 Loop 2

Mental saccades network Associative memory Winner takes all AHF Dual neurons Winner inhibition Primed memory Next mental saccade Attention spotlight Mental saccades Continue search? Yes Loop 1 Attention spotlight Mental saccades Continue search? Continue search? Loop 1 No

Mental saccades network Associative memory Winner takes all F Dual neurons Winner inhibition Primed memory Next mental saccade AH

Mental saccades network Associative memory Winner takes all F Dual neurons Winner inhibition Primed memory Next mental saccade AH

Mental saccades network Associative memory Winner takes all F Dual neurons Winner inhibition Primed memory Next mental saccade AH

Inhibition of the next mental saccade Associative memory AHnF Primed memory PAPA PHPH PnPn PFPF A2A2 A3A3 A4A4 A k-1 M1M1 M2M2 M3M3 MkMk A1A1 AkAk Next mental saccadePotential pain reduction detected New winner established To next mental saccade To learning Advancement of a goal? Yes Learning Advancement of a goal? Advancement of a goal? Yes Learning No Attention spotlight Evaluation delay

Inhibition of the next mental saccade Associative memory AHF Primed memory PAPA PHPH PFPF A3A3 A4A4 A k-1 M1M1 M2M2 M3M3 MkMk A1A1 AkAk Next mental saccadePotential pain reduction detected New winner established To next mental saccade To learning Evaluation delay Learning delay n PnPn A2A2

Inhibition of the next mental saccade Associative memory AHF Primed memory PAPA PHPH PFPF A3A3 A4A4 A k-1 M1M1 M2M2 M3M3 MkMk A1A1 AkAk Next mental saccade Potential pain reduction detected New winner established To next mental saccade To learning Evaluation delay Learning delay n PnPn A2A2

Inhibition of the next mental saccade Associative memory AHF Primed memory PAPA PHPH PFPF A3A3 A4A4 A k-1 M1M1 M2M2 M3M3 MkMk A1A1 AkAk Next mental saccade Potential pain reduction detected New winner established To next mental saccade To learning Evaluation delay Learning delay n PnPn A2A2

Associative memory H’n’F’ Dual memory P’ K P’ H P’ n P’ F A2A2 A3A3 A4A4 A k-1 M1M1 M2M2 M3M3 MkMk A1A1 AkAk I3I3 I4I4 I k-1 IkIk H nF PKPK PHPH PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning Plan action? No Yes Action? Yes No Changing motivation Loop 3 Loop 4 Plan action? No Yes Action? Yes Changing motivation Loop 3 Loop 4 Action control K I1I1 K’ I2I2

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk PHPH I1I1 A 2 - desired action “add sugar to tea” is blocked H sugar P H no sugar I 1 intended action buy sugar A2A2 M1M1 A1A1 H K’ K B2B2 H’ I2I2 B1B1

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk PHPH I1I1 K’ search for money K money observed A2A2 M1M1 A1A1 H K’ K B2B2 I2I2 B1B1 H’

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk B1B1 PHPH I1I1 A2A2 M1M1 H K B2B2 H’ I2I2 B 1 action block removed A 1 action performed H’ sugar expected A1A1 K’

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk B1B1 M1M1 H’ I2I2 H sugar observed B 2 action block removed A 2 action performed K H B2B2 A2A2 PHPH I1I1 K’ A1A1

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk PHPH However if K’ search for money K money not observed I 4 desired action get money A2A2 M1M1 A1A1 H K’ K B2B2 I2I2 B1B1 H’ I1I1 PKPK

Predicted changes known Pain increase Predicted changes Intended action Associative memory Cognitive action control Lower level action control Action? Cognitive abort Action control

Action and subgoal planning Intended action Induced pain Dual pain Perception Pain reduction Next mental saccade Perform action Learning Pain Environment Decide action Attention spotlight Desired item

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning M2M2 M3M3 MkMk PHPH A 2 - desired action “add sugar to tea” is blocked H sugar P H no sugar I 1 intended action buy sugar A2A2 M1M1 A1A1 H K’ K H’ I2I2 I1I1

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk PHPH I1I1 K’ search for money K money observed A2A2 M1M1 A1A1 H K’ K B2B2 I2I2 B1B1 H’

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk B1B1 PHPH I1I1 A2A2 M1M1 H K B2B2 H’ I2I2 B 1 action block removed A 1 action performed H’ sugar expected A1A1 K’

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PKPK PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk B1B1 M1M1 H’ I2I2 H sugar observed B 2 action block removed A 2 action performed K H B2B2 A2A2 PHPH I1I1 K’ A1A1

Associative memory n’F’ Dual memory P’ K P’ H P’ n P’ F A3A3 A4A4 A k-1 AkAk I3I3 I4I4 I k-1 IkIk nF PnPn PFPF Primed memory Pain Dual pain Action and subgoals planning B3B3 B4B4 B k-1 M2M2 M3M3 MkMk BkBk PHPH However if K’ search for money K money not observed I 4 desired action get money A2A2 M1M1 A1A1 H K’ K B2B2 I2I2 B1B1 H’ I1I1 PKPK

Computational Model: Summary  Self-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines.  A central executive controls conscious and subconscious processes driven by its attention switching mechanism.  Attention switching is a dynamic process resulting from competition between representations, sensory inputs and internal thoughts  Mental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring Photo: training.html

Computational Model: Implications  Motivations for actions are physically distributed o competing signals are generated in various parts of machine’s mind  Before a winner is selected, machine does not interpret the meaning of the competing signals  Cognitive processing is predominantly sequential o winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner  Top down activation for perception, planning, internal thought or motor functions o results in conscious experience decision of what is observed and where is it planning how to respond o a train of such experiences constitutes consciousness

Conclusions 1.Consciousness is computational 2.Intelligent machines can be conscious

Sounds like science fiction?  If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong.  But if it doesn’t seem like science fiction, it’s definitely wrong. From presentation by Feresight Institute

Questions ?? Photo:

References  J. A. Fodor, "The big idea: can there be science of the mind," Times Literary Supplement, pp. 5-7, July  P.A.O. Haikonen, “The cognitive approach to conscious machines”. UK: Imprint Academic,  J. Bach, “Principles of Synthetic Intelligence PSI: An Architecture of Motivated Cognition”, Oxford Univ. Press,  B. J. Baars “A cognitive theory of consciousness,” Cambridge Univ. Press,  M. Velmans, "Where experiences are: Dualist, physicalist, enactive and reflexive accounts of phenomenal consciousness," Phenomenology and the Cognitive Sciences, vol. 6, pp , 2007  A. Sloman, "Developing concept of consciousness," Behavioral and Brain Sciences, vol. 14 (4), pp , Dec  W. H. Calvin and G. A. Ojemann, Conversation with Neil's brain: the neural nature of thought and language: Addison-Wesley,  J. Hawkins and S. Blakeslee, On intelligence. Henry Holt & Company, LLC.,  S. Greenfield, The private life of the brain. New York: John Wiley & Sons,  Nisargadatta, I am that. Bombay: Chetana Publishing,  D. C. Dennett, Consciousness Explained, Penguin Press,1993.  D. M. Rosenthal, The nature of Mind, Oxford University Press, Photo:

Embodied Intelligence –Mechanism: biological, mechanical or virtual agent with embodied sensors and actuators –EI acts on environment and perceives its actions –Environment hostility is persistent and stimulates EI to act –Hostility: direct aggression, pain, scarce resources, etc –EI learns so it must have associative self-organizing memory –Knowledge is acquired by EI Definition  Embodied Intelligence (EI) is a mechanism that learns how to minimize hostility of its environment

Embodiment of a Mind  Embodiment is a part of the environment that EI controls to interact with the rest of the environment  It contains intelligence core and sensory motor interfaces under its control  Necessary for development of intelligence  Not necessarily constant or in the form of a physical body  Boundary transforms modifying brain’s self- determination

 Brain learns own body’s dynamic  Self-awareness is a result of identification with own embodiment  Embodiment can be extended by using tools and machines  Successful operation is a function of correct perception of environment and own embodiment Embodiment of a Mind