1. EE141 1 Broca’s area Pars opercularis Motor cortexSomatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex Artificial Brain Organization Janusz Starzyk, Ohio University

2. EE141 2  Abstract thinking and action planning  Capacity to learn and memorize useful things  Spatio-temporal memories  Ability to talk and communicate  Intuition and creativity  Consciousness  Emotions and understanding others  Surviving in complex environment and adaptation  Perception  Motor skills in relation to sensing and anticipation Elements of Intelligence

3. EE141 3 Problems of Classical AI  Lack of robustness and generalization  No real-time processing  Central processing of information by a single processor  No natural interface to environment

4. EE141 4 Intelligent Behavior  Emergent from interaction with environment  Based on large number of sparsely connected neurons  Asynchronous  Interact with environment through sensory- motor system  Value driven  Adaptive

5. EE141 5 Sensors Actuators Reactive Associations Sensory Inputs Motor Outputs Simple Brain Organization

6. EE141 6 Simple Brain Properties  Interacts with environment through sensors and actuators  Uses distributed processing in sparsely connected neurons  Uses spatio-temporal associative learning  Uses feedback for input prediction and screening input information for novelty

7. EE141 7 Sensors Actuators Value System Anticipated Response Reinforcement Signal Action Planning Sensory Inputs Motor Outputs Brain Structure with Value System

8. EE141 8 Brain Structure with Value System Properties  Interacts with environment through sensors and actuators  Uses distributed processing in sparsely connected neurons  Uses spatio-temporal associative learning  Uses feedback for input prediction and screening input information for novelty  Develops an internal value system to evaluate its state in environment using reinforcement learning  Plans output actions for each input to maximize the internal state value in relation to environment  Uses redundant structures of sparsely connected processing elements

9. EE141 9 Value System in Reinforcement Learning Control Value System in Reinforcement Learning Control Value System States Controller Reinforcement Signal Environment Optimization

10. EE141 10 Sensors Actuators Value System Anticipated Response Reinf. Signal Sensory Inputs Motor Outputs Action Planning Understanding Decision making Artificial Brain Organization

11. EE141 11  Learning should be restricted to unexpected situation or reward  Anticipated response should have expected value  Novelty detection should also apply to the value system  Need mechanism to improve and compare the value Artificial Brain Organization

12. EE141 12 Sensors Actuators Value System Anticipated Response Reinf. Signal Sensory Inputs Motor Outputs Action Planning Understanding Improvement Detection Expectation Novelty Detection Inhibition Comparison Artificial Brain Organization

13. EE141 13  Anticipated response block should learn the response that improves the value  A RL optimization mechanism may be used to learn the optimum response for a given value system and sensory input  Random perturbation of the optimum should be used to the optimum response in case the value system changed  New situation will result in new value and WTA will chose the winner  Problem is how to do Artificial Brain Organization

14. EE141 14 Artificial Brain Organization

15. EE141 15 Positive Reinforcement Negative Reinforcement Sensory Inputs Motor Outputs Artificial Brain Organization

16. EE141 16 Artificial Brain Selective Processing  Sensory inputs is represented by more and more abstract features in the sensory inputs hierarchy  Possible implementation is to use winner takes all or Hebbian circuits to select the best match  Random wiring may be used to preselect sensory features  Uses feedback for input prediction and screening input information for novelty  Uses redundant structures of sparsely connected processing elements

17. EE141 17 WTA Artificial Brain Organization

18. EE141 18  V. Mountcastle argues that all regions of the brain perform the same algorithm V. Mountcastle  SOLAR combines many groups of neurons (microcolumns) in a pseudorandom way  Each microcolumn has the same structure  Thus it performs the same computational algorithm satisfying Mountcastle’s principle  Mindful Brain Cortical Organization and the Group-Selective Theory of Higher Brain Function G. M. Edelman and V. B. Mountcastle MIT Press, March 1982 Mindful BrainG. M. EdelmanV. B. Mountcastle Microcolumn Organization

19. EE141 19 Microcolumn Organization Positive Reinforcement Negative Reinforcement Sensory Inputs Motor Outputs WTA superneuron

20. EE141 20  Each microcolumn contains a number of superneurons  Within each microcolumn, superneurons compete on different levels of signal propagation  Superneuron contains a predetermined configuration of  Sensory (blue)  Motor and (yellow)  Reinforcement neurons (positive green and negative red)  Superneurons internally organize to perform operations of  Input selection and recognition  Association of sensory inputs  Feedback based anticipation  Learning inhibition  Associative value learning, and  Value based motor activation Superneuron Organization

21. EE141 21  Sensory neurons are primarily responsible for providing information about environment  They receive inputs from sensors or other sensory neurons on lower level  They interact with motor neurons to represent action and state of environment  They provide an input to reinforcement neurons  They help to activate motor neurons  Motor neurons are primarily responsible for activation of motor functions  They are activated by reinforcement neurons with the help from sensory neurons  They activate actuators or provide an input to lower level motor neurons  They provide an input to sensory neurons  Reinforcement neurons are primarily responsible for building the internal value system  They receive inputs from reinforcement learning sensors or other reinforcement neurons on lower level  They receive inputs from sensory neurons  They provide an input to motor neurons  They help to activate sensory neurons Superneuron Organization

22. EE141 22 WTA Sensory Neurons Interactions

23. EE141 23 Sensory Neurons Functions  Sensory neurons are responsible for  Representation of inputs from environment  Interactions with motor functions  Anticipation of inputs and screening for novelty  Selection of useful information  Identifying invariances  Making spatio-temporal associations WTA

24. EE141 24 Sensory Neurons Functions Sensory neurons  Represent inputs from environment by  Responding to activation from lower level (summation)  Selecting most likely scenario (WTA)  Interact with motor functions by  Responding to activation from motor outputs (summation)  Anticipate inputs and screen for novelty by  Correlation to sensory inputs from higher level  Inhibition of outputs to higher level  Select useful information by  Correlating its outputs with reinforcement neurons  Identify invariances by  Making spatio-temporal associations between neighbor sensory neurons

25. EE141 25