1. IJCNN 2013 Panel in Higher Emotion 1 Emotion in Brain-Inspired Developmental Networks Juyang (John) Weng Computer Sci., Neurosci., Cognitive Sci. Michigan State University East Lansing, MI 49924 USA weng@cse.msu.edu

2. IJCNN 2013 Panel in Higher Emotion 2 Patterns of Communication (a) Point to Point (b) Release hormones into blood (c) Autonomic nervous system; (d) Diffuse modulatory systems Beart, Connors & Paradiso 2007

3. IJCNN 2013 Panel in Higher Emotion 3 Major Neurotransmitters Beart, Connors & Paradiso 2007

4. IJCNN 2013 Panel in Higher Emotion 4 BMI 831 Cognitive Science for Brain-Mind Research Lecturer: Juyang Weng

5. IJCNN 2013 Panel in Higher Emotion 5 BMI 871 Introduction to Computational Brain-Mind Lecturer: Juyang Weng

6. IJCNN 2013 Panel in Higher Emotion 6 Motivation/emotion in the Brain l Motivational system in the brain: biological causality for the development of emotion l Lower (physiology) emotion: l Arousal: noradrenaline, oxytocin, cortisol Habituation sensitization, familiarization (acetylcholine and norepinephrine systems ) l Pain avoidance (serotonin system) l Pleasure seeking (dopamine system) l Basic emotion: angry, happy, sad, scared, disgust, etc. l Higher emotion: moods, dispositions (e.g., jealousy)

7. IJCNN 2013 Panel in Higher Emotion 7 Neuromodulatory Systems Jeff Krichmar, AB, 2008

8. IJCNN 2013 Panel in Higher Emotion 8 Quiz: Why Modulations Quiz: The main purpose of modulatory systems is: A. Animal like B. Establish bounds with human C. Look intelligent D. Speed-up learning E. Being quick to fight or flee

9. IJCNN 2013 Panel in Higher Emotion 9 Quiz: Why Modulations Quiz: The main purpose of modulatory systems is: A. Animal like B. Establish bounds with human C. Look intelligent D. Speed-up learning E. Being quick to fight or flee

10. IJCNN 2013 Panel in Higher Emotion 10 Why Motivation? l Evolution: Pressure of survival l Understanding of the environment should not be the primary goal of a life l Fruit flies, Rats, Humans l Sexual selection: survival is not sufficient l Males: Fight with males, court with females l Females: Select the male winners l Motivation: quick ways to sense a variety of values l Neuromodulators, diffused transmission

11. IJCNN 2013 Panel in Higher Emotion 11 In the blind, visual cortex is reassigned to audition and touch. Therefore, we chose not to statically model brain areas! Brain areas should emerge

12. IJCNN 2013 Panel in Higher Emotion 12 Motivation: Architectural Concepts

13. IJCNN 2013 Panel in Higher Emotion 13 Symbolic Value Systems l Sutton & Barto 1981: l reward as positive values l Delayed rewards l Ogmen 1997: Punishments, rewards, novelty l Sporns et al. 1999: Darwin robot l Kakade & Dayan 2002: novelty and shaping l Oudeyer et al. 2007: error max; progress max; similarity- based progress maximization l Huang & Weng 2007: punishment, reward, novelty l Cox & Krichmar 2009: Neuromodulation as a robot controller l Singh et al. 2010: reward and evolution

14. IJCNN 2013 Panel in Higher Emotion 14 Symbolic: Q-Learning l Symbolic reinforcement learning l No need for state transition probability l Value driven l Does not allow state inconsistency l Future value estimation for delayed rewards l Time discount model: state-action value:

15. IJCNN 2013 Panel in Higher Emotion 15 Q-Learning Limitations l Symbolic: l Exponential number of states, static l Brittle l Fixed, greedy value function: Immediate rewards are preferred l Does not allow perception l Does not allow creativity for other non- modeled concepts

16. IJCNN 2013 Panel in Higher Emotion 16 Theory: For Any AFA There Is a GDN Weng IJCNN 2010 GDN: Generative Developmental Network AFA: Agent Finite Automaton Marvin Minsky criticized ANNs

17. IJCNN 2013 Panel in Higher Emotion 17 Emergent: 5-HT & DA Systems l Blue: 5-HT system l Red: DA system l Mechanism 1 (extra cell): l Mechanism 2 (within cell): z ip increases the threshold T z is reduces the threshold T If pre-action > T, fire

18. IJCNN 2013 Panel in Higher Emotion 18 5-HT, DA, Ach, NE Serotonin (5-HT): pain, stress, threats and punishment Dopamine: (DA) pleasure, wanting, anticipation, and reward Acetylcholine (Ach): expected uncertainty? Norepinephrine (NE): novelty (unexpected uncertainty)?

19. IJCNN 2013 Panel in Higher Emotion 19 Emergent Value Systems l Daly, Brown, Weng 2011, Paslaski, VanDam, & Weng 2011, Weng et al. Neural Networks 2013: l Neuromorphic Motivated Systems based on DN: 5-HT and DA l Wandering or foraging, face recognition l Wang, Wu, & Weng 2011, 2012 l Neuromorphic Motivated Systems based on DN: Ach and NE l Novelty and Uncertainty l Synapse maintenance l Segmentation of objects from backgrounds l Zheng, Qian, Weng, Zheng 2013 l Effects on internal brain areas: change learning rates

20. IJCNN 2013 Panel in Higher Emotion 20 Effects of 5-HT and DA on Y? Effect on Z area: Inhibit and excite actions Weng et al. IJCNN 2011 Effect on Y area: on learning rate Zheng, Qian, Weng & Zhang, IJCNN 2013

21. IJCNN 2013 Panel in Higher Emotion 21 Dopamine System

22. IJCNN 2013 Panel in Higher Emotion 22 Serotonin System

23. IJCNN 2013 Panel in Higher Emotion 23 Acetylcholine System

24. IJCNN 2013 Panel in Higher Emotion 24 Norepinephrine System

25. IJCNN 2013 Panel in Higher Emotion 25 Bottom-Up Weights and 5-HT

26. IJCNN 2013 Panel in Higher Emotion 26 Y Weights for Z Neurons Paslaski et al. IJCNN 2011

27. IJCNN 2013 Panel in Higher Emotion 27 Error Rates for Recognition Paslaski et al. IJCNN 2011

28. IJCNN 2013 Panel in Higher Emotion 28 Navigation, Wandering, Foraging l Three synthetic agents: l Self (S) l Attractor (A) l Repulsor (R) l 5-HT: distance-lonely and distance- fear l DA: d < distance-desire

29. IJCNN 2013 Panel in Higher Emotion 29 Setting

30. IJCNN 2013 Panel in Higher Emotion 30 Environmental Settings l No brainer (control): no pain, no pleasure l Love or War (D-lonely, D-fear) = (50, 50) l Little Danger (D-lonely, D-fear) = (25, 125) l Much Danger (D-lonely, D-fear) = (125, 25) l Looking over the Fence (D-lonely, D-fear) = (125, 125)

31. IJCNN 2013 Panel in Higher Emotion 31 Average Distance Daly et al. IJCNN 2011

32. IJCNN 2013 Panel in Higher Emotion 32 Ach and NE: Goal for Segmentation Wang, Wu and Weng, IJCNN 2011 (a) Bottom-up input to a neuron. (b) True object contour. (c) Estimated synaptogenic factor

33. IJCNN 2013 Panel in Higher Emotion 33 Global Confidence: Face Recognition

34. IJCNN 2013 Panel in Higher Emotion 34 Ach & NE: Receiver Operating Curve

35. IJCNN 2013 Panel in Higher Emotion 35 ACh Estimates

36. IJCNN 2013 Panel in Higher Emotion 36 DN: Motivational System as Emotion l Emergent nervous system l Modeled: l 5-HT (serotonin) l DA (dopamine) l Ach (acetylcholine) l NE (norepinephrine) l Basic and higher emotion: developed from experience