Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 1 Laurent Itti: CS564 - Brain Theory and Artificial Intelligence Lecture 1. Introduction and Brain Overview Reading Assignments:* TMB2: Chapters 1; 2.4 HBTNN: I.1. Introducing the Neuron (Arbib) * Unless indicated otherwise, the TMB2 material is the required reading, and the other readings supplementary.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 2 CS 564: Brain Theory and Artificial Intelligence URL: for syllabus, instructor and TA information, handouts, homework and grades Instructor: Laurent Itti; (Office Hour: Mon 3-5, HNB30A) TA: Yoo-Hee Shin This course provides a basic understanding of brain function, and of artificial neural networks which provide tools for a new paradigm for adaptive parallel computation. No background in neuroscience is required, nor is specific programming expertise, but knowledge of C++ will be useful for homeworks.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 3 Texts and Grading Text: M.A. Arbib, 1989, The Metaphorical Brain 2: Neural Networks and Beyond, Wiley-Interscience. Supplementary reading: M.A. Arbib, Ed., 1995, The Handbook of Brain Theory and Neural Networks, MIT Press (paperback). One mid-term and a final will cover the entire contents of the readings so far as well as the lectures. The final exam will cover all of the course, but emphasizing material not covered in the mid-term. Distribution of Grades: Homeworks: 40%; Mid-term: 30%; Final Exam: 30%.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 4 Syllabus Overview Introduction Overview Charting the brain The Brain as a Network of Neurons

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 5 Syllabus Overview Introduction (cont.) Experimental techniques Introduction to Vision Schemas

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 6 Syllabus Overview Basic Neural Modeling & Adaptive Networks Didday Model of Winner-Take-All Hopfield networks Adaptive networks: Hebbian learning; Perceptrons; landmark learning E = - ½  ij s i s j w ij +  i s i  i

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 7 Syllabus Overview Neural Modeling & Adaptive Networks (cont.) Adaptive networks: gradient descent and backpropagation Reinforcement learning Competition and cooperation Visual plasticity; self-organizing feature maps; Kohonen maps                  

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 8 Syllabus Overview Examples of Large-scale Neural Modeling System concepts Model of saccadic eye movements Feedback and the spinal cord; mass-spring model of muscle

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 9 Syllabus Overview Large-scale Neural Models of Vision Early visual processing Depth perception Optic flow

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 10 Syllabus Overview Large-scale Neural Models of Vision (cont.) Visual attention Object recognition Scene perception

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 11 Task Constraints (F6) Working Memory (46?) Instruction Stimuli (F2) AIP Dorsal Stream: Affordances IT Ventral Stream: Recognition Ways to grab this “thing” “It’s a mug” PFC Syllabus Overview Other Advanced Neural Modeling Reaching, grasping and affordances Cerebellar adaptation Memory and consciousness Visual Cortex Parietal Cortex Inferotemporal Cortex How (dorsal) What (ventral) reach programming grasp programming

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 12 Syllabus Overview Applications and Outlook Towards highly-capable robots Overview and summary

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 13 Three Frameworks Artificial intelligence (AI): build a “packet of intelligence” into a machine Cognitive psychology: explain human behavior by interacting processes (schemas) “in the head” but not localized in the brain Brain Theory: interactions of components of the brain - - computational neuroscience - neurologically constrained models: e.g., networks of neurologically localized schemas and abstracting from them as both Artificial intelligence and Cognitive psychology: - connectionism: networks of trainable “quasi-neurons” to provide “parallel distributed models” little constrained by neurophysiology - abstract (computer program or control system) information processing models

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 14 The Aim of the Course To gain an understanding of biological neurons as the basis for: Brain Theory: modeling interactions of components of the brain, especially more or less realistic biological neural networks localized in specific brain regions Connectionism in both Artificial intelligence (AI) and Cognitive psychology: modeling artificial neural networks -- networks of trainable “quasi-neurons” -- to provide “parallel distributed models” of intelligence in humans, animals and machines This lecture: A tourist’s guide to the brain ;-)

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 15 A motivating theme: Vision Vision as a progressive change in representation Marr (1982): through 2 ½ D primal sketch Because vision is by far the most studied sense (because it is easy to experiment with), we will use it as a basis for many examples of models studied in this course.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 16 Vision and the brain Roughly speaking, about half of the brain is concerned with vision. Although most of it is highly auto- mated and unconscious, vision hence is a major component of brain function. Ryback et al, 1998

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 17 Vision, AI and robots 1940s: beginning of Artificial Intelligence McCullogh & Pitts, 1942  i w i x i   Perceptron learning rule (Rosenblatt, 1962) Backpropagation Hopfield networks (1982) Kohonen self-organizing maps …

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 18 Vision, AI and Robots 1950s: beginning of computer vision Aim: give to machines same or better vision capability as ours Drive: AI, robotics applications and factory automation Initially: passive, feedforward, layered and hierarchical process that was just going to provide input to higher reasoning processes (from AI) But soon: realized that could not handle real images 1980s: Active vision: make the system more robust by allowing the vision to adapt with the ongoing recognition/interpretation

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 19 A tourist’s guide to the brain Gross anatomy Non-neural structures Major cortical areas

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 20 Central vs. Peripheral Nervous System The brain is not the entire nervous systems; there is also the spinal cord, many peripheral “ganglia” (small clusters of neurons), and neurons extend connections to locations all over the body (e.g., sensory neurons, motor neurons).

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 21 Autonomic Nervous System

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 22

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 23 Axes in the brain

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 24 The “Bauplan” for the Mammalian Brain

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 25 Medical Orientation Terms for Slices

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 26 Main Arterial Supply to the Brain

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 27 Arterial Supply is Segmented Occlusion/damage to one artery will affect specific brain regions. Important to remember for patient studies.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 28 Ventricular System Ventricules: Cavities filled with fluid inside and around the brain. One of their functions is to drain garbage out of the brain.

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 29 Cortical Lobes Sulcus (“fissure” if very large): Grooves in folded cortex Gyrus: cortex between two sulci 1 sulcus, many sulci; 1 gyrus, many gyri

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 30

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 31

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 32

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 33

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 34

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 35

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 36 Neurons Cell body (soma): where computation takes place Dendrites: input branches Axon: unique output (but may branch out) Synapse: connection between presynaptic axon and postsynaptic dendrite (in general).

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 37 Electron Micrograph of a Real Neuron

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 38 Neurons and Synapses

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 39 Grey and White Matters Grey matter: neurons (cell bodies), at outer surface of brain White matter: interconnections, inside the brain Deep nuclei: clusters of neurons deep inside the brain

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 40 Major Functional Areas Primary motor: voluntary movement Primary somatosensory: tactile, pain, pressure, position, temp., mvt. Motor association: coordination of complex movements Sensory association: processing of multisensorial information Prefrontal: planning, emotion, judgement Speech center (Broca’s area): speech production and articulation Wernicke’s area: comprehen- sion of speech Auditory: hearing Auditory association: complex auditory processing Visual: low-level vision Visual association: higher-level vision

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 41 Major Functional Areas

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 42 A View of the Monkey Brain

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 43

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 44

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 45

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 46

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 47

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 48

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 49

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 50

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 51

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 52 Limbic System Cortex “inside” the brain. Involved in emotions, sexual behavior, memory, etc (not very well known)

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 53 Major Functional Areas

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 54 Visual Input to the Brain

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 55 Eye and retina

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 56 Human Visual System

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 57 Primary Visual Pathway

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 58 Layered Organization of Cortex Cortex is 1 to 5mm-thick, folded at the surface of the brain (grey matter), and organized as 6 superimposed layers. Layer names: 1: Molecular layer 2: External granular layer 3: External pyramidal layer 4: internal granular layer 5: Internal pyramidal layer 6: Fusiform layer Basic layer functions: Layers 1/2: connectivity Layer 4: Input Layers 3/5: Pyramidal cell bodies Layers 5/6: Output

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 59 Layered Organization of Cortex

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 60 Slice through the thickness of cortex

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 61 Columnar Organization Very general principle in cortex: neurons processing similar “things” are grouped together in small patches, or “columns,” or cortex. In primary visual cortex… as in higher (object recognition) visual areas… and in many, non-visual, areas as well (e.g., auditory, motor, sensory, etc).

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 62 Retinotopy Many visual areas are organized as retinotopic maps: locations next to each other in the outside world are represented by neurons close to each other in cortex. Although the topology is thus preserved, the mapping typically is highly non- linear (yielding large deformations in representation). Stimulus shown on screen… and corresponding activity in cortex!

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 63 Retinotopy

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 64 Mammalian and Frog Visual Systems

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 65

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 66

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 67 Felleman & Van Essen, 1991 Interconnect

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 68 Interconnect… (other source)

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 69 More on Connectivity

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 70 Frog Snake Horse Primitive Mammal Catfish Alligator Goose Varieties of Vertebrate Brains

Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Introduction and Brain Overview 71 Outlook There is a lot to learn about the brain! … but don’t feel overwhelmed, we will smoothly introduce all new concepts. Principled theoretical and engineering methods will allow us to abstract some of these complications. Starting with fundamental techniques, we will then study fairly complex, large-scale neural models.