1. The ICSI/Berkeley Neural Theory of Language Project Learning early constructions (Chang, Mok) ECG

2. Moving from Spatial Relations to Verbs Open class vs. closed class –How do we represent verbs (say of hand motion) Can we build models of verbs based on motor control primitives? If so, how can models overcome central limitations of Regier’s system? –Inference –Abstract uses

3. Perceptual And Motor Schemas A perceptual schema embodies the process whereby the system determines whether a given domain of interaction is present in the environment. A schema assemblage combines an estimate of environmental state with a representation of goals and needs The internal state is also updated by knowledge of the state of execution of current plans made up of motor schemas which are akin to control systems but distinguished by the fact that they can be combined to form coordinated control programs

4. Coordination of Pattern Generators

5. Coordination PATTERN GENERATORS, separate neural networks that control each limb, can interact in different ways to produce various gaits. –In ambling (top) the animal must move the fore and hind leg of one flank in parallel. –Trotting (middle) requires movement of diagonal limbs (front right and back left, or front left and back right) in unison. –Galloping (bottom) involves the forelegs, and then the hind legs, acting together

6. Preshaping While Reaching to Grasp

7. Hypothetical coordinated control program for reaching and grasping Dashed lines — activation signals; solid lines — transfer of data. (Adapted from Arbib 2004) Perceptual Schemas Motor Schemas

9. Internal Model and Efference Copy

11. Many areas code for motion parameters

13. Multiple, chronically implanted, intracranial microelectrode arrays would be used to sample the activity of large populations of single cortical neurons simultaneously. The combined activity of these neural ensembles would then be transformed by a mathematical algorithm into continuous three-dimensional arm-trajectory signals that would be used to control the movements of a robotic prosthetic arm. A closed control loop would be established by providing the subject with both visual and tactile feedback signals generated by movement of the robotic arm.

14. Rizzolatti et al. 1998 A New Picture

15. The fronto-parietal networks Rizzolatti et al. 1998

16. F5 Mirror Neurons Gallese and Goldman, TICS 1998

17. Category Loosening in Mirror Neurons (~60%) (Gallese et al. Brain 1996) Observed: A is Precision Grip B is Whole Hand Prehension Action: C: precision grip D: Whole Hand Prehension

18. PF Mirror Neurons (Gallese et al. 2002) 1.Neuron responds to execution (grasping) but to grasping and releasing in observation. 2. Mirror neurons in parietal cortex. 3. Difference in left hand and right hand.

19. Umiltà et al. Neuron 2001 A (Full vision) B (Hidden) C (Mimicking) D (HiddenMimicking)

20. F5 Audio-Visual Mirror Neurons Kohler et al. Science (2002)

21. Summary of Fronto-Parietal Circuits Motor-Premotor/Parietal Circuits PMv (F5ab) – AIP Circuit “grasp” neurons – fire in relation to movements of hand prehension necessary to grasp object F4 (PMC) (behind arcuate) – VIP Circuit transforming peri-personal space coordinates so can move toward objects PMv (F5c) – PF Circuit F5c different mirror circuits for grasping, placing or manipulating object Together suggest cognitive representation of the grasp, active in action imitation and action recognition

22. Evidence in Humans for Mirror, General Purpose, and Action-Location Neurons Mirror: Fadiga et al. 1995; Grafton et al. 1996; Rizzolatti et al. 1996; Cochin et al. 1998; Decety et al. 1997; Decety and Grèzes 1999; Hari et al. 1999; Iacoboni et al. 1999; Buccino et al. 2001. General Purpose: Perani et al. 1995; Martin et al. 1996; Grafton et al. 1996; Chao and Martin 2000. Action-Location: Bremmer, et al., 2001.

24. Itti: CS564 - Brain Theory and Artificial Intelligence. FARS Model FARS (Fagg-Arbib-Rizzolatti-Sakata) Model Task Constraints (F6) Working Memory (46?) Instruction Stimuli (F2) AIP Dorsal Stream: Affordances IT Ventral Stream: Recognition Ways to grab this “thing” “It’s a mug” PFC AIP extracts the set of affordances for an attended object.These affordances highlight the features of the object relevant to physical interaction with it.

25. Hypothetical coordinated control program for reaching and grasping Dashed lines — activation signals; solid lines — transfer of data. (Adapted from Arbib 2004) Perceptual Schemas Motor Schemas

26. MULTI-MODAL INTEGRATION The premotor and parietal areas, rather than having separate and independent functions, are neurally integrated not only to control action, but also to serve the function of constructing an integrated representation of: (a)Actions, together with (b)objects acted on, and (c)locations toward which actions are directed. In these circuits sensory inputs are transformed in order to accomplish not only motor but also cognitive tasks, such as space perception and action understanding.

27. Modeling Motor Schemas Relevant requirements (Stromberg, Latash, Kandel, Arbib, Jeannerod, Rizzolatti) –Should model coordinated, distributed, parameterized control programs required for motor action and perception. –Should be an active structure. –Should be able to model concurrent actions and interrupts. –Should model hierarchical control (higher level motor centers to muscle extensor/flexors. Computational model called x-schemas (http://www.icsi.berkeley.edu/NTL)

28. An Active Model of Events At the Computational level, actions and events are coded in active representations called x- schemas which are extensions to Stochastic Petri nets. x-schemas are fine-grained action and event representations that can be used for monitoring and control as well as for inference.

29. Model Review: Stochastic Petri Nets 3 1 2 Basic Mechanism [1] Precondition arc Resource arc Inhibition arc [1] Firing function -- conjunctive -- logistic -- exponential family

30. 3 1 2 Firing Semantics Model Review

31. 1 1 1 1 2 Result of Firing Model Review

32. Active representations Many inferences about actions derive from what we know about executing them Representation based on stochastic Petri nets captures dynamic, parameterized nature of actions Generative model: action, recognition, planning, language Walking: bound to a specific walker with a direction or goal consumes resources (e.g., energy) may have termination condition (e.g., walker at goal ) ongoing, iterative action walker =Harry goal =home energy walker at goal

33. Lecture Overview Moving beyond Spatial Prepositions to Verbs Motor Control Schemas: Recap A Computational Model of Motor Control Schemas –Basic Primitives –Demo of the X-schema model Applications of Model Actions and Inference

54. Preshaping While Reaching to Grasp

71. The ICSI/Berkeley Neural Theory of Language Project Learning early constructions (Chang, Mok) ECG

72. Representing concepts using triangle nodes triangle nodes: when two of the neurons fire, the third also fires

73. BarrettHamContainerPush dept~CS Color ~pink Inside ~region Schema ~slide sid~001 Taste ~salty Outside ~region Posture ~palm emp~GSI Bdy. ~curve Dir. ~ away ChangPeaPurchaseStroll dept~Ling Color ~green Buyer ~person Schema ~walk sid~002 Taste ~sweet Seller ~person Speed ~slow emp~Gra Cost ~money Dir. ~ ANY Goods ~ thing Feature Structures in Four Domains

74. Simulation hypothesis We understand utterances by mentally simulating their content. –Simulation exploits some of the same neural structures activated during performance, perception, imagining, memory… –Linguistic structure parameterizes the simulation. Language gives us enough information to simulate

75. Simulation Semantics BASIC ASSUMPTION: SAME REPRESENTATION FOR PLANNING AND SIMULATIVE INFERENCE – Evidence for common mechanisms for recognition and action (mirror neurons) in the F5 area (Rizzolatti et al (1996), Gallese 96, Buccino 2002, Tettamanti 2004) and from motor imagery (Jeannerod 1996) IMPLEMENTATION: –x-schemas affect each other by enabling, disabling or modifying execution trajectories. Whenever the CONTROLLER schema makes a transition it may set, get, or modify state leading to triggering or modification of other x-schemas. State is completely distributed (a graph marking) over the network. RESULT: INTERPRETATION IS IMAGINATIVE SIMULATION!

76. Simulation-based language understanding Analysis Process Semantic Specification “Harry walked into the cafe.” Utterance CAFE Simulation Belief State General Knowledge Constructions construction W ALKED form self f.phon  [wakt] meaning : Walk-Action constraints self m.time before Context.speech-time self m..aspect  encapsulated

77. Simulation specification A simulation specification consists of: - schemas evoked by constructions - bindings between schemas

78. Language Development in Children 0-3 mo: prefers sounds in native language 3-6 mo: imitation of vowel sounds only 6-8 mo: babbling in consonant-vowel segments 8-10 mo: word comprehension, starts to lose sensitivity to consonants outside native language 12-13 mo: word production (naming) 16-20 mo: word combinations, relational words (verbs, adj.) 24-36 mo: grammaticization, inflectional morphology 3 years – adulthood: vocab. growth, sentence-level grammar for discourse purposes

79. food toys misc. people sound emotion action prep. demon. social Words learned by most 2-year olds in a play school (Bloom 1993)

80. Regier Model Limitations Scale Uniqueness/Plausibility Grammar Abstract Concepts Inference Representation Biological Realism

81. Learning Verb Meanings David Bailey A model of children learning their first verbs. Assumes parent labels child’s actions. Child knows parameters of action, associates with word Program learns well enough to: 1) Label novel actions correctly 2) Obey commands using new words (simulation) System works across languages Mechanisms are neurally plausible.

83. Reasoning about Actions in Artificial Intelligence (AI) The earliest work on actions in AI took a deductive approach –designers hoped to represent all the system's `world knowledge' explicitly as axioms, and use ordinary logic - the predicate calculus - to deduce the effects of actions Envisaging a certain situation S was modeled by having the system entertain a set of axioms describing the situation To this set of axioms the system would apply an action - by postulating the occurrence of some action A in situation S - and then deduce the effect of A in S, producing a description of the outcome situation S'

84. Grasping: the action A set of pre-conditions in S –free_top(y), free_hand(x), accessible(y) The grasp action (effect axiom): –Result(Grasp(x,y, S), hold(x,y,S’)) A set of effects describing the new situation S’ –Hold(x,y), not(free-hand(x))

85. Actions An action is described as an axiom linking preconditions (literals and terms true in the before situation) to effects (literals and terms true in the after situation). The action specification is called an effect axiom

86. Assumptions Necessary and sufficient conditions –Object categories –Event categories Monotonicity –Axioms, once asserted, cannot be retracted –Complex for actions since actions are about change Closed World Assumption

87. Problems with action concepts Frame problem Qualification problem Ramification problem

88. The Frame Problem Which things don’t change in an action –S1: blue(x), on_table(x), free_hand(y) –Action grasp(y,x) –S2: in_hand(x,y), hold(x,y), ?

89. Frame axioms are needed in logic Consider some typical frame axioms associated with the action-type: move x onto y. –If z != x and I move x onto y, then if z was on w before, then z is on w after. –If x is blue before, and I move x onto y, then x is blue after.

90. Active Representations don’t need frame axioms X-schemas directly model change, so no need for frame axioms. Also, they deal with concurrency, so no need to treat one action at a time. Based on x-schema type models there are a new set of logics called resource logics which attempt to model the frame problem directly.

91. Logical approaches to the Frame Problem STRIPS assumptions –Preconditions and effects –Add and Delete Lists for actions –Only one action at a time Non Monotonic Logics –Circumscription (explicitly specify abnormal conditions)

92. Qualification problem How do I specify all the pre-conditions for an action? Problem arises out of the necessary and sufficient conditions for the pre-conditions

93. Approaches to the Qualification problem Traditional Models –STRIPS assumption Modern AI Approach –Probabilistic Models of Actions

94. Ramification Problem How do I specify all the effects –Direct (if I move, I change my location) and –Indirect (things that were accessible before I moved may not be anymore) Central issue is to propagate changes of an action to all the connected knowledge that might be impacted. How might the brain do this? Spreading Activation

95. Solutions to the Ramification Problem Traditional Solution –One action at a time –Closed World assumption Modern AI Solution –Bayes Nets and Probabilistic Models

96. General Modern Solution Use Probabilistic Models to model the indirect effects of actions –Graphical Models –Stochastic Causal Models

97. But… Actions and events (and concepts in general) –Are context sensitive –Admit detailed structure Events can have starts, middles, ends.. –Have prototype structure –Have basic level categorization structure –Can be executed and reasoned about in parallel Concurrency and synchronization

98. Somatotopy of Action Observation Foot Action Hand Action Mouth Action Buccino et al. Eur J Neurosci 2001

99. Pattern Generator Networks a), Four-cell network for bipedal locomotor CPG; b), eight-cell network for quadrupeds; c), 4n-cell network for 2n-legged animals. Double lines indicate contralateral coupling; single lines indicate ipsilateral coupling. Direction of ipsilateral coupling is indicated by arrows; contralateral coupling is bidirectional.