1. CS 182 Sections 103 - 104 Some languages use an absolute orientation system, e.g. 1.Guugu Yimithirr (Cape York, Queensland, Australia) 2.Hai//om (Khoisan, Kalahari, Namibia, Africa) 3.Tzeltal (Mayan, Chiapas, Mexico) They describe all relations with East, South, West, North (e.g. the knife is south of the fork) Imagine using this system: The bathroom is ___________ of this classroom. I am standing ___________ of you guys. Eva Mok (emok@icsi.berkeley.edu) March 3, 2004

2. Announcements a4 is due on Friday. Midterm in class this Tuesday, March 7 th Be there on time! Format: –closed books, closed notes –short answers, no blue books –up to March 2 th lecture Review Session: Monday, March 6 th Time TBA (check the website), Location TBA

3. Quick Recap Last Week –Image Schemas and related experiments This Week –Regier’s Model of learning spatial relation terms Coming up –Motor control

4. Quiz! 1.What are image schemas? What are they useful for? 2.How many senses of the English on can you think of? How would you distinguish them? 3.How is Regier’s model capable of learning without explicit negative examples? How does it learn spatial relation terms in different languages? 4.What mechanism do we use to link concepts together? How does it work? What are other models of learning?

5. Quiz! 1.What are image schemas? What are they useful for? 2.How many senses of the English on can you think of? How would you distinguish them? 3.How is Regier’s model capable of learning without explicit negative examples? How does it learn spatial relation terms in different languages? 4.What mechanism do we use to link concepts together? How does it work? What are other models of learning?

6. Why image schemas? Different languages make different distinctions in their use of spatial relation terms The weaker claim: Image schemas give us a ‘vocabulary’ to talk about the different dimensions of spatial structure that languages care about The stronger claim: These dimensions are embodied -- our bodies constrain the way we observe and interact with the world. Therefore these schemas are universal.

7. English ON AROUND OVER IN Bowerman & Pederson

8. Dutch Bowerman & Pederson ANN OM BOVEN IN OP

9. Chinese Bowerman & Pederson SHANG ZHOU LI

10. The collection of image schemas Trajector / Landmark (asymmetric) –The bike is near the house – ? The house is near the bike Boundary / Bounded Region –a bounded region has a closed boundary Topological Relations –Separation, Contact, Overlap, Inclusion, Surround Orientation –Vertical (up/down), Horizontal (left/right, front/back) –Absolute (E, S, W, N) LM TR bounded region boundary

11. More image schemas Proximal / Distal –distance from center (near/far) Part / Whole –top of the hill, cover of the magazine Container –interior, exterior, boundary, portal Source-Path-Goal –source, path, goal, trajector Force-Dynamics –support, force SG P TR

12. Quiz! 1.What are image schemas? What are they useful for? 2.How many senses of the English on can you think of? How would you distinguish them? 3.How is Regier’s model capable of learning without explicit negative examples? How does it learn spatial relation terms in different languages? 4.What mechanism do we use to link concepts together? How does it work? What are other models of learning?

13. The English ‘on’ 1.The computer is on the desk 2.The picture is on the wall 3.The projector is on the ceiling LM TR DN UP TR/LM, verticality, contact, support LM TR TR/LM, contact, attaching force LM TR TR/LM, contact, attaching force

14. Writing out the on construction The first sense of on SCHEMA TR-LM ROLES tr: trajector lm: landmark SCHEMA Contact ROLES r1: bounded-region r2: region SCHEMA Support ROLES supporter: bounded-region supportee: bounded-region CONSTRUCTION On FORM: Word self.f.orth  “on” MEANING EVOKES TR-LM as trlm EVOKES Contact as c EVOKES Support as s trlm.tr  c.r1 trlm.lm  c.r2 trlm.tr  s.supportee trlm.lm  s.supporter

15. Quiz! 1.What are image schemas? What are they useful for? 2.How many senses of the English on can you think of? How would you distinguish them? 3.How is Regier’s model capable of learning without explicit negative examples? How does it learn spatial relation terms in different languages? 4.What mechanism do we use to link concepts together? How does it work? What are other models of learning?

16. Language Development in Children 0-3 mo: prefers sounds in native language 3-6 mo: imitation of vowel sounds only 6-8 mo: babbling in consonant-vowel segments 8-10 mo: word comprehension, starts to lose sensitivity to consonants outside native language 12-13 mo: word production (naming) 16-20 mo: word combinations, relational words (verbs, adj.) 24-36 mo: grammaticization, inflectional morphology 3 years – adulthood: vocab. growth, sentence-level grammar for discourse purposes

17. Regier’s Model Training input: configuration of TR/LM and the correct spatial relation term Learned behavior: input TR/LM, output spatial relation Learning System abovebelowleftrightinoutonoff Input: TR LM above

18. Issue #1: Implicit Negatives Children usually do not get explicit negatives But we won’t know when to stop generalizing if we don’t have negative evidence Yet spatial relation terms aren’t entirely mutually exclusive The same scene can often be described with two or more spatial relation terms (e.g. above and outside) How can we make the learning problem realistic yet learnable?

19. Dealing with Implicit Negatives Explicit positive for above Implicit negatives for below, left, right, etc in Regier: E = ½ ∑ i,p (( t i,p – o i,p ) * β i,p ) 2, where i is the node, p is the pattern, β i,p = 1 if explicit positive, β i,p < 1 if implicit negative

20. Issue #2: Shift Invariance Backprop cannot handle shift invariance (it cannot generalize from 0011, 0110 to 1100) But the cup is on the table whether you see it right in the center or from the corner of your eyes (i.e. in different areas of the retina map) What structure can we utilize to make the input shift- invariant?

21. Learning System We’ll look at the details next lecture dynamic relations (e.g. into) structured connectionist network (based on visual system)

22. Quiz! 1.What are image schemas? What are they useful for? 2.How many senses of the English on can you think of? How would you distinguish them? 3.How is Regier’s model capable of learning without explicit negative examples? How does it learn spatial relation terms in different languages? 4.What mechanism do we use to link concepts together? How does it work? What are other models of learning?

23. Models of Learning Hebbian – coincidence Reinforcement – delayed reward Recruitment – one-trial Supervised – correction (backprop) Unsupervised – similarity

24. Elman Nets & Jordan Nets Basically the same recurrent nets we mentioned a few weeks ago Updating the context as we receive input In Jordan nets we model “forgetting” as well The recurrent connections have fixed weights You can train these networks using good ol’ backprop Output Hidden ContextInput 1 α Output Hidden ContextInput 1

25. Recurrent Backprop we’ll pretend to step through the network one iteration at a time backprop as usual, but average equivalent weights (e.g. all 3 highlighted edges on the right are equivalent) abc unrolling 3 iterations abc abc abc w2 w1w3 w4 w1w2w3w4 abc

26. The Idea of Recruitment Learning Suppose we want to link up concept X to concept Y The idea is to pick the two nodes in the middle to link them up Can we be sure that we can find a path to get from X to Y? the point is, with a fan-out of 1000, if we allow 2 intermediate layers, we can almost always find a path X Y BNK F = B/N

27. Remember triangle nodes? Let’s say we are trying to remember a green circle currently weak connections between concepts (dotted lines) has-color bluegreenroundoval has-shape

28. Strengthen these connections and you end up with this picture has-color bluegreenroundoval has-shape Green circle

29. Demo of the Regier System on the English above

30. above – positive examples

31. above – negative examples

32. above – after training

33. above – test examples