1. Modelling the evolution of language for modellers and non-modellers IJCAI-05 1 Modelling the evolution of language for modellers and non-modellers Introduction and techniques

2. Modelling the evolution of language for modellers and non-modellers IJCAI-05 2 Today’s presenters Paul Vogt –Language evolution and computation research unit, University of Edinburgh, UK –Induction of linguistic knowledge group, Tilburg University, The Netherlands Tony Belpaeme (not here) –Center for Interactive Intelligent Systems Univeristy of Plymouth, UK Bart de Boer –Department of artificial intelligence Rijksuniversiteit Groningen, the Netherlands All alumni of the Brussels AI-lab All AI-researchers with strong “cognitive” focus

3. Modelling the evolution of language for modellers and non-modellers IJCAI-05 3 Why this Tutorial? AI in the sense of using computer models to understand human intelligence benefits from interaction with other disciplines –Evolution of language is such a discipline The study of language evolution deals with systems that are so complex that they need to be modeled with computers –Linguists/paleontologists appreciate modeling, but are usually no good with computers themselves

4. Modelling the evolution of language for modellers and non-modellers IJCAI-05 4 Our aims To introduce the field of language evolution to the AI community To present examples of possible models –Based on our own work To explain how to communicate outside the field of AI –Linguists/paleontologists read AI papers in a different way than AI researchers

5. Modelling the evolution of language for modellers and non-modellers IJCAI-05 5 Organisation of the tutorial Theory 14:00–14:45Introduction and techniques (Bart) 14:45–15:30 Topics of research (Paul) BREAK 16:00–16:45 Communication and caveats (Paul/Tony) Practical Examples 16:45–17:00 Vowel systems (Bart) 17:00–17:15 Talking Heads simulator (Paul) 17:15–17:30 Hands-on demonstration (Bart/ Tony)

6. Modelling the evolution of language for modellers and non-modellers IJCAI-05 6 Language Evolution Bart de Boer

7. Modelling the evolution of language for modellers and non-modellers IJCAI-05 7

8. Modelling the evolution of language for modellers and non-modellers IJCAI-05 8 Early Scientific Experiments Pharaoh Psamtik I Frederick II von Hohenstaufen James IV of Scotland

9. Modelling the evolution of language for modellers and non-modellers IJCAI-05 9 And speculation… Jespersen’s critique –Bow-wow theory –Pooh-pooh theory –Ding-dong theory –Yo-he-ho theory But his own theory: –La-la theory

10. Modelling the evolution of language for modellers and non-modellers IJCAI-05 10 As a result: Also: Chomsky considered it impossible (and uninteresting) to study language evolution

11. Modelling the evolution of language for modellers and non-modellers IJCAI-05 11 Can we do better today? 1990: Pinker & Bloom “Natural Language and Natural Selection” Since 1996 biannual Evolang Conference –Palaeontology –Archaeology –Anthropology –Linguistics –Biology –Ethology –Etc… –And of course: Computer modelling

12. Modelling the evolution of language for modellers and non-modellers IJCAI-05 12 Why language? Interesting and difficult question –Many factors play a role (including chance), complex dynamics –Possibilities for modelling

13. Modelling the evolution of language for modellers and non-modellers IJCAI-05 13 Communication in Animals Humans are not the only ones with complex communication

14. Modelling the evolution of language for modellers and non-modellers IJCAI-05 14 Relation with primates Chimpanzees are very smart –But do not learn how to speak –And only learn sign language with difficulty Apes do communicate vocally –But more comparable with involun- tary human cries of pain, joy, laughter etc. Neural structures in ape and monkey brains for manipulation and vocalization are analogues of human brain structures for speech

15. Modelling the evolution of language for modellers and non-modellers IJCAI-05 15 What evolved? Very specific mechanisms? –“Universal Grammar” –Principles and Parameters More general learning mechanisms, some specialised for communication? Completely general mechanisms –Language itself evolved culturally “Nature versus nurture”

16. Modelling the evolution of language for modellers and non-modellers IJCAI-05 16 Which of these had language? Australopithecus africanus Homo erectus Homo neanderthalensis

17. Modelling the evolution of language for modellers and non-modellers IJCAI-05 17 When did language emerge? Two extremes: –Late emergence (~30 000 years ago) –Early emergence (A. africanus) But there is no direct archaeological evidence

18. Modelling the evolution of language for modellers and non-modellers IJCAI-05 18 The argument for late emergence Symbolic explosion –About 30 000 years ago humans started to produce art –Problem: European bias, nowadays earlier and earlier finds –Appears to have emerged and disappeared repeatedly

19. Modelling the evolution of language for modellers and non-modellers IJCAI-05 19 Against late emergence How can complex language evolve so quickly? How does one explain biological adaptations to language? Homo sapiens started to spread much earlier than 50–70 000 years ago –Would language have emerged in different places?

20. Modelling the evolution of language for modellers and non-modellers IJCAI-05 20 Fossil evidence Hypoglossal canal (Kay, Cartmill & Balow) –For tongue control –Not enlarged in Homo erectus, but in early sapiens and Neanderthal (>400 000 years) Thoracic vertebral canal (MacLarnon & Hewitt) –For diaphragm control –Not enlarged in Homo ergaster, but in Neanderthal and modern man

21. Modelling the evolution of language for modellers and non-modellers IJCAI-05 21 Modern Language Are there primitive languages? –No, not if native –No data on language evolution –But data on possibilities of language But: pidgin-languages –Jargons, second language etc. –And creolisation Or emergence of new language –Nicaraguan sign language Idea: proto-language –Bickerton

22. Modelling the evolution of language for modellers and non-modellers IJCAI-05 22Jackendoff Pre-existing primate conceptual structure Use of symbols in non-situation-specific fashion Use of an open, unlimited class of symbols Concatenation of symbols Development of a phonological combinatorial system to enlarge open, unlimited class of symbols Use of symbol position to convey basic semantic relations (Protolanguage about here) Hierarchical phrase structure Symbols that explicitly encode abstract semantic relations Grammatical categories System of inflections to convey semantic relations System of grammatical functions to convey semantic relations (Modern language

23. Modelling the evolution of language for modellers and non-modellers IJCAI-05 23 Deep history of language Historical linguistics can reconstruct older forms of a language (e.g. indo-european) –Traditional linguistics up to ~8000 years ago But: –Ruhlen claims proto-world –Very unlikely –Human expansion started about 150 000 yrs ago –After this time all similarities are gone

24. Modelling the evolution of language for modellers and non-modellers IJCAI-05 24 Language and Genes

25. Modelling the evolution of language for modellers and non-modellers IJCAI-05 25 Conclusion Language >400 000 years old No primitive languages exist, and we know little about how they spread very long ago –But we can observe emergence of new languages –That all have certain special properties –We can also observe incomplete languages Language probably emerged as primitive proto- language –Why is an interesting, but hard-to-answer question How did it spread, how did it emerge? –Cannot be reconstructed from fossils –But possible to model

26. Modelling the evolution of language for modellers and non-modellers IJCAI-05 26 Techniques Bart de Boer

27. Modelling the evolution of language for modellers and non-modellers IJCAI-05 27 The process of modelling Choose a cognitive/linguistic problem Gather data and theories on which to base a model Implement the theory as a computer model –Make abstractions of reality –Make mappings abstractions  reality –Make measures of performance of the model –Implement the model using abstractions and tradeoffs Run tests for different parameter settings Check whether predictions of your model conform with reality Communicate your results…

28. Modelling the evolution of language for modellers and non-modellers IJCAI-05 28 About compromises Reality is too complex to model completely –Especially true for models of systems involving humans Computing power is limited Our knowledge of the underlying systems is limited –Especially of the cognitive aspects One is forced to make compromises/tradeoffs –Identify the bottlenecks: it is no use to make one part of the system extremely realistic if other parts are not Always communicate your compromises Realism Computation

29. Modelling the evolution of language for modellers and non-modellers IJCAI-05 29 About abstractions Making abstractions is one form of increasing performance –Do not model certain aspects of the system and consider them a “black box” Abstractions are perfectly acceptable –Ensure to not abstract away the baby with the bathwater –Describe and defend your abstractions –Keep in mind how the abstractions map onto reality –Do not use the model for something you abstracted out SpeakerHearer Errorless words Noisy Speech Errorless words Not modelled

30. Modelling the evolution of language for modellers and non-modellers IJCAI-05 30 About implementation Choose a language and a programming environment in which you are comfortable –Describe your model independent from your chosen programming language –No single programming language is best Models are often on the edge of what is computationally feasible –(Evolution of) language is complex, so you want to model as much of it as possible –Optimization of implementation becomes crucial Preferably choose algorithms that are cognitively plausible –No exponential complexity –No global knowledge

31. Modelling the evolution of language for modellers and non-modellers IJCAI-05 31 Techniques Optimization Genetic Algorithms Agent-based models Mathematical analysis

32. Modelling the evolution of language for modellers and non-modellers IJCAI-05 32 Agent-based models (1) Two aspects of language/linguistics –Individual Psycholinguistics/language acquisition/speech errors… “performance”, “parole” –Collective Historical linguistics, general linguistics “competence”, “langue” These aspects influence each other –Individual performance based on group conventions –Collective behaviour caused by individuals This link is difficult to investigate –Complex feedback –Non-linear behaviour (influences are not separable) –Difficult to understand with “pen-and-paper”

33. Modelling the evolution of language for modellers and non-modellers IJCAI-05 33 Agent-based models (2) Computer simulations have no problems with complex systems –Ideal to investigate interaction of individual and collective levels Model a population of individuals –Individual: learning behavior, language behavior –Population: Interactions, population dynamics “Langue” Speaker

34. Modelling the evolution of language for modellers and non-modellers IJCAI-05 34 Architecture of an individual … Language behaviour PerceptionProduction Language Learning Social behaviour speech chromosomes  Age

35. Modelling the evolution of language for modellers and non-modellers IJCAI-05 35 The population Agent Remove from population Add to population Language interaction Agent mating Spatial structure Agent Social interaction

36. Modelling the evolution of language for modellers and non-modellers IJCAI-05 36 Agent-based paradigms Iterated learning model (Edinburgh) –Vertical transmission (Transfer over generations) –Small populations Language game model (Brussels) –Horizontal transmission (Transfer within generations) –Larger population Sometimes the differences are accentuated, but we would like to stress the similarities Population size HorizontalVertical transmission Small Large Language Games Iterated Learning

37. Modelling the evolution of language for modellers and non-modellers IJCAI-05 37 On measuring performance Important to define measures of performance –Optimization needs quality function –GA needs a fitness function –Agent-based models need to be monitored The whole model contains too much data (too many degrees of freedom) –Especially true in agent-based models Large complex systems can sometimes be described by a smaller number of parameters –Compare statistical mechanics: properties of a gas are temperature, pressure and specific gravity

38. Modelling the evolution of language for modellers and non-modellers IJCAI-05 38 Measures Examples –“Energy” (Liljencrants and Lindblom model) –Communicative success –Number of words –Coherence in the population –Productivity of a grammar –… Important to clearly define and describe your measures Important to explain how the measures map from the simulation onto real language –Measures are abstractions from an already abstract model

39. Modelling the evolution of language for modellers and non-modellers IJCAI-05 39 About mathematical analysis Mathematical analysis can also be used to gain understanding of complex linguistic phenomena –Has been used successfully in a number of cases (i.e. Nowak et al.) –Comparable to mathematical biology –Is considered more exact than computational models: insight in the why, not just the how In order to do mathematical analysis, models must be made even more simple and abstract –Complex, non-linear models are often not solvable Complementary to computational models –Perhaps design models with analysis in mind –Use analysis to gain deeper understanding of model’s dynamics

40. Modelling the evolution of language for modellers and non-modellers IJCAI-05 40 What have we seen? Steps in modeling –Design an abstract model of a linguistic phenomenon –Specify mappings from the abstraction to reality –Choose a technique for implementing your model –Make decisions about computational simplifications –Design measures on your system –Do mathematical analysis Techniques for modeling –Optimization –Genetic Algorithms –Agent-based models (Language Games and Iterated Learning) –(mathematical analysis)