1. Lectures I. Overview 2. Simulation Semantics 3. ECG and Best-fit Analysis 4. Compositionality 5. Simulation, Counterfactuals, and Inference Constructions Simulation Utterance Discourse & Situational Context Semantic Specification : image schemas, bindings, action schemas Analyzer: incremental, competition-based, psychologically plausibleA

2. Introduction: NTL NTL’s main tenets – direct neural realization, and – continuity of thought and language both of which entail a commitment to parallel processing and spreading activation – existence of language communities conventional beliefs, grammars – simulation semantics language understanding involves some of the brain circuitry involved in perception, motion, and emotion – best-fit process underlying learning, understanding, and production of language

3. Levels in a Neural Theory of Language The Neural Observation Level: Discoveries made via experimental neuroscience. The Neural Computation Level: A hypothesized (connectionist) account of what “Neural Computation” is and how the brain uses it to function. The Formal Level: The use of a single formal notation linking the Neural Computational and Cognitive Linguistics levels. In Embodied Construction Grammar (ECG), the notation is used in standard forms of computation, both to model the functionality of various aspects of the brain and for use in automatic language analysis. The Cognitive Linguistics Level: The analysis of language and thought using ideas that fit empirical results from the cognitive and brain sciences. The Cognitive and Linguistic Observation Level: Empirical observations about language and thought.

4. Introduction: ECG Embodied Construction Grammar – part of the Construction Grammar tradition (Croft 2001, Fillmore 1998, Fried & Boas 2005) – adds embodied semantics – Designed as a tool to formally explore the NTL principles in a tractable, expressive way not the only way to formalize NTL; cannot directly describe some of its aspects (e.g., spreading activation)

5. Embodied Construction Grammar ECG (Formalizing Cognitive Linguistics) 1.Community Grammar and Core Concepts 2.Deep Grammatical Analysis 3.Computational Implementation a.Test Grammars b.Applied Projects – Question Answering 4.Map to Connectionist Models, Brain 5.Models of Grammar Acquisition

6. ECG for linguistic analysis ECG unifies insights from construction grammars and cognitive linguistics ECG is not just about representation: – A computationally precise model makes it possible to build systems for linguistic analysis and interpretation Some history: – Jurafsky (1996) first used construction grammar in a model of interpretation – Bryant (2003): robust child-language interpretation – Steels and de Beule (2006): language learning over populations – Ball (2007): psychologically plausible language interpretation

7. ECG for linguistic analysis Constructional Analyzer – fits into the unified cognitive science (Feldman 2006) – and builds on cognitive linguistics construction grammar psycholinguistics simulation-based language inference (Narayanan 1997) Natural Language Processing techniques

8. ECG for linguistic analysis Constructional Analyzer (Bryant 2008) – Input: Grammar Utterance Context Model – Output Semantic Specification, or SemSpec

9. Simplifying grammar by exploiting the understanding process Mok and Bryant, BLS 2006 Omission of arguments in Mandarin Chinese Construction grammar framework Model of language understanding Our best-fit approach

10. Mother (I) give you this (a toy). CHILDES Beijing Corpus (Tardiff, 1993; Tardiff, 1996) ma1+magei3ni3zhei4+ge mothergive2PSthis+CLS You give auntie [the peach]. Oh (go on)! You give [auntie] [that]. Productive Argument Omission (in Mandarin) 1 2 3 ni3gei3yi2 2PSgiveauntie aoni3gei3ya EMP2PSgiveEMP 4 gei3 give [I] give [you] [some peach].

11. Arguments are omitted with different probabilities All arguments omitted: 30.6% No arguments omitted: 6.1%

12. Problem: Proliferation of constructions SubjVerbObj1Obj2 ↓↓↓↓ GiverTransferRecipientTheme VerbObj1Obj2 ↓↓↓ TransferRecipientTheme … SubjVerbObj2 ↓↓↓ GiverTransferTheme SubjVerbObj1 ↓↓↓ GiverTransferRecipient

13. If the analysis process is smart, then... The grammar needs only state one construction Omission of constituents is flexibly allowed The analysis process figures out what was omitted SubjVerbObj1Obj2 ↓↓↓↓ GiverTransferRecipientTheme

14. physicslowest energy state chemistrymolecular fit biology fitness, MEU N euroeconomics vision threats, friends language errors, NTL, OT Constrained Best Fit in Nature inanimate animate society, politics framing, compromise

15. Competition-based analyzer finds the best analysis An analysis is made up of: – A constructional tree – A set of resolutions – A semantic specification The best fit has the highest combined score

16. Combined score that determines best-fit Syntactic Fit: – Constituency relations – Combine with preferences on non-local elements – Conditioned on syntactic context Antecedent Fit: – Ability to find referents in the context – Conditioned on syntactic information, feature agreement Semantic Fit: – Semantic bindings for frame roles – Frame roles’ fillers are scored

17. Analyzing ni3 gei3 yi2 (You give auntie) Syntactic Fit: – P(Theme omitted | ditransitive cxn) = 0.65 – P(Recipient omitted | ditransitive cxn) = 0.42 Two of the competing analyses: ni3gei3yi2omitted ↓↓↓↓ GiverTransferRecipientTheme ni3gei3omittedyi2 ↓↓↓↓ GiverTransferRecipientTheme ( 1-0.78)*(1-0.42)*0.65 = 0.08(1-0.78)*(1-0.65)*0.42 = 0.03

18. Using frame and lexical information to restrict type of reference Lexical Unit gei3 Giver (DNI) Recipient (DNI) Theme (DNI) The Transfer Frame Giver Recipient Theme Manner Means Place Purpose Reason Time

19. Can the omitted arg be recovered from context? Antecedent Fit: ni3gei3yi2omitted ↓↓↓↓ GiverTransferRecipientTheme ni3gei3omittedyi2 ↓↓↓↓ GiverTransferRecipientTheme Discourse & Situational Context childmother peachauntie table ?

20. How good of a theme is a peach? How about an aunt? The Transfer Frame Giver (usually animate) Recipient (usually animate) Theme (usually inanimate) ni3gei3yi2omitted ↓↓↓↓ GiverTransferRecipien t Theme ni3gei3omittedyi2 ↓↓↓↓ GiverTransferRecipientTheme Semantic Fit: ni3gei3yi2omitted ↓↓↓↓ GiverTransferRecipien t Theme

21. The argument omission patterns shown earlier can be covered with ONE construction Each construction is annotated with probabilities of omission Language-specific default probability can be learned SubjVerbObj1Obj2 ↓↓↓↓ GiverTransferRecipientTheme 0.780.420.65P(omitted|cxn):

22. Leverage processing to simplify representation The processing model is complementary to the theory of grammar By using a competition-based analysis process, we can: – Find the best-fit analysis with respect to constituency structure, context, and semantics – Eliminate the need to enumerate allowable patterns of argument omission in grammar This is currently being applied in models of language understanding and grammar learning.

23. ECG for linguistic analysis Workbench by Luca Gilardi – wraps the Constructional Analyzer – two different uses simplifies creation and revising of grammars helps testing grammars

24. ECG for linguistic analysis ECG: the notation – two basic primitives: schemas constructions – organized in subcase lattices i.e., hierarchical inheritance structures with (possibly) multiple parents – Ex.: SlidePast is a subcase of Verb, which is a subcase of Word, which in turn is a subcase of RootType (not shown)

25. ECG for linguistic analysis Workbench – single window simple! – lattices on the left – editing area in center – grammar file view on the right – top, center: input utterance

26. ECG for linguistic analysis Workbench – one adds new schemas and constructions in the central pane – they are shown automatically in the lattice representation

27. ECG for linguistic analysis ECG: the notation – we’ll see what’s needed for analyzing a simple sentence he slid – we need some notation first keyword are in bold – ECG is a Construction Grammar two poles: form and meaning – constructions: pair form and meaning – schemas represent the meaning constraint of a construction – subcase of introduces an inheritance relation in a construction or a schema – other features: role: introduces a part (or feature) in the structure evokes: an associated structure that’s neither a part nor a subcase – bindings: ECG is also a unification grammar specified by double arrows:

28. ECG for linguistic analysis ECG: the notation – the semantics of he slid TrajectorLandmark, SPG – conventional image schemas – related by inheritance SPG inherits all TL’s roles: – trajector, landmark, profiledArea MotionAlongAPath – actions involving a protagonist – the path is represented by the evoked SPG evokes introduces a new role (spg in this case) – the mover is bound to the trajector of the evoked SPG schema TrajectorLandmark roles trajector landmark profiledArea schema SPG subcase of TrajectorLandmark roles source path goal schema MotionAlongAPath subcase of Motion evokes SPG as spg constraints mover ↔ spg.trajector

29. ECG for linguistic analysis ECG: the notation – the semantics of he slid Motion – a subcase of Process – the mover and the protagonist are bound together by the double arrows i.e., the mover is the primary participant in a Motion action – the x-net role is typed (via the “:”) to be of the x-schematic type motion – @process is in external ontology x-schemas – fine-grained process structure representations e.g. walking, pushing, sliding can all be represented as x-schematic structures (Narayanan 1997) schema Process roles protagonist x-net: @process schema Motion subcase of Process roles mover: @entity speed// scale heading// place x-net: @motion // modified constraints mover ↔ protagonist

30. Schema Lattice MotorControl Motion SPG Effector Motion Effector MotionPath ForceTransfer ForceApplication Contact SpatiallyDirectedAction CauseEffect Contact Agentive Impact SelfMotion Path MotionPath

31. Verb Constructions schema ForceApplication subcase of MotorControl schema Agentive Impact subcase of ForceApplication cxn BITE meaning: ForceApplication schema MotorControl cxn GRASP meaning: ForceApplication cxn PUSH meaning: ForceApplication cxn SLAP meaning: AgentiveImpact cxn KICK meaning: AgentiveImpact cxn HIT meaning: AgentiveImpact

32. ECG for linguistic analysis ECG: the notation – the semantics of he slid Just two more schemas – EventDescriptor (or ED) the meaning of an entire scene the verbal argument structure is typically bound to the eventType role the verb’s meaning is usually bound to profiledProcess – ReferentDescriptor (or RD) typically represents constraints associated with referents of nominal and pronominal constructions schema EventDescriptor roles eventType: Process profiledProcess: Process profiledParticipant profiledState spatialSetting temporalSetting schema RD roles ontological-category givenness referent number

33. ECG for linguistic analysis ECG: the notation – the analysis of he slid Now for the constructions – pair form and meaning cname.f refers to the form pole of the construction cname cname.m refers to its meaning pole Verb – Word gives a Verb an orthographic form – HasVerbFeatures verbal agreement features (number and person) – its meaning is a Process SlidePast – a Verb with an orthographic form – and an x-schematic motor program – its meaning is MotionAlongAPath general construction Verb subcase of Word, HasVerbFeatures meaning: Process construction SlidePast subcase of Verb form constraints self.f.orth ← "slid" meaning : MotionAlongAPath constraints self.m.x-net ← @slide

34. ECG for linguistic analysis ECG: the notation – the analysis of he slid Clause-level construction – Declarative: brings together a subject (an NP constituent), – the construction for He is a subcase of NP and a finite verb phrase, fin, of type VerbPlusArguments IntransitiveArgumentStructure is a subcase of this (green marks the inherited structure) construction Declarative subcase of S-With-Subj constructional constituents subj: NP fin: VerbPlusArguments form constraints subj.f before fin.f meaning constraints subj.m.referent ↔ self.m.profiledParticipant self.m ↔ fin.ed self.m.speechAct ← "Declarative”

35. ECG for linguistic analysis ECG: the notation – the analysis of he slid NP – construction He is one of its subcases – NominalFeatures: agreement features of nominals (number, case, gender,...) – meaning: a Referent Descriptor general construction NP subcase of RootType constructional: NominalFeatures meaning: RD

36. ECG for linguistic analysis ECG: the notation – the analysis of he slid VerbPlusArguments – an ancestor of IntransitiveArgumentStrucure – also a subcase of ArgumentStructure – meaning: a Process (in green the inherited structure) general construction ArgumentStructure subcase of HasVerbFeatures meaning: Process evokes EventDescriptor as ed constraints self.m ↔ ed.eventType general construction VerbPlusArguments subcase of ArgumentStructure constructional constituents v: Verb constraints self.features ↔ v.features meaning: Process constraints v.m ↔ ed.profiledProcess evokes EventDescriptor as ed self.m ↔ ed.eventType

37. ECG for linguistic analysis ECG: the notation – the analysis of he slid SemSpec synthesis – after the best-fit process has terminated – the VerbPlusArgument construction binds the Verb’s meaning pole with the profiledProcess role of the ED bind its own meaning pole with the ED’s eventType role – the Declarative cxn binds that same ED to its meaning pole constrains the subject’s referent to be the same as its meaning pole’s profiledParticipant in the form block, simply constrains the subject to appear before the verb

38. ECG for linguistic analysis ECG: the notation – the analysis of he slid SemSpec synthesis – after the best-fit process has terminated general construction VerbPlusArguments subcase of ArgumentStructure constructional constituents v: Verb constraints self.features ↔ v.features meaning: Process constraints v.m ↔ ed.profiledProcess evokes EventDescriptor as ed self.m ↔ ed.eventType construction Declarative subcase of S-With-Subj constructional constituents subj: NP fin: VerbPlusArguments form constraints subj.f before fin.f meaning constraints subj.m.referent ↔ self.m.profiledParticipant self.m ↔ fin.ed self.m.speechAct ← "Declarative”

39. ECG for linguistic analysis ECG: the notation – the analysis of he slid SemSpec synthesis – last piece of analysis: the argument structure chosen by the best-fit process – IAS binds its meaning pole with the Verb’s – constrains the protagonist of the action to be the same as the evoked ED’s profiledParticipant together with the constraint described above for VerbPlusArguments, implies that the event described by the intransitive argument structure is the same as the one described by its verb constituent. Goldberg (1995) describes for cases in which the meaning of the verb and argument structure constructions do not unify. (inherited structure in green) construction IntransitiveArgumentStructure subcase of VerbPlusArguments constructional constituents v: Verb constraints self.features ↔ v.features self.features.verbform ← FiniteOrGerund meaning: Process constraints evokes EventDescriptor as ed self.m ↔ ed.eventType self.m.protagonist ↔ ed.profiledParticipant self.m ↔ v.m

42. ECG for psycholinguistic modeling The best-fit process in the Analyzer – inspired by cognitive science, psychology, computer science – algorithm is cognitively plausible scans and incorporates in an interpretation one word at a time can only entertain a limited number of interpretations approximates spreading activation with probabilities combines syntactic and semantic evidence to rank competing interpretations – such process is what we call the best-fit heuristic

43. ECG for psycholinguistic modeling The best-fit process in the Analyzer – best fit heuristic: cognitive motivation – psychology and psycholinguistics constraint-based (or interactionist) paradigm – [...] constraint-based models assume that multiple syntactic alternatives are evaluated using both linguistic and non-linguistic sources of constraint. The comprehension system continuously integrates all the relevant and available information in order to compute the interpretation that best satisfies those constraints. (McRae, Spivey-Knowlton, & Tannenhaus, 1998) – models that fit the constraint-based paradigm Narayanan & Jurafsky (1998) McRae et al. (1998) Pado (2007)

44. ECG for psycholinguistic modeling The best-fit process in the Analyzer – best fit heuristic: cognitive motivation Connectionist models – best-fit models that use spreading activation to combine multiple domains competition between the connectionist model’s units to model competing hypotheses – Examples: Lane & Henderson (1998): connectionist network for syntactic parsing Feldman (2006): reduction of language interpretation to connectionist models

45. ECG for psycholinguistic modeling The best-fit process in the Analyzer – best fit heuristic: cognitive motivation Construction grammars – defines grammaticality in terms of formal properties (syntax) and function (semantic and pragmatic constraints)

46. ECG for psycholinguistic modeling The best-fit process in the Analyzer – best fit heuristic: cognitive motivation Natural Language Processing (CS) – joint models of lexicalized PCFGs can be seen as best-fit models – they use lexical dependency as a proxy for direct semantic information

47. ECG for psycholinguistic modeling Analyzer: modeling reading times – the best-fit machinery has been tested with real psycholinguistic data McRae, Spivey, Tannenhaus (McRae at el., 1998) – self-paced reading paradigm with pairs of reduced relative sentences: 1.The cop arrested by the detective was guilty 2.The crook arrested by the detective was guilty – Sentences differed on whether the subject was a good agent of the p.p. (cop) or a good patient (crook) sentence 1 is initially easier at the p.p. harder at the prepositional phrase and main verb

48. ECG for psycholinguistic modeling Analyzer: modeling reading times – words presented two at a time – semantic fit affects reading time – explanation: consequence of violation of semantic expectations The cop arrested by the detective was guilty – the cop arrested is biased towards the cop doing the arresting – by the detective violates such expectation

49. ECG for psycholinguistic modeling Analyzer: modeling reading times – data from Penn TreeBank, Propbank, original data from McRea et al. to approximate constituent filler probabilities – simple grammar – 40 reduces samples from McRea et al. – 40 unreduced samples as baseline

50. ECG for psycholinguistic modeling Analyzer: modeling reading times – some discrepancies due to best-fit heuristic chosen – results qualitatively accurate nonetheless

52. NTL and ECG http://ecgweb.pbworks.com/ An introduction

53. Lectures I. Overview 2. Simulation Semantics 3. ECG and Best-fit Analysis 4. Compositionality 5. Simulation, Counterfactuals, and Inference Constructions Simulation Utterance Discourse & Situational Context Semantic Specification: image schemas, bindings, action schemas Analyzer: incremental, competition-based, psychologically plausible

54. Schema Lattice MotorControl Motion SPG Effector Motion Effector MotionPath ForceTransfer ForceApplication Contact SpatiallyDirectedAction CauseEffect Contact Agentive Impact SelfMotion Path MotionPath

55. Verb Constructions schema ForceApplication subcase of MotorControl schema Agentive Impact subcase of ForceApplication cxn BITE meaning: ForceApplication schema MotorControl cxn GRASP meaning: ForceApplication cxn PUSH meaning: ForceApplication cxn SLAP meaning: AgentiveImpact cxn KICK meaning: AgentiveImpact cxn HIT meaning: AgentiveImpact

56. Introduction: ECG More specifically, ECG serves: 1.as a technical tool for linguistic analysis 2.to specify shared grammar, conceptual conventions of a linguistic community 3.as a computer specification for implementing linguistic theories 4.as a representation for models and theories of language acquisition 5.as a front-end system for applied language- understanding tasks 6.as a high-level functional description for biological and behavioral experiments

57. Introduction: ECG NTL assumptions can lead to the formulation of questions and experiments not obvious from other perspectives To facilitate that, a precise notation is needed: ECG – Embodied Construction Grammar

58. Best-fit analysis reduces burden on the grammar representation Constructions Simulation Utterance Discourse & Situational Context Semantic Specification: image schemas, frames, action schemas Analyzer: incremental, competition-based, psycholinguistically plausible