1. LING 2000 - 2006 NLP 1 Introduction to Computational Linguistics Martha Palmer April 19, 2006

2. LING 2000 - 2006 NLP 2 Natural Language Processing Machine Translation Predicate argument structures Syntactic parses Producing semantic representations Ambiguities in sentence interpretation

3. LING 2000 - 2006 NLP 3 Machine Translation One of the first applications for computers –bilingual dictionary > word-word translation Good translation requires understanding! –War and Peace, The Sound and The Fury? What can we do? Sublanguages. –technical domains, static vocabulary –Meteo in Canada, Caterpillar Tractor Manuals, Botanical descriptions, Military Messages

4. LING 2000 - 2006 NLP 4 Example translation

5. LING 2000 - 2006 NLP 5 Translation Issues: Korean to English - Word order - Dropped arguments - Lexical ambiguities - Structure vs morphology

6. LING 2000 - 2006 NLP 6 Common Thread Predicate-argument structure –Basic constituents of the sentence and how they are related to each other Constituents –John, Mary, the dog, pleasure, the store. Relations –Loves, feeds, go, to, bring

7. LING 2000 - 2006 NLP 7 Abstracting away from surface structure

8. LING 2000 - 2006 NLP 8 Transfer lexicons

9. LING 2000 - 2006 NLP 9 Machine Translation Lexical Choice- Word Sense Disambiguation Iraq lost the battle. Ilakuka centwey ciessta. [Iraq ] [battle] [lost]. John lost his computer. John-i computer-lul ilepelyessta. [John] [computer] [misplaced].

10. LING 2000 - 2006 NLP 10 Natural Language Processing Syntax –Grammars, parsers, parse trees, dependency structures Semantics –Subcategorization frames, semantic classes, ontologies, formal semantics Pragmatics –Pronouns, reference resolution, discourse models

11. LING 2000 - 2006 NLP 11 Syntactic Categories Nouns, pronouns, Proper nouns Verbs, intransitive verbs, transitive verbs, ditransitive verbs (subcategorization frames) Modifiers, Adjectives, Adverbs Prepositions Conjunctions

12. LING 2000 - 2006 NLP 12 Syntactic Parsing The cat sat on the mat. Det Noun Verb Prep Det Noun Time flies like an arrow. Noun Verb Prep Det Noun Fruit flies like a banana. Noun Noun Verb Det Noun

13. Context Free Grammar S -> NP VP NP -> det (adj) N NP -> Proper N NP -> N VP -> V, VP -> V PP VP -> V NP VP -> V NP PP, PP -> Prep NP VP -> V NP NP LING 2000 - 2006 NLP 13

14. LING 2000 - 2006 NLP 14 Parses V PP VP S NP the mat satcat on NP Prep The cat sat on the mat Det N N

15. LING 2000 - 2006 NLP 15 Parses V PP VP S NP time an arrow flies like NP Prep Time flies like an arrow. N DetN

16. LING 2000 - 2006 NLP 16 Parses VNP VP S NP flies like an N Det Time flies like an arrow. N time arrow N

17. LING 2000 - 2006 NLP 17 Features C for Case, Subjective/Objective –She visited her. P for Person agreement, (1 st, 2 nd, 3 rd ) –I like him, You like him, He likes him, N for Number agreement, Subject/Verb –He likes him, They like him. G for Gender agreement, Subject/Verb –English, reflexive pronouns He washed himself. –Romance languages, det/noun T for Tense, –auxiliaries, sentential complements, etc. –* will finished is bad

18. LING 2000 - 2006 NLP 18 Probabilistic Context Free Grammars Adding probabilities Lexicalizing the probabilities

19. LING 2000 - 2006 NLP 19 Simple Context Free Grammar in BNF S → NP VP NP → Pronoun | Noun | Det Adj Noun |NP PP PP → Prep NP V→ Verb | Aux Verb VP → V | V NP | V NP NP | V NP PP | VP PP

20. LING 2000 - 2006 NLP 20 Simple Probabilistic CFG S → NP VP NP → Pronoun [0.10] | Noun [0.20] | Det Adj Noun [0.50] |NP PP [0.20] PP → Prep NP[1.00] V→ Verb [0.33] | Aux Verb[0.67] VP → V[0.10] | V NP [0.40] | V NP NP [0.10] | V NP PP [0.20] | VP PP[0.20]

21. LING 2000 - 2006 NLP 21 Simple Probabilistic Lexicalized CFG S → NP VP NP → Pronoun [0.10] | Noun [0.20] | Det Adj Noun [0.50] |NP PP [0.20] PP → Prep NP[1.00] V→ Verb [0.33] | Aux Verb[0.67] VP → V[0.87] {sleep, cry, laugh} | V NP [0.03] | V NP NP [0.00] | V NP PP [0.00] | VP PP[0.10]

22. LING 2000 - 2006 NLP 22 Simple Probabilistic Lexicalized CFG VP → V[0.30] | V NP [0.60] {break,split,crack..} | V NP NP [0.00] | V NP PP [0.00] | VP PP[0.10] VP → V[0.10] what about | V NP [0.40] leave? | V NP NP [0.10] leave1, leave2? | V NP PP [0.20] | VP PP[0.20]

23. LING 2000 - 2006 NLP 23 Language to Logic John went to the book store.  John  store1, go(John, store1) John bought a book. buy(John,book1) John gave the book to Mary. give(John,book1,Mary) Mary put the book on the table. put(Mary,book1,table1)

24. LING 2000 - 2006 NLP 24 Semantics Same event - different sentences John broke the window with a hammer. John broke the window with the crack. The hammer broke the window. The window broke.

25. LING 2000 - 2006 NLP 25 Same event - different syntactic frames John broke the window with a hammer. SUBJ VERB OBJ MODIFIER John broke the window with the crack. SUBJ VERB OBJ MODIFIER The hammer broke the window. SUBJ VERB OBJ The window broke. SUBJ VERB

26. LING 2000 - 2006 NLP 26 Semantics -predicate arguments break(AGENT, INSTRUMENT, PATIENT) AGENT PATIENT INSTRUMENT John broke the window with a hammer. INSTRUMENT PATIENT The hammer broke the window. PATIENT The window broke. Fillmore 68 - The case for case

27. LING 2000 - 2006 NLP 27 AGENT PATIENT INSTRUMENT John broke the window with a hammer. SUBJ OBJ MODIFIER INSTRUMENT PATIENT The hammer broke the window. SUBJ OBJ PATIENT The window broke. SUBJ

28. LING 2000 - 2006 NLP 28 Canonical Representation break (Agent: animate, Instrument: tool, Patient: physical-object) Agent subj Instrument subj, with-pp Patient obj, subj

29. LING 2000 - 2006 NLP 29 Syntax/semantics interaction Parsers will produce syntactically valid parses for semantically anomalous sentences Lexical semantics can be used to rule them out

30. LING 2000 - 2006 NLP 30 Headlines Police Begin Campaign To Run Down Jaywalkers Iraqi Head Seeks Arms Teacher Strikes Idle Kids Miners Refuse To Work After Death Juvenile Court To Try Shooting Defendant