1. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6801 עיבוד שפות טבעיות - שיעור שמונה Context Free Grammars and Parsing עידו דגן המחלקה למדעי המחשב אוניברסיטת בר אילן

2. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6802 Problems with FS grammars Reasons why finite-state grammars may be inadequate: Cannot represent constituency adequately Cannot represent structural ambiguity Cannot deal with recursion Recursion occurs when an expansion of a non- terminal includes the non-terminal itself, eg.Nominal Nominal PP

3. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6803

4. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6804 Context-free grammars A context-free grammar consists of –a set of rules or productions –a lexicon corresponds to lexical rules POS  Word

5. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6805 A baby CF grammar for NPs NP  Det Nominal NP  ProperNoun NP  Pronoun Nominal  Noun Nominal  Noun Nominal Det  a Det  the Pronoun  I Noun  flight

6. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6806 Context-free rules 2 classes of symbols: terminal and non-terminal symbols Each context-free rule is of the form: A  γ a single non-terminal symbol  an ordered list of one or more terminals and non-terminals

7. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6807 Context-free grammars CFGs can be used for analyzing or for generating „  “ means: rewrite the symbol on the left with the string of symbols on the right Example: NP  Det Nominal  Det Noun  a Noun  a flight

8. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6808 Context-free grammars A sequence of rule expansions is called the derivation of a string of words. Formally, a particular CF language is a set of strings which can be derived from a particular CF grammar, starting from the start non-terminal S. A derivation is represented by a parse tree

9. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-6809

10. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68010

11. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68011 The combined baby CFG S  NP VP NP  Pronoun |Det Nominal |ProperNoun Nominal  Noun|Noun Nominal VP  Verb NP | Verb NP PP | Verb PP PP  Preposition NP Noun  flight|flights |trip |morning |... Verb  is |prefer |like|need |want |fly |... Adjective  cheapest|non-stop|first|direct|... Pronoun  me|I |you|it|... ProperNoun  Los Angeles |Chicago |Alaska |... Det  a |the|an |this | these|that|... Preposition  from|to |on |near|... Conjunction  and |or|but|...

12. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68012

13. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68013 Formal definition of a CFG A context-free grammar is a 4-tuple: –A set of non-terminal symbols N –A set of terminal symbols Σ (disjoint from N) –A set of productions P, each of the form A  α, where A is a non-terminal and α is a string of symbols from the infinite set of strings (Σ  N)* –A designated start symbol S

14. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68014 CF language If A  β is a production of P and α and γ are any strings in the set (∑  N)*,we say that αAγ directly derives α β γ, or αAγ  α β γ Derivation is a generalization of direct derivation –Let α1, α2,..., αm be strings in (∑  N)*, m  1, such that α1  α2, α2  α3,...,  αm –we say that α1 derives αm, or α1  * αm

15. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68015 CF language The language L(G) generated by a grammar G is the set of strings composed of terminal symbols which can be derived from the designed start symbol S: L(G)= {W| w is in ∑ * and S  * w}

16. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68016 Grammar equivalence Two grammars are strongly equivalent if they generate the same set of strings and if they assign the same phrase structure to each sentence Two grammars are weakly equivalent if they generate the same set of strings but do not assign the same phrase structure to each sentence

17. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68017 Chomsky normal form A CFG is in Chomsky normal form –if it is ε-free –if each production is either of the form A  BC or A  α –Any CFG can be converted into a weakly- equivalent Chomsky normal form grammar –Example: A  BCD eq A  BX, X  CD

18. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68018 Sentence Level Constructions Declarative Sentences –I want a flight from Boston to Chicago Imperative Sentences –Show me all flights available Yes-No Questions –Do any of these flights have stops? Wh-Questions –What Airlines fly from Boston to Chicago?

19. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68019 Declarative Structure Subject NP followed by VP Statements or assertions Examples: –The flight should leave at eleven a.m. –I need a flight to Seattle. S  NP VP

20. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68020 Imperative Structure Begin with a VP and have no subject Commands Examples: –Show me the lowest fare. –List all flights from Dallas to Denver. S  VP

21. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68021 Yes-No-Question Structure Auxiliary verb, subject NP, VP Questions Examples: –Does this flight have stops? S  Aux NP VP

22. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68022 Wh-Question Structure Contain a wh-phrase constituent that includes a wh-word (who, what, when, where, why, which, how) Questions Two classes: –Wh-subject-question –Wh-non-subject-question

23. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68023 Wh-Subject-Question Wh-phrase is the subject of the sentence Same as declarative structure except the first NP contains some wh-word Examples: –What airlines fly from Dallas to Denver? –Which flights serve breakfast? S  Wh-NP VP

24. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68024 Noun Phrase (NP) Components of an NP: –Head: central noun –Pre-nominal (pre-head) modifiers –Post-nominal (post-head) modifiers

25. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68025 Pre-head Modifiers Determiners –a stop, the flights, this fare, those flights Determiners can be omitted –Mass nouns: "Water is precious" Pre-determiners –Appear before the determiners –Number or amount: "all the flights"

26. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68026 Pre-head Modifiers Others that can appear between the determiner and the head noun: Cardinal numbers: one stop, two friends Ordinal numbers: the first child, the last day Quantifiers: many stops, several friends Adjective phrase (AP): a nonstop flight, the least expensive fare NP  (Det) (Card) (Ord) (Quant) (AP) Nominal

27. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68027 Post-head Modifiers Three kinds: –Prepositional phrases –Non-finite clauses –Relative clauses

28. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68028 Post-head Modifiers Prepositional phrases (PP) Examples: –any stopovers [ for Delta seven fifty one ] –all flights [ from Dallas ] [ to Denver ] –arrival [ in Dallas ] [ before seven p.m. ] Nominal  Nominal PP (PP) (PP)

29. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68029 Post-head Modifiers Non-finite clauses –-ed the aircraft used by this flight –infinitive The last flight to arrive in Dallas –-ing (gerundive) flights arriving after eleven a.m. those leaving on Monday

30. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68030 Post-head Modifiers To add gerundive non-finite clauses: Nominal  NominalGerundVP GerundVP  GerundV GerundVP  GerundVNP GerundVP  GerundVPP GerundVP  GerundVNP PP GerundV  arriving | leaving | …

31. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68031 Post-head Modifiers Relative clauses –A clause that begins with a relative pronoun (that, who) –Examples: a flight that serves breakfast flights that leave in the morning –Nominal  Nominal RelClause –RelClause  who VP –RelClause  that VP

32. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68032 Coordination NPs, VPs, and sentences can be joined with conjunctions (and, or) NP  NP and NP VP  VP and VP S  S and S Examples: –Please repeat [NP[NP the flights] and [NP the costs]] –What flights do you have [VP[VP leaving Dallas] and [VP arriving in Denver]] –[S[S I’m interested in a flight to Dallas] and [S I’m also interested in going to Baltimore]]

33. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68033 Problems with context-free grammars Agreement: Subject noun and verb have to agree in person and number –What flights leave in the morning? –What flight leaves in the morning? –But simple CFGs over generate: *What flights leaves in the morning? *What flight leave in the morning?

34. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68034 Problems with context-free grammars Agreement contd. Possible rules which account for agreement: –S  3sgNP 3sgVP –S  Non3sgNP Non3sgVP –Instead of S  NP VP Not satisfactory due to rule proliferation –Massive redundancy –Lack of linguistically significant generalizations

35. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68035 Problems with context-free grammars Subcategorization Possible complements of a verb are called the subcategorization frame of that verb –The problem disappeared. –He finds a solution. –The lady wants to fly to Paris. But again simple CFGs over generate: –*The teacher disappeared the problem. –*He finds. –*She finds to fly to Paris

36. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68036

37. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68037 CF Parsing Syntactic parsing means recognizing a sentence and assigning a structure to it CF grammars are a declarative formalism Many possible CF parsing algorithms

38. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68038 Parsing as a search problem Syntactic parsing can be viewed as a searching through the space of all possible parse trees to find the correct parse tree(s) whose root is the start symbol S and which cover exactly the input words The search space is defined by the grammar The search strategy can be –top-down (or goal-driven) –bottom-up (or data-driven)

39. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68039 Top-Down Parsing A top-down parsing algorithm starts with the root S and builds trees down to the input words

40. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68040 Bottom-up parsing A bottom-up parsing algorithm starts with the input words and builds trees up toward the root S

41. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68041 Top-down vs. Bottom-up Parsing Top-down parser: –Only generates trees that result in S –Spends much time on trees that are not consistent with the input Bottom-up parser: –Spends much time on generating trees that do not result in S –Only generates trees that are consistent with the input

42. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68042 Top-down depth-first left-to- right parser Depth-first strategy only explores one state at a time Further choices: –Which node of a tree to expand (e.g. left- most unexpanded node of the current tree) –The order in which grammar rules are applied (e.g. according to their alphabetic order in the grammar)

43. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68043 A miniature English Grammar

44. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68044 TD-DF-LR parsing “Does this flight include a meal?”

45. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68045

46. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68046

47. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68047

48. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68048 Problems with the basic parser Left-recursion: rules of the type: NP  NP PP solution: rewrite each rule of the form A  A  |  using a new symbol: A   A   A ’, A’   A’   A ’

49. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68049 Left Recursion

50. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68050 Problems with the basic parser Ambiguity: attachment ambiguity, coordination ambiguity, noun-phrase bracketing ambiguity Attachment ambiguity: –“I saw the Grand Canyon flying to New York” Coordination ambiguity: –“old men and women”

51. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68051 Ambiguity Ambiguity is a problem to all parsers Parsing requires disambiguation Number of possible parses should be reduced (potentially exponential number of parses, and exponential runtime) e.g. Show me the meal on Flight 386 from Munich to Denver(14 parses)

52. Syllabus Text Books Classes Reading Material Assignments Grades Links Forum Text Books 88-68052 Ambiguity Example: President Kennedy today pushed aside other White House business to devote all his time and attention to working on the Berlin crisis address he will deliver tomorrow night to the American people over nationwide television and radio. Solutions: return all parses or include disambiguation in the parser.