1. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca1 HPSGs How its done, Examples, Examples, Examples

2. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca2 HPSG Feature Structure Descriptions – recall for pizza’s

3. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca3 Head Driven Phrase Structure Grammars (HPSGs)

4. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca4 Head Driven Phrase Structure Grammars (HPSGs) A type hierarchy....... states what kinds of objects we claim exist (the types)... organizes the objects hierarchically into classes with shared properties (the type hierarchy)... states what general properties each kind of object has (the feature and feature value declarations).

5. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca5 Head Driven Phrase Structure Grammars (HPSGs) Pizza Descriptions and Pizza Models How many pizza models (by definition, fully resolved) satisfy this description?

6. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca6 Head Driven Phrase Structure Grammars (HPSGs) Answer: 2 {, <TOPPINGS, { <OLIVES, + >,, }>} {, <TOPPINGS, { <OLIVES, + >,, }>}

7. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca7 Head Driven Phrase Structure Grammars (HPSGs) Pizza Descriptions and Pizza Models How many pizzas-in-the-world do the pizza models correspond to? Answer: A large, constantly-changing number.

8. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca8 Head Driven Phrase Structure Grammars (HPSGs) ‘type’/‘token’ distinction applies to sentences as well

9. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca9 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

10. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca10 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

11. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca11 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

12. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca12 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

13. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca13 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

14. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca14 Head Driven Phrase Structure Grammars (HPSGs) A new theory of pizzas

15. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca15 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

16. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca16 Head Driven Phrase Structure Grammars (HPSGs) Identity Constraints (tags)

17. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca17 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

18. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca18 Head Driven Phrase Structure Grammars (HPSGs) Note

19. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca19 Head Driven Phrase Structure Grammars (HPSGs) Combining Constraints

20. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca20 Head Driven Phrase Structure Grammars (HPSGs) Why combine constraints? The pizza example illustrates how unification can be used to combine information from different sources. In our grammar, information will come from lexical entries, grammar rules, and general principles.

21. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca21 Head Driven Phrase Structure Grammars (HPSGs) Linguistic Application of Feature Structures: Making the Mnemonic Meaningful What do these CFG categories have in common? NP & VP: are both phrases N & V: are both words NP & N: are both ‘nouny’ VP & V: are both ‘verby’

22. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca22 Head Driven Phrase Structure Grammars (HPSGs) The Beginnings of our type hierarchy

23. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca23 Head Driven Phrase Structure Grammars (HPSGs) A Feature for Part of Speech

24. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca24 Head Driven Phrase Structure Grammars (HPSGs) Type Hierarchy for Parts of Speech I

25. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca25 Head Driven Phrase Structure Grammars (HPSGs) Type Hierarchy for Parts of Speech II

26. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca26 Head Driven Phrase Structure Grammars (HPSGs) A Feature for Valence

27. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca27 Head Driven Phrase Structure Grammars (HPSGs) Underspecification

28. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca28 Head Driven Phrase Structure Grammars (HPSGs) Another Valence Feature

29. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca29 Head Driven Phrase Structure Grammars (HPSGs) SPR and Verbs

30. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca30 Head Driven Phrase Structure Grammars (HPSGs) S and NP We created a monster our creation of a monster

31. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca31 Head Driven Phrase Structure Grammars (HPSGs) Type Hierarchy So Far

32. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca32 Head Driven Phrase Structure Grammars (HPSGs) Reformulating Grammar Rules 1 Which rules do these correspond to?

33. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca33 Head Driven Phrase Structure Grammars (HPSGs) Reformulating Grammar Rules 2

34. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca34 Head Driven Phrase Structure Grammars (HPSGs) Reformulating the Grammar Rules 3

35. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca35 Head Driven Phrase Structure Grammars (HPSGs) Advantages of the New Formulation Subject-verb agreement is stipulated only once (where?) Common properties of verbs with different valences are expressed by common features Parallelisms across phrase types are captured

36. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca36 Head Driven Phrase Structure Grammars (HPSGs) Disadvantages of the New Formulation We still have three head complement rules We still have two head specifier rules We only deal with three verb valences (which ones? what are some others?) The non-branching rule does not really account for any empirical work Others?

37. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca37 Head Driven Phrase Structure Grammars (HPSGs) Heads Intuitive idea: A phrase typically contains a word that determines its most essential properties, including – where it occurs in larger phrases, and – what is its internal structure This is called the head The term “head” is used both for the head word in a phrase and for all the intermediate phrases containing that word NB: Not all phrases have heads

38. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca38 Head Driven Phrase Structure Grammars (HPSGs) Formalizing the Notion of Head Expressions have a feature HEAD HEAD’s values are of type pos For HEAD values of type agr-cat, HEAD’s value also includes the feature AGR Well-formed trees are subject to the Head Feature Principle

39. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca39 Head Driven Phrase Structure Grammars (HPSGs) The Head Feature Principle Intuitive idea: Key properties of phrases are shared with their heads The HFP: In any headed phrase, the HEAD value of the mother and the head daughter must be identical. Sometimes described in terms of properties “percolating up” or “filtering down”, but this is just metaphorical talk

40. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca40 Head Driven Phrase Structure Grammars (HPSGs) A Tree is Well-Formed if … It and each subtree are licensed by a grammar rule or lexical entry All general principles (like the HFP) are satisfied. NB: Trees are part of our model of the language, so all their features have values (even though we will often be lazy and leave out the values irrelevant to our current point).

41. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca41 Head Driven Phrase Structure Grammars (HPSGs) Question: Do phrases that are not headed have HEAD features?

42. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca42 Head Driven Phrase Structure Grammars (HPSGs)

43. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca43 Head Driven Phrase Structure Grammars (HPSGs)

44. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca44 Head Driven Phrase Structure Grammars (HPSGs) Reminder: Where We Are Attempting to model English with CFG led to problems with the granularity of categories, e.g. – Need to distinguish various subtypes of verbs – Need to identify properties common to all verbs So we broke categories down into feature structures and began constructing a hierarchy of types of feature structures. This allows us to schematize rules and state crosscategorial generalizations, while still making fine distinctions.

45. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca45 Head Driven Phrase Structure Grammars (HPSGs) But it’s still not quite right… There’s still too much redundancy in the rules. The rules and features encode the same information in different ways.

46. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca46 Head Driven Phrase Structure Grammars (HPSGs) Solution: More Elaborate Valence Feature Values The rules just say that heads combine with whatever their lexical entries say they can (or must) combine with. The information about what a word can or must combine with is encoded in list-valued valence features. – The elements of the lists are themselves feature structures – The elements are “cancelled” off the lists once heads combine with their complements and specifiers.

47. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca47 Head Driven Phrase Structure Grammars (HPSGs) Complements This allows for arbitrary numbers of complements, but only applies when there is at least one. – Heads in English probably never have more than 3 or 4 complements – This doesn’t apply where Head-Complement Rule 1 would. This covers lots of cases not covered by the old Head- Complement Rules 1-3.

48. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca48 Head Driven Phrase Structure Grammars (HPSGs) Question: What would the grammar change if English had postpositions instead of prepositions?

49. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca49 Head Driven Phrase Structure Grammars (HPSGs) Specifiers - Head Specifier Rule (version 1) Combines the rules expanding S and NP. In principle also generalizes to other categories. Question: Why is SPR list-valued?

50. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca50 Head Driven Phrase Structure Grammars (HPSGs) QUESTION: Why are these rightbranching? That is, what formal property of our grammar forces the COMPS to be lower inthe tree than the SPR?

51. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca51 Head Driven Phrase Structure Grammars (HPSGs) Another Question… What determines the VAL value of phrasal nodes? ANSWER: The Valence Principle Unless the rule says otherwise, the mother’s values for the VAL features (SPR and COMPS) are identical to those of the head daughter.

52. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca52 Head Driven Phrase Structure Grammars (HPSGs) More on the Valence Principle Intuitively, the VAL features list the contextual requirements that haven’t yet been found. This way of thinking about it (like talk of “cancellation”) is bottom-up and procedural. But formally, the Valence Principle (like the rest of our grammar) is just a well-formedness constraint on trees, without inherent directionality

53. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca53 Head Driven Phrase Structure Grammars (HPSGs) So far, we have: Replaced atomic-valued VAL features with list- valued ones. Generalized Head-Complement and Head- Specifier rules, to say that heads combine with whatever their lexical entries say they should combine with. Introduced the Valence Principle to “cancel” things off the COMPS and SPR lists.

54. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca54 Head Driven Phrase Structure Grammars (HPSGs) The Parallelism between S and NP Motivation: – pairs like Chris lectured about syntax and Chris’s lecture about syntax. – both S and NP exhibit agreement The bird sings/*sing vs. The birds sing/*sings this/*these bird vs. these/*this birds So we treat NP as the saturated category of type noun and S as the saturated category of type verb.

55. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca55 Head Driven Phrase Structure Grammars (HPSGs) Question: Is there any other reason to treat V as the head of S? In standard English, sentences must have verbs. (How about non-standard English or other languages?) Verbs taking S complements can influence the form of the verb in the complement: I insist/*recall (that) you be here on time. Making V the head of S helps us state such restrictions formally

56. CSE6339 3.0 Introduction to Computational Linguistics Tuesdays, Thursdays 14:30-16:00 – South Ross 101 Fall Semester, 2011 Instructor: Nick Cercone - 3050 CSEB - nick@cse.yorku.ca56 Concluding Remarks NAÏVE Naive you are if you believe life favours those who aren't naive.