1. Generation of Referring Expressions: Modeling Partner Effects Surabhi Gupta Advisor: Amanda Stent Department of Computer Science

2. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

3. Referring Expressions A referring expression denotes (or points to) an object in the world of a discourse. A referring expression denotes (or points to) an object in the world of a discourse. Examples of referring expressions include the red chair, the 400 dollar red chair and 5 red chairs. Examples of referring expressions include the red chair, the 400 dollar red chair and 5 red chairs. Referring expressions are usually noun phrases Referring expressions are usually noun phrases Improper construction of a referring expression can result in Improper construction of a referring expression can result in referring expressions that are ambiguous (e.g. the book when there are two books). referring expressions that are ambiguous (e.g. the book when there are two books). referring expressions that are too descriptive and lead to false implicatures (e.g. the 400 dollar chair when there is only one chair) referring expressions that are too descriptive and lead to false implicatures (e.g. the 400 dollar chair when there is only one chair)

4. Structure of a Noun Phrase A definite/indefinite noun phrase is constructed of: A definite/indefinite noun phrase is constructed of: An (optional) determiner or quantifier e.g. a, three An (optional) determiner or quantifier e.g. a, three A number of premodifiers (adjectives, adverbs, noun modifiers) e.g. red A number of premodifiers (adjectives, adverbs, noun modifiers) e.g. red A number of postmodifiers (prepositional phrases, relative clauses) e.g. worth 400 dollars, that is red A number of postmodifiers (prepositional phrases, relative clauses) e.g. worth 400 dollars, that is red Other noun phrases include pronouns, proper nouns, deictics Other noun phrases include pronouns, proper nouns, deictics green

5. Adaptation in Conversation When people talk with each other, they adapt to the other’s choice of referring expression (Clark 1996, Levinson 1983, Brennan 1987). When people talk with each other, they adapt to the other’s choice of referring expression (Clark 1996, Levinson 1983, Brennan 1987). Example: Example: (A): Lets buy the 400 dollar red chair (A): Lets buy the 400 dollar red chair (B): That’s a good idea. The chair matches with the red table. (B): That’s a good idea. The chair matches with the red table. (A): The chair it is then. (A): The chair it is then.

6. Generation of Referring Expressions in Dialog When a computer constructs human language, it is called generation When a computer constructs human language, it is called generation NewsBlaster summaries, or Google translation NewsBlaster summaries, or Google translation Generation for dialog must involve consideration of the dialog partner (the human) Generation for dialog must involve consideration of the dialog partner (the human)

7. Good Generation of Referring Expressions The algorithm should generate a referring expression for which the human reader can identify the referent. The algorithm should generate a referring expression for which the human reader can identify the referent. The algorithm should generate referring expressions that do not lead the human reader to make false implicatures (Grice 1968). The algorithm should generate referring expressions that do not lead the human reader to make false implicatures (Grice 1968). The algorithm should model how conversational partners adapt to each other. The algorithm should model how conversational partners adapt to each other. The algorithm should be able to generate the whole range of referring expressions observed in discourse. The algorithm should be able to generate the whole range of referring expressions observed in discourse. The algorithm should be computationally feasible. The algorithm should be computationally feasible.

8. Our Objective We are building a model of referring expression generation that captures adaptation to partners in conversation. We are building a model of referring expression generation that captures adaptation to partners in conversation. Related work in this field does not include partner adaptation for dialog (Dale and Reiter 1995, Siddharthan and Copestake 2004). Related work in this field does not include partner adaptation for dialog (Dale and Reiter 1995, Siddharthan and Copestake 2004).

9. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

10. Data Two corpora of spoken dialog rich in noun phrases: Two corpora of spoken dialog rich in noun phrases: Maptask - Speaker A giving instructions to Speaker B about following directions in a map Maptask - Speaker A giving instructions to Speaker B about following directions in a map Coconut - Two participants trying to buy furniture by using both of their inventories and money. Coconut - Two participants trying to buy furniture by using both of their inventories and money. For each corpus, we: For each corpus, we: Automatically extracted the noun phrases Automatically extracted the noun phrases Annotated the noun phrases by hand for referent (in a knowledge representation we built), type (noun phrase or pronoun), and to indicate whether the noun phrase was embedded in another noun phrase. Annotated the noun phrases by hand for referent (in a knowledge representation we built), type (noun phrase or pronoun), and to indicate whether the noun phrase was embedded in another noun phrase.

11. CoconutMaptask Def1162118 Indef9671411 1st person pronoun 440563 2nd person pronoun 1651275 3rd person pronoun 79614 Deictics00 Proper Nouns 00 Quantity Nouns 291160 Mass Nouns 00 No Modifiers 13113 Not Embedded 2291633 Embedded24226 Set Constructions 00 Not in KR 6121875 NPs Used 4711294 Total17675986

12. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

13. Algorithms Compared Rule Based Rule Based Dale and Reiter 1995 Dale and Reiter 1995 With partner effects (x 2) With partner effects (x 2) With postmodifier ordering (x 4) With postmodifier ordering (x 4) Siddharthan and Copestake 2004 Siddharthan and Copestake 2004 With partner effects (x 2) With partner effects (x 2) With postmodifier ordering (x 4) With postmodifier ordering (x 4) Statistical Statistical Support Vector Machines Support Vector Machines

14. Rule-Based Algorithms Terms used: Terms used: Contrast Set: contains information of all the objects in the world. Contrast Set: contains information of all the objects in the world. Preferred list of attributes: the attributes that are known for the objects. Preferred list of attributes: the attributes that are known for the objects. For Coconut: type, quantity, cost, color, state For Coconut: type, quantity, cost, color, state E.g. three green high tables worth $400 E.g. three green high tables worth $400 Intended Referent: The object from the world, which we are trying to describe. Intended Referent: The object from the world, which we are trying to describe.

15. Dale and Reiter Basic idea: Basic idea: Specify the preference list by hand Specify the preference list by hand Repeat until all members of the contrast set are gone: Repeat until all members of the contrast set are gone: Add the value for the next attribute from the preference list for the intended referent to the noun phrase to be generated Add the value for the next attribute from the preference list for the intended referent to the noun phrase to be generated

16. Example: Example: Preference list: {Type, Color, Cost, Quantity, State} Preference list: {Type, Color, Cost, Quantity, State} Contrast set: {300 dollar red couch, 200 dollar green couch, 250 dollar brown table } Contrast set: {300 dollar red couch, 200 dollar green couch, 250 dollar brown table } Intended referent: 200 dollar green couch Intended referent: 200 dollar green couch Generated NP: green couch Generated NP: green couch 300 dollar red couch 200 dollar green couch 250 dollar brown table

17. Siddharthan and Copestake Basic idea: See Dale and Reiter Basic idea: See Dale and Reiter Preference list is reordered by using synonyms and antonyms of words in each attribute Preference list is reordered by using synonyms and antonyms of words in each attribute

18. Benefits to Rule Based Algorithms They consider the way humans actually converse ie. humans use unnecessary attributes, they also begin mentioning a referring expression without scanning the entire list of distractors. They consider the way humans actually converse ie. humans use unnecessary attributes, they also begin mentioning a referring expression without scanning the entire list of distractors. They do not attempt to look for the optimal number of attributes. They just go through the list of preferred attributes and iteratively includes those attributes that rule out at least one distractor from the contrast set. They do not attempt to look for the optimal number of attributes. They just go through the list of preferred attributes and iteratively includes those attributes that rule out at least one distractor from the contrast set. There is no backtracking and the head noun is always included. There is no backtracking and the head noun is always included.

19. Disadvantages to Rule Based Algorithms They don’t generate the whole range of referring expressions They don’t generate the whole range of referring expressions Ones with postmodifiers Ones with postmodifiers Pronouns Pronouns Deictics Deictics They don’t model adaptation to partners. They don’t model adaptation to partners.

20. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

21. Adding Partner Effects A rule based algorithm A rule based algorithm Basic idea: See Dale and Reiter, Siddharthan and Copestake Basic idea: See Dale and Reiter, Siddharthan and Copestake Preference list is reordered to match selection of attributes in previous mentions of the intended referent. Preference list is reordered to match selection of attributes in previous mentions of the intended referent. Variant to this where those attributes mentioned previously are definitely included even if all the competitors have been eliminated. Variant to this where those attributes mentioned previously are definitely included even if all the competitors have been eliminated.

22. Evaluation Metric: Correct / Correct + Inserted + Deleted + Moved Metric: Correct / Correct + Inserted + Deleted + Moved Example: Example: Human: the big fat green cat Human: the big fat green cat Computer: the green happy cat Computer: the green happy cat Correct: the, cat Correct: the, cat Inserted: happy Inserted: happy Deleted: big, fat Deleted: big, fat Moved: green Moved: green Score: 2 / 6 Score: 2 / 6

23. Results The variant to our partner effects algorithms performs significantly better that our Baseline, Dale and Reiter and Siddharthan and Copestake for both the cropora used. The variant to our partner effects algorithms performs significantly better that our Baseline, Dale and Reiter and Siddharthan and Copestake for both the cropora used.

24. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

25. Discussion and Conclusions The corpus you choose makes a difference The corpus you choose makes a difference Maptask: Few distractors, no significant different between Baseline, Dale and Reiter and Siddharthan and Copestake Maptask: Few distractors, no significant different between Baseline, Dale and Reiter and Siddharthan and Copestake Do partner effects make a difference? Do partner effects make a difference?

26. References Advaith Siddharthan and Ann Copestake. 2004. Generating Referring Expressions in Open Domains. In Proceedings of the 42th Meeting of the Association for Computational Linguistics Annual Conference (ACL 2004), Barcelona, Spain. Advaith Siddharthan and Ann Copestake. 2004. Generating Referring Expressions in Open Domains. In Proceedings of the 42th Meeting of the Association for Computational Linguistics Annual Conference (ACL 2004), Barcelona, Spain. Grice, H P (1975). Logic and conversation. In P. Cole and J. Morgan, editors, Syntax and Semantics: Vol 3, Speech Acts, pages 43-58. New York: Academic Press. Grice, H P (1975). Logic and conversation. In P. Cole and J. Morgan, editors, Syntax and Semantics: Vol 3, Speech Acts, pages 43-58. New York: Academic Press. Grosz, B and Sidner, C (1986). Attention, intention, and the structure of discourse. Computational Linguistics, 12: 175-206. Grosz, B and Sidner, C (1986). Attention, intention, and the structure of discourse. Computational Linguistics, 12: 175-206. Robert Dale and Ehud Reiter. 1995. Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science, 19:233–263. Robert Dale and Ehud Reiter. 1995. Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science, 19:233–263.

27. Acknowledgements Dr. Amanda Stent, for all her time and efforts during the last three years. Dr. Amanda Stent, for all her time and efforts during the last three years. The Natural Language Processing Lab in Computer Science. The Natural Language Processing Lab in Computer Science. The Honors College for giving me the chance of working on this year long project. The Honors College for giving me the chance of working on this year long project. NSF NSF

28. Questions?

29. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

30. Generating with Postmodifiers Why? -- because previous algorithms don’t but it’s a big part of the corpus we have used. Why? -- because previous algorithms don’t but it’s a big part of the corpus we have used. Random - randomly decide whether the attribute selected should be a post modifier or premodifier Random - randomly decide whether the attribute selected should be a post modifier or premodifier Unigrams - see where the attribute is in relation to the ‘type’. Unigrams - see where the attribute is in relation to the ‘type’. Bigrams - statistics of pairs of attributes. E.g probability of finding an attribute given another. Bigrams - statistics of pairs of attributes. E.g probability of finding an attribute given another.

31. Results

32. Outline Introduction Introduction Data Data Previous work Previous work Modeling partner effects Modeling partner effects Generating NP postmodifiers Generating NP postmodifiers A little statistical experiment A little statistical experiment Discussion and Future Work Discussion and Future Work

33. Support Vector Machines SVMs are a set of machine learning algorithms for binary classification that have been applied to NLP. SVMs are a set of machine learning algorithms for binary classification that have been applied to NLP. We used a set of SVMs, one per attribute, that voted “yes” or “no” to use this attribute at this point in the noun phrase. We used a set of SVMs, one per attribute, that voted “yes” or “no” to use this attribute at this point in the noun phrase. Maptask: 6 attributes, Coconut: 5 attributes Maptask: 6 attributes, Coconut: 5 attributes We evaluated using: We evaluated using: 10-fold cross-validation for Maptask. 10-fold cross-validation for Maptask. 4-fold cross-validation for Coconut. 4-fold cross-validation for Coconut.

34. Evaluation

35. Results