Generation of Referring Expressions: Managing Structural Ambiguities I.H. KhanG. Ritchie K. van Deemter University of Aberdeen, UK

A natural language generator should avoid generating those phrases, which are too ambiguous to understand. But, how the generator can know whether a phrase is too ambiguous or not? We use corpus-based heuristics, backed by empirical evidence, that estimate the likelihood of different readings of a phrase, and guide the generator to choose an optimal phrase from the available alternatives.

Process of generating text in natural language (e.g., English) from some non-linguistic data (Reiter & Dale, 2000) Example NLG system –Pollen Forecast: generates reports from pollen forecast data Natural Language Generation (NLG) Grass pollen levels for Tuesday have decreased from the high levels of yesterday with values of around 4 to 5 across most parts of the country. However, in South Eastern areas, pollen levels will be high with values of 6. [courtesy E. Reiter]

Referring Expression = Noun Phrase –e.g., the black cat; the black cats and dogs (etc.) A key component in most NLG systems Task of GRE: –Given a set of intended referents, compute the properties of these referents that distinguish them from distractors in a KB Generation of Referring Expressions (GRE)

GRE: An Example ObjectsProperties Object 1 (Type, Sheep), (Color, Black) Object 2 (Type, Sheep), (Color, Brown) Object 3 (Type, Sheep), (Color, Black) Object 4 (Type, Goat), (Color, Black) Object 5 (Type, Goat), (Color, Yellow) Object 6 (Type, Goat), (Color, Black) Object 7 (Type, Goat), (Color, Brown) Object 8 (Type, Cow), (Color, Black) Output: Distinguishing Description (DD) – (Black Sheep) (Black Goat) KB Input: KB, Intended Referents R Task: find properties that distinguish R from distractors

The Problem NP 1 : The black sheep and the black goats NP 2 : The black sheep and goats (Black Sheep) (Black Goat) = {Object 1,Object 3,Object 4,Object 6 } (Black Sheep) Goat = {Object 1,Object 3,Object 4,Object 5,Object 6,Object 7 } NP 1 unambiguous and long; NP 2 ambiguous and brief Question: How the generator might chose between NP 1 and NP 2 ? Linguistic ambiguities can arise when DDs are realised

Our Approach Psycholinguistic evidence –Avoidance of all ambiguity is not feasible (Abney, 1996) Avoid only distractor interpretations –An interpretation is distractor if it is more likely or almost as likely as the intended one. Question –How to make distractor interpretation precise? Our solution –Getting likelihood using word sketches (cf. Chantree et el., 2004) –Word sketches provide detailed information about word relationships, based on corpus frequencies –Relationships are grammatical

Hypothesis 1 –If Adj modifies N 1 more often than N 2, then a narrow-scope reading is likely (no matter how frequently N1 and N2 co- occur). bearded men and women handsome men and women Hypothesis 2 –If Adj does not modify N 1 more often than N 2, then a wide- scope reading is likely (no matter how frequently N1 and N2 co-occur).. old men and women tall men and trees Pattern: the Adj N 1 and N 2

Experiment 1 Please, remove the roaring lions and horses.

Experiment 1: Results Hypothesis 2 (i.e., predictions for WS reading) is confirmed Hypothesis 1 (i.e., predictions for NS reading) is not confirmed –Tendency for WS (even though results are not stat. sig.) Tentative conclusion –An intrinsic bias in favour of WS reading BUT: The use of *unusual* features may have made peoples judgements unreliable

Experiment 2 Please, remove the figure containing the young lions and horses.

Experiment 2 (cont.) Please, remove the figure containing the barking dogs and cats. Results: Both hypotheses are confirmed

Word Sketches can make reasonable predictions about how an NP would be understood. But we need more to know from generation point of view: which of the following two NPs is best? The black sheep and the black goats The black sheep and goats (Black Sheep) (Black Goat) (Black Sheep) Goat We seek the answer in next experiment

Clarity-brevity trade-off Recall the pattern: the Adj Noun 1 and Noun 2 Brief descriptions (+b) take the form –the Adj Noun 1 and Noun 2 Non-brief descriptions (-b) take the form –the Adj Noun 1 and the Adj Noun 2 (IR = WS) –the Adj Noun 1 and the Noun 2 (IR = NS) Clear descriptions (+c) –Which have no distractor interpretations Non-clear descriptions (-c) –Which have some distractor interpretations

Hypothesis 1 –(+c, +b) descriptions are preferred over (+c, -b) Hypothesis 2 –(+c, -b) descriptions are preferred over (-c, +b) Each hypothesis is tested under two conditions –C 1 : intended reading is WS –C 2 : intended reading is NS The Hypotheses (Readers Preferences)

Experiment 3: NS Case Which phrase works best to identify the filled area? 1.The barking dogs and cats 2.The barking dogs and the cats

Experiment 3: WS Case Which phrase works best to identify the filled area? 1.The young lions and the young horses 2.The young lions and horses

Experiment 3: Results Both hypotheses are confirmed: –(+c, +b) descriptions are preferred over (+c, -b) –(+c, -b) descriptions are preferred over (-c, +b) Role of length : –In WS cases preferences are very strong –In NS cases preference is not as strong as in WS cases

Summary of Empirical Evidence For the pattern the Adj Noun 1 and Noun 2 –Word Sketches can make reliable predictions –Keeping clarity the same, a brief NP is better than a longer one

Algorithm Development Main knowledge sources –WordNet (for lexicalisation) –SketchEngine (for predicting the most likely reading) Main steps 1.Choose words 2.Use these to construct description in DNF 3.Use transformations to generate alternative structures from DNF 4.Select optimal phrase

Transformation Rules Input –Logical formula in DNF Rule Base 1.(A B 1 ) (A B 2 ) A (B 1 B 2 ) 2.(X Y) (Y X) [A = Adj, B 1 =B 2 =Noun, X=Y=(Adj and/or Noun)] Output –Set of logical formulae

Select optimal phrase 1.(black sheep) (black goats) DNF 2.(black goats) (black sheep) 3.black (goats sheep) 4.black (sheep goats) Optimal (4): Adj has high collocational frequency with N 1 and N 2, so the intended (wide- scope) reading is more likely. Therefore, (4) is selected.

Conclusions GRE should deal with surface ambiguities Word sketches can make distractor interpretation precise Keeping clarity the same, brief descriptions are preferred over longer ones A GRE algorithm is sketched that balances clarity and brevity

THANK YOU