Centering theory and its direct applications Lecture 2

Some definitions Discourse = coherent sequence of utterances Several sentences following one another do not make a readable text Defining specific computable measures of coherence is the goal of this seminar

Centering theory ingredients Deals with local coherence What happens to the flow from sentence to sentence Does not deal with global structuring of the text (paragraphs/segments) Defines coherence as an estimate of the processing load required to “understand” the text

Processing load Upon hearing a sentence a person Cognitive effort to interpret the expressions in the utterance Integrates the meaning of the utterance with that of the previous sentence Creates some expectations on what might come next

Example John met his friend Mary today. He was surprised to see her. He thought she is still in Italy. Form of referring expressions Anaphora needs to be resolved “Create” a discourse entity at first mention with full noun phrase Creating expectations

Creating and meeting expectations (1) a. John went to his favorite music store to buy a piano. b. He had frequented the store for many years. c. He was excited that he could finally buy a piano. d. He arrived just as the store was closing for the day. (2) a. John went to his favorite music store to buy a piano. b. It was a store John had frequented for many years. d. It was closing just as John arrived.

Interpreting pronouns Terry really goofs sometimes. Yesterday was a beautiful day and he was excited about trying out his new sailboat. He wanted Tony to join him on a sailing expedition. He called him at 6am. He was sick and furious at being woken up so early.

Basic center definitions Centers of an utterance Set of entities serving to link that utterance to the other utterances in the discourse segment that contains it Not words or phrases themselves Semantic interpretations of noun phraes

Types of centers Forward looking centers An ordered set of entities What could we expect to hear about next Ordered by salience as determined by grammatical function Subject > Indirect object > Object > Others John gave the textbook to Mary. Cf = {John, Mary, textbook} Preferred center Cp The highest ranked forward looking center High expectation that the next utterance in the segment will be about Cp

Backward looking center Single backward looking center, Cb (U) For each utterance other than the segment-initial one The backward looking center of utterance Un+1 connects with one of the forward looking centers of Un Cb (U+1) is the most highly ranked element from Cf (Un) that is also realized in U+1

Centering transitions ordering Cb(Un+1)=Cb(Un) OR Cb(Un)=[?] Cb(Un+1) != Cb(Un) Cb(Un+1) = Cp (Un+1) continue smooth-shift Cb(Un+1) != Cp (Un+1) retain rough-shift

Centering constraints There is precisely one backward-looking center Cb(Un) Cb(Un+1) is the highest-ranked element of Cf(Un) that is realized in Un+1

Centering rules If some element of Cf(Un) is realized as a pronoun in Un+1 then so is Cb(Un+1) Transitions not equal continue > retain > smooth-shift > rough-shift

Centering analysis Terry really goofs sometimes. Cf={Terry}, Cb=?, undef Yesterday was a beautiful day and he was excited about trying out his new sailboat. Cf={Terry,sailboat}, Cb=Terry, continue He wanted Tony to join him in a sailing expedition. Cf={Terry, Tony, expedition}, Cb=Terry, continue He called him at 6am. Cf={Terry,Tony}, Cb=Terry, continue

Tony was sick and furious at being woken up so early. He called him at 6am. Cf={Terry,Tony}, Cb=Terry, continue Tony was sick and furious at being woken up so early. Cf={Tony}, Cb=Tony, smooth shift He told Terry to get lost and hung up. Cf={Tony,Terry}, Cb=Tony, continue Of course, Terry hadn’t intended to upset Tony. Cf={Terry,Tony}, Cb = Tony, retain

Rough shifts in evaluation of writing skills One of the graders of student essays in standardized tests is an automatic program ETS researchers have developed a number of applications that use natural language processing technologies to evaluate and score the writing abilities of test takers: The CriterionSM Online Essay Evaluation Service automatically evaluates essay responses using e-rater and the Critique writing analysis tools. E-rater® gives holistic scores for essays. CritiqueTM provides real-time feedback about grammar, usage, mechanics and style, and organization and development. C-raterTM offers automated analysis of conceptual information in short-answer, free responses.

E-rater features Syntactic variety Clear transitions Represented by features that quantify the occurrence of clause types Clear transitions Cue phrases in certain syntactic constructions Existence of main and supporting points Appropriateness of the vocabulary content of the essay What about local coherence?

Ranking forward looking centers Subject > Indirect object > Object > Others > Quantified indefinite subjects (people, everyone) > Arbitrary plural pronominals

Essay score model Human score available E-rater prediction available Percentage of rough-shifts in each essay: analysis done manually Negative correlation between the human score and the percentage of rough-shifts

Karamanis’07 Why are we reading this paper? Gives quite complete list of references on later work on centering Centering variants Reminds that entity coherence is not the only factor in text flow We’ll be discussing rhetorical structure theory during the next class Applications---can some aspects of the work be done differently/improved upon?

Information ordering task Given a set of sentences/clauses, what is the best presentation? Take a newspaper article and jumble the sentences---the result will be much more difficult to read than the original Criteria for deciding which of two orderings is better Centering would definitely be applicable Summarization, question answering, generation

Linear multi-factor regression Approximate the human score as a linear function of the e-rater prediction and the percentage of rough-shifts Adding rough shifts significantly improves the model of the score 0.5 improvement on 1—6 scale How easy/difficult would it be to fully automate the rough-shift variable

Centering variations Continuity (NOCB=lack of continuity) Coherence Cf(Un) and Cf(Un+1) share at least one element Coherence Cb(Un) = Cb(Un+1) Salience The Cb(U) = Cp(U) Coherence is more important than salience Cheapness (fulfilled expectations) Cb (Un+1) = Cp(Un)

GNOME corpus 20 descriptions of museum artifacts Split into finite unites (clauses) Semi-automatic centering annotation Item 144 is a torc. Its present arrangement, twisted into three rings, may be a modern alteration; it should probably be a single ring, worn around the neck. The terminals are in the form of goats’ heads.

Rhetorical coherence Each text can be seen as a hierarchical tree structure Different spans are related by some rhetorical relation Elaboration (adding more information) Contrast Sequence Purpose Summary etc

Local rhetorical coherence Applies only locally rather than on the text as a whole Signaled by cue phrases Contrast: but, however, on the other hand Continuation: and, then, later Consequence: because, in order to, so These local rhetorical relations structure the text When missing, entity coherence determines the flow 8 out of the 20 texts do not have any explicitly marked rhetorical relations

Joint centering and local rhetorical coherence In clauses directly marked for a rhetorical relation Merge the Cf lists of the two clauses Apply centering transitions on the resulting Cf list rather than the original GNOME-RR contains 1.58 fewer CF lists compared to the original average number (8.35)

Metrics of coherence M.NOCB (no continuity) M.CHEAP (expectations not met) M.KP sum of the violations of continuity, cheapness, coherence and salience M. BFP seeks to maximize transitions according to Rule 2

Experimental methodology Gold-standard ordering The original order of the text (object description, news article) Assume that other orderings are inferior Classification error rate Percentage orderings that score better than the gold-standard + 0.5*percentage of the orderings that score the same

Results NOCB gives best results M.BFP is the second best metric Significantly better than the other metrics M.BFP is the second best metric Adding the local rhetorical relations hurts performance---is this surprising?

Reminders Select topics you would like to present Should schedule next week now The second time you present one of the goals will be to relate the papers with previous topics we have covered Start thinking about the topic of your literature overview About 15 papers 5/6 pages Due Nov 12