Discourse & Dialogue CMSC 35900-1 October 28, 2004 What’s New? Discourse & Dialogue CMSC 35900-1 October 28, 2004

Agenda Attention and Information Given/New dichotomy Implications & Applications Given/New-based paraphrase Speech recognition & synthesis Stress and accent Gestural synthesis

Discourse So far Now, generation and synthesis Analytic models (G&S,M&T) Discourse structure recognition, segmentation Now, generation and synthesis Sentence/paragraph surface realization Grammatical forms of discourse entities – pron, def NP Sentence ordering of information – subj/not Acoustic form of entities- accented/not; accent type

Attention and Information Perspective: Focus of Attention Coherence, Reference Perspective: Information Flow Goal of discourse: Communication of information Speaker to hearer

Given/New Dichotomy Each “information unit” contains “New” information the “News” New to hearer New to discourse “Given/Old” information What is being talked about Known to hearer Already evoked in discourse

Given/New Effects Influences structure of utterance Word order Form of referring expression Prosodic prominence Guides interpretation by hearer

Given/New & Word Order Default word order (English, declarative) Left-to-right increase in “New-ness” Subject -> Given Discourse-old - present in context Predicate -> New

Given/New & Referring Expressions Hierarchy of salience Tied to Given/New status Given+Salient -> Pronoun Given, less salient -> Definite NP New -> Indefinite NP

Given/New & Prosody Prosody Unstressed -> Given, salient Pitch, Loudness, Duration, … Tone group = Information Unit Given + New Unstressed -> Given, salient Stressed -> New, less salient

Application of Information Flow Paraphrase (McKeown 1983) Natural language is ambiguous Semantic - word senses - e.g. bank Syntactic - structural E.g. prepositional phrase attachment Reference… Paraphrase makes explicit system interpretation Especially modification

Given/New Perspective Word order affected by role in sentence What speaker thinks hearer knows or not Wh-items are “new”, rest “given”, assume true Question: 3 parts 2:Lack of knowledge: wh-item with no subclause 3:Angle: Direct/Indirect modifiers of wh-items 1:Given info: Everything else

Example Q: Which active users advised by Tom Wirth work on projects in area 3? P: Assuming that there are projects in area 3, which active users work on those projects? Look for users advised by Wirth. New: lack info Work on Active users projects New: Angle Advised by TW Given In area 3

Syntax & Information Structure Link parse tree to Given/new info Root = Main verb = Inorder traversal Left subtree= Subject = Preorder Right subtree = Object = Preorder Traversal order + Part information+Transform > Linearization

Paraphrase by Given/New Advantages: Corrective response: e.g. if given info isn’t More flexible/portable that template-based paraphrase

Applications of Info Structure Speech recognition and synthesis Prosody Pitch, loudness, length New - more likely stressed; Old: often unstressed “Tunes” for given/new

Understanding Acoustic Realization Motivation Synthetic speech Experimental evidence Key components Prosody Syntax Contextually “appropriate” speech synthesis

Speech Synthesis Generally INTELLIGIBLE “Default” sentence intonation But not NATURAL Requires high attention to listen to “Default” sentence intonation May be misleading Speaking of BILL, A) JOHN thought he would WIN, but he DIDN’T B) JOHN thought he would WIN, but HE didn’t

Accent Assignment: Analysis Increased loudness, duration, pitch movement Basic view: “available”/Given: no accent; New(er): accent Attend to new information Questions: Does accent continue to decrease with repetition? How does discourse “structure” affect accent?

Accent Assignment: Results “Topic” status & First/Later mention vs De-/Accenting, form of referring expression Results: First,+Topic: Accented, Full NP Later,+Topic: De-accented , probably pronoun Later,+Topic,+Refinement: Accented (even Pron) First,-Topic: Accented Full NP Later,-Topic: Accented Full NP, Implicit Later,-Topic,+past-topic/+contrast: Accented NP (mod)

“ToBI” Intonation Framework ToBI: Tone and Break Indices Describe English sentence intonation Tones: Two pitch levels: H(igh) and L(ow) * - on stressed syllable, e.g. H*, L*, L+H* Types: Pitch accents, Phrase Tones (L-,L%) Last accent in phrase = ‘nuclear’ accent Units: Intermediate and Intonational Phrases

“ToBI” Intonation Framework Break indices Mark groupings in speech 0 - most closely linked; 5 - most disjoint 4 = Intermediate phrase boundary (-) ~ comma 5 = Intonational phrase boundary (%,$) ~ period - sentence

ToBI Examples

Contrast Examples

Contrast Examples

Contrast Examples

Syntax & Information Status Intonation units more flexible than standard syntactic constituents, e.g. subject, predicate CCG - Combinatory Categorial Grammar Allows multiple analyses (parses) to fit Link syntactic, semantic, and pragmatic/prosodic function with each unit

Generating Appropriate Intonation Basic “previous mention” strategy Accent first mention of content words De-accent closed class function words De-accent content words already mentioned Inadequate Need contrastive stress TOO

Generating Appropriate Intonation Identify theme (topic: links to previous info) Identify rheme (contributes new information) Shared propositional content Assign appropriate basic intonation contour rheme:H* L-L%; theme:L+H* L-H% (at most)

Generating Appropriate Intonation Identify focus element in theme/rheme Word to get accent Focus First mention, and Contrastive What is contrastive????

Contrastive Items: Domain For each entity x: 0: find alternatives in discourse and KB 1: RSET= x and alternatives, PROPS= features of x CSET= features of x to mark for contrast 2: For each p in PROPS, r in RSET, IF p is not property of r, add p to CSET. 3: Focus p of x E.g. She broke her left LEG, NOT her RIGHT leg.

Contrastive Items: WordNet WordNet: Semantic KB 4 parts of speech: N,V,Adj, Adv Category/word: one or more synonym sets Hierarchies linked by relations: e.g. IS-A Content Word W is new if NOT: In focus history or history’s equivalence class Equiv. Class: reachable by N hypernym/synset links Content Word W is contrastive if: In history’s contrast list Contrast: hyponyms of hypernyms of W

Examples (84) Q: I know which AMPLIFIER produces clean BASS, but WHICH amplifier produces clean TREBLE? L+H* L(H%) H* LL$ A: The BRITISH amplifier produces clean TREBLE. H* L(L%) L+H* LH$ (85) Q: I know which AMPLIFIER produces MUDDY treble, but WHICH amplifier produces CLEAN treble? L+H* L(H%) H* LL$ A: The BRITISH amplifier produces CLEAN treble. H* L(L%) L+H* LH$

Summary Assigns contextually based intonation Uses given/new information status Extended to fine-grained contrastive status Identifies contrast based on Knowledge base if available WordNet Lexical DB for greater generality

Conclusions Theme/Rheme identification difficult Contrast/Similarity measures for WordNet Still oversimplified Evaluation: How do you tell if it’s right? Many alternatives Incorporate in larger discourse structure Discourse segments, plans, ….

Examples The X4 is a SOLID-state AMPLIFIER L+H*L- H* H* L- L$ The X5 is a TUBE amplifier. L+H*L- H* L-L$ It COSTS EIGHT HUNDRED DOLLARS, H* H* H* H* L-H% IT costs NINE hundred dollars, L+H*L- H* L-H%