1. Discourse & Dialogue: Introduction Ling 575 A Topics in NLP March 30, 2011

2. 2 Roadmap Definition(s) of Discourse Different Types of Discourse Goals, Modalities Topics, Tasks in Discourse & Dialogue Course structure Overview of Theoretical Approaches Points of Agreement Points of Variance Dialogue Models and Challenges Issues and Examples in Practice Spoken dialogue systems

3. 3 What is a Discourse? Discourse is: Extended span of text

4. 4 What is a Discourse? Discourse is: Extended span of text Spoken or Written

5. 5 What is a Discourse? Discourse is: Extended span of text Spoken or Written One or more participants

6. 6 What is a Discourse? Discourse is: Extended span of text Spoken or Written One or more participants Language in Use

7. 7 What is a Discourse? Discourse is: Extended span of text Spoken or Written One or more participants Language in Use Expresses goals of participants Processes to produce and interpret

8. 8 Why Discourse? Understanding depends on context Referring expressions: it, that, the screen Word sense: plant Intention: Do you have the time?

9. 9 Why Discourse? Understanding depends on context Referring expressions: it, that, the screen Word sense: plant Intention: Do you have the time? Applications: Discourse in NLP Question-Answering Information Retrieval Summarization Spoken Dialogue Automatic Essay Grading

10. 10 Different Parameters of Discourse Number of participants Multiple participants -> Dialogue

11. 11 Different Parameters of Discourse Number of participants Multiple participants -> Dialogue Modality Spoken vs Written

12. 12 Different Parameters of Discourse Number of participants Multiple participants -> Dialogue Modality Spoken vs Written Goals Transactional (message passing) Interactional (relations, attitudes) Task-oriented

13. 13 Spoken vs Written Discourse Speech Written text

14. 14 Spoken vs Written Discourse Speech Written text No paralinguistic effects “Permanent” Off-line. Edited, Crafted More “structured” Full sentences Complex sentences Subject-Predicate Complex modification More structural markers No disfluencies

15. 15 Spoken vs Written Discourse Speech Paralinguistic effects Intonation, gaze, gesture Transitory Real-time, on-line Less “structured” Fragments Simple, Active, Declarative Topic-Comment Non-verbal referents Disfluencies Self-repairs False Starts Pauses Written text No paralinguistic effects “Permanent” Off-line. Edited, Crafted More “structured” Full sentences Complex sentences Subject-Predicate Complex modification More structural markers No disfluencies

16. Major Topics & Tasks Reference: Resolution, Generation, Information Structure

17. Major Topics & Tasks Reference: Resolution, Generation, Information Structure Intention Recognition

18. Major Topics & Tasks Reference: Resolution, Generation, Information Structure Intention Recognition Discourse Structure Segmentation, Relations

19. Major Topics & Tasks Reference: Resolution, Generation, Information Structure Intention Recognition Discourse Structure Segmentation, Relations Fundamental components: How do they interact with dimensions of discourse? # Participants, Spoken vs Written,..

20. Dialogue Systems Components Dialogue Management Evaluation Turn-taking Politeness Stylistics

21. Course Structure Discussion-oriented course:

22. Course Structure Discussion-oriented course: Class participation

23. Course Structure Discussion-oriented course: Class participation Presentations Topic survey

24. Course Structure Discussion-oriented course: Class participation Presentations Topic survey Project: Proposal Progress Final report

25. Course Perspectives Foundational: Linguistic view: Understanding basic discourse phenomena Analyzing language use in context

26. Course Perspectives Foundational: Linguistic view: Understanding basic discourse phenomena Analyzing language use in context Practical/Implementational: Computational view: Developing systems and algorithms for discourse tasks

27. Course Projects Reflect linguistic and/or computational perspectives

28. Course Projects Reflect linguistic and/or computational perspectives Option 1: Analytic (Required for Ling elective credit) In-depth analysis of linguistic discourse phenomena Reflect understanding of literature Analyze real data ~15 page term paper

29. Course Projects Reflect linguistic and/or computational perspectives Option 1: Analytic (Required for Ling elective credit) In-depth analysis of linguistic discourse phenomena Reflect understanding of literature Analyze real data ~15 page term paper Option 2: Implementational Implement, extend algorithms for discourse/dialogue tasks Shorter write-up of approach, evaluation

30. 30 U: Where is A Bug’s Life playing in Summit? S: A Bug’s Life is playing at the Summit theater. U: When is it playing there? S: It’s playing at 2pm, 5pm, and 8pm. U: I’d like 1 adult and 2 children for the first show. How much would that cost? Reference & Knowledge Knowledge sources: From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

31. 31 U: Where is A Bug’s Life playing in Summit? S: A Bug’s Life is playing at the Summit theater. U: When is it playing there? S: It’s playing at 2pm, 5pm, and 8pm. U: I’d like 1 adult and 2 children for the first show. How much would that cost? Reference & Knowledge Knowledge sources: Domain knowledge From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

32. 32 U: Where is A Bug’s Life playing in Summit? S: A Bug’s Life is playing at the Summit theater. U: When is it playing there? S: It’s playing at 2pm, 5pm, and 8pm. U: I’d like 1 adult and 2 children for the first show. How much would that cost? Reference & Knowledge Knowledge sources: Domain knowledge Discourse knowledge From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

33. 33 U: Where is A Bug’s Life playing in Summit? S: A Bug’s Life is playing at the Summit theater. U: When is it playing there? S: It’s playing at 2pm, 5pm, and 8pm. U: I’d like 1 adult and 2 children for the first show. How much would that cost? Reference &Knowledge Knowledge sources: Domain knowledge Discourse knowledge World knowledge From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

34. 34 U: What time is A Bug’s Life playing at the Summit theater? Intention Recognition From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

35. 35 U: What time is A Bug’s Life playing at the Summit theater? Intention Recognition Using keyword extraction and vector-based similarity measures: From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

36. 36 U: What time is A Bug’s Life playing at the Summit theater? Intention Recognition Using keyword extraction and vector-based similarity measures: Intention: Ask-Reference: _time From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

37. 37 U: What time is A Bug’s Life playing at the Summit theater? Intention Recognition Using keyword extraction and vector-based similarity measures: Intention: Ask-Reference: _time Movie: A Bug’s Life From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

38. 38 U: What time is A Bug’s Life playing at the Summit theater? Intention Recognition Using keyword extraction and vector-based similarity measures: Intention: Ask-Reference: _time Movie: A Bug’s Life Theater: the Summit quadplex From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

39. 39 Computational Models of Discourse 1) Hobbs (1985): Discourse coherence based on small number of recursively applied relations

40. 40 Computational Models of Discourse 1) Hobbs (1985): Discourse coherence based on small number of recursively applied relations 2) Grosz & Sidner (1986): Attention (Focus), Intention (Goals), and Structure (Linguistic) of Discourse

41. 41 Computational Models of Discourse 1) Hobbs (1985): Discourse coherence based on small number of recursively applied relations 2) Grosz & Sidner (1986): Attention (Focus), Intention (Goals), and Structure (Linguistic) of Discourse 3) Mann & Thompson (1987): Rhetorical Structure Theory: Hierarchical organization of text spans (nucleus/satellite) based on small set of rhetorical relations

42. 42 Computational Models of Discourse 1) Hobbs (1985): Discourse coherence based on small number of recursively applied relations 2) Grosz & Sidner (1986): Attention (Focus), Intention (Goals), and Structure (Linguistic) of Discourse 3) Mann & Thompson (1987): Rhetorical Structure Theory: Hierarchical organization of text spans (nucleus/satellite) based on small set of rhetorical relations 4) McKeown (1985): Hierarchical organization of schemata

43. 43 Computational Models of Discourse 1) Hobbs (1985): Discourse coherence based on small number of recursively applied relations 2) Grosz & Sidner (1986): Attention (Focus), Intention (Goals), and Structure (Linguistic) of Discourse 3) Mann & Thompson (1987): Rhetorical Structure Theory: Hierarchical organization of text spans (nucleus/satellite) based on small set of rhetorical relations 4) McKeown (1985): Hierarchical organization of schemata

44. 44 Discourse Models: Common Features Hierarchical, Sequential structure applied to subunits Discourse “segments” Need to detect, interpret

45. 45 Discourse Models: Common Features Hierarchical, Sequential structure applied to subunits Discourse “segments” Need to detect, interpret Referring expressions provide coherence Explain and link

46. 46 Discourse Models: Common Features Hierarchical, Sequential structure applied to subunits Discourse “segments” Need to detect, interpret Referring expressions provide coherence Explain and link Meaning of discourse more than that of component utterances

47. 47 Discourse Models: Common Features Hierarchical, Sequential structure applied to subunits Discourse “segments” Need to detect, interpret Referring expressions provide coherence Explain and link Meaning of discourse more than that of component utterances Meaning of units depends on context

48. 48 Theoretical Differences Informational ( Hobbs/RST) Meaning and coherence/reference based on inference/abduction Versus

49. 49 Theoretical Differences Informational ( Hobbs/RST) Meaning and coherence/reference based on inference/abduction Versus Intentional (G&S) Meaning based on (collaborative) planning and goal recognition, coherence based on focus of attention

50. 50 Theoretical Differences Informational ( Hobbs/RST) Meaning and coherence/reference based on inference/abduction Versus Intentional (G&S) Meaning based on (collaborative) planning and goal recognition, coherence based on focus of attention “Syntax” of dialog act sequences versus

51. 51 Theoretical Differences Informational ( Hobbs/RST) Meaning and coherence/reference based on inference/abduction Versus Intentional (G&S) Meaning based on (collaborative) planning and goal recognition, coherence based on focus of attention “Syntax” of dialog act sequences versus Rational, plan-based interaction

52. 52 Challenges Relations: What type: Text, Rhetorical, Informational, Intention, Speech Act? How many? What level of abstraction?

53. 53 Challenges Relations: What type: Text, Rhetorical, Informational, Intention, Speech Act? How many? What level of abstraction? Are discourse segments psychologically real or just useful? How can they de recognized/generated automatically?

54. 54 Challenges Relations: What type: Text, Rhetorical, Informational, Intention, Speech Act? How many? What level of abstraction? Are discourse segments psychologically real or just useful? How can they de recognized/generated automatically? How do you define and represent “context”? How does representation interact with ambiguity resolution (sense/reference)

55. 55 Challenges Relations: What type: Text, Rhetorical, Informational, Intention, Speech Act? How many? What level of abstraction? Are discourse segments psychologically real or just useful? How can they de recognized/generated automatically? How do you define and represent “context”? How does representation interact with ambiguity resolution (sense/reference) How do you identify topic, reference, and focus?

56. 56 Challenges Relations: What type: Text, Rhetorical, Informational, Intention, Speech Act? How many? What level of abstraction? Are discourse segments psychologically real or just useful? How can they de recognized/generated automatically? How do you define and represent “context”? How does representation interact with ambiguity resolution (sense/reference) How do you identify topic, reference, and focus? Identifying relations without cues? Discourse and domain structures

57. 57 Challenges Relations: What type: Text, Rhetorical, Informational, Intention, Speech Act? How many? What level of abstraction? Are discourse segments psychologically real or just useful? How can they de recognized/generated automatically? How do you define and represent “context”? How does representation interact with ambiguity resolution (sense/reference) How do you identify topic, reference, and focus? Identifying relations without cues? Computational complexity of planning/plan recognition Discourse and domain structures

58. 58 Dialogue Modeling Two or more participants – spoken or text Often focus on task-oriented collaborative dialogue

59. 59 Dialogue Modeling Two or more participants – spoken or text Often focus on task-oriented collaborative dialogue Models: Dialogue Grammars: Sequential, hierarchical constraints on dialogue states with speech acts as terminals Small finite set of dialogue acts, often “adjacency pairs” Question/response, check/confirm

60. 60 Dialogue Modeling Two or more participants – spoken or text Often focus on task-oriented collaborative dialogue Models: Dialogue Grammars: Sequential, hierarchical constraints on dialogue states with speech acts as terminals Small finite set of dialogue acts, often “adjacency pairs” Question/response, check/confirm Plan-based Models: Dialogue as special case of rational interaction, model partner goals, plans, actions to extend

61. 61 Dialogue Modeling Two or more participants – spoken or text Often focus on task-oriented collaborative dialogue Models: Dialogue Grammars: Sequential, hierarchical constraints on dialogue states with speech acts as terminals Small finite set of dialogue acts, often “adjacency pairs” Question/response, check/confirm Plan-based Models: Dialogue as special case of rational interaction, model partner goals, plans, actions to extend Multi-layer Models: Incorporate high-level domain plan, discourse plan, adjacency pairs

62. 62 Dialogue Modeling Challenges How rigidly do speakers adhere to dialogue grammars? How many acts? Which ones? How can we recognize these acts? Pairs? Larger structures?

63. 63 Dialogue Modeling Challenges How rigidly do speakers adhere to dialogue grammars? How many acts? Which ones? How can we recognize these acts? Pairs? Larger structures? Mental models How do we model the beliefs and knowledge state of speakers?

64. 64 Dialogue Modeling Challenges How rigidly do speakers adhere to dialogue grammars? How many acts? Which ones? How can we recognize these acts? Pairs? Larger structures? Mental models How do we model the beliefs and knowledge state of speakers? Discourse and domain structures

65. 65 Practical Considerations Full reference resolution, interpretation:

66. 66 Practical Considerations Full reference resolution, planning: Worst case NP-complete, AI-complete

67. 67 Practical Considerations Full reference resolution, planning: Worst case NP-complete, AI-complete Systems must be (close to) real-time

68. 68 Practical Considerations Full reference resolution, planning: Worst case NP- complete, AI-complete Systems must be (close to) real-time Complex models of reference -> Interaction history Often stack-based recency of mention

69. 69 Practical Considerations Full reference resolution, planning: Worst case NP-complete, AI-complete Systems must be (close to) real-time Complex models of reference -> Interaction history Often stack-based recency of mention Planning/Inference -> state-based interaction model

70. 70 Practical Considerations Full reference resolution, planning: Worst case NP-complete, AI-complete Systems must be (close to) real-time Complex models of reference -> Interaction history Often stack-based recency of mention Planning/Inference -> state-based interaction model Questions: Initiative (system/user driven?) Corpus collection Evaluation

71. 71 Spoken Dialogue Modeling Building interactive spoken language systems Based on speech recognition and (often) synthesis

72. 72 Spoken Dialogue Modeling Building interactive spoken language systems Based on speech recognition and (often) synthesis Dominated by practical considerations Limitations of: speech recognizer accuracy, speed, coverage; speech synthesizer speed, fluency, naturalness; plan/intention recognition and reasoning speech and effectiveness

73. 73 Spoken Dialogue Modeling Building interactive spoken language systems Based on speech recognition and (often) synthesis Dominated by practical considerations Limitations of: speech recognizer accuracy, speed, coverage; speech synthesizer speed, fluency, naturalness; plan/intention recognition and reasoning speech and effectiveness Often simplistic but implementable models

74. 74 Spoken Dialogue Modeling Building interactive spoken language systems Based on speech recognition and (often) synthesis Dominated by practical considerations Limitations of: speech recognizer accuracy, speed, coverage; speech synthesizer speed, fluency, naturalness; plan/intention recognition and reasoning speech and effectiveness Often simplistic but implementable models Design and evaluation challenges What is the best dialogue? Fastest? Fewest errors? Most “natural”?

75. 75 Dialogue Management: Motivating Examples Dialogue 1: S: Would you like movie showtime or theater playlist information? U: Movie showtime. S: What movie do you want showtime information about? U: Saving Private Ryan. S: At what theater do you want to see Saving Private Ryan? U: Paramount theater. S: Saving Private Ryan is not playing at the Paramount theater. From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

76. 76 DM: Motivating Examples (Cont’d) Dialogue 2: S: How may I help you? U: When is Saving Private Ryan playing? S: For what theater? U: The Paramount theater. S: Saving Private Ryan is not playing at the Paramount theater, but it’s playing at the Madison theater at 3:00, 5:30, 8:00, and 10:30. From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

77. 77 DM: Motivating Examples (Cont’d) Dialogue 3: S: How may I help you? U: When is Saving Private Ryan playing? S: For what theater? U: The Paramount theater. S: Saving Private Ryan is playing at the Fairmont theater at 6:00 and 8:30. U: I wanted to know about the Paramount theater, not the Fairmont theater. S: Saving Private Ryan is not playing at the Paramount theater, but it’s playing at the Madison theater at 3:00, 5:30, 8:00, and 10:30. From Carpenter and Chu-Carroll, Tutorial on Spoken Dialogue Systems, ACL ‘99

78. 78 Comparison of Sample Dialogues Dialogue 1: System-initiative Implicit confirmation Merely informs user of failed query Mechanical Least efficient Dialogue 2: Mixed-initiative No confirmation Suggests alternative when query fails More natural Most efficient Dialogue 3: –Mixed-initiative –No confirmation –Suggests alternative when query fails –More natural –Moderately efficient

79. 79 Comparison of Sample Dialogues Dialogue 1: System-initiative Dialogue 2: Mixed-initiative Dialogue 3: –Mixed-initiative

80. 80 Comparison of Sample Dialogues Dialogue 1: System-initiative Implicit confirmation Dialogue 2: Mixed-initiative No confirmation Dialogue 3: –Mixed-initiative –No confirmation

81. 81 Comparison of Sample Dialogues Dialogue 1: System-initiative Implicit confirmation Merely informs user of failed query Dialogue 2: Mixed-initiative No confirmation Suggests alternative when query fails Dialogue 3: –Mixed-initiative –No confirmation –Suggests alternative when query fails

82. 82 Comparison of Sample Dialogues Dialogue 1: System-initiative Implicit confirmation Merely informs user of failed query Mechanical Least efficient Dialogue 2: Mixed-initiative No confirmation Suggests alternative when query fails More natural Most efficient Dialogue 3: –Mixed-initiative –No confirmation –Suggests alternative when query fails –More natural –Moderately efficient

83. 83 Dialogue Evaluation System-initiative, explicit confirmation better task success rate lower WER longer dialogues fwer recovery subdialogues less natural Mixed-initiative, no confirmation lower task success rate higher WER shorter dialogues more recovery subdialogues more natural Candidate measures from Chu-Carroll and Carpenter

84. 84 Dialogue System Evaluation Black box: Task accuracy wrt solution key Simple, but glosses over many features of interaction

85. 85 Dialogue System Evaluation Black box: Task accuracy wrt solution key Simple, but glosses over many features of interaction Glass box: Component-level evaluation: E.g. Word/Concept Accuracy, Task success, Turns-to- complete More comprehensive, but Independence? Generalization?

86. 86 Dialogue System Evaluation Black box: Task accuracy wrt solution key Simple, but glosses over many features of interaction Glass box: Component-level evaluation: E.g. Word/Concept Accuracy, Task success, Turns-to-complete More comprehensive, but Independence? Generalization? Performance function: PARADISE[Walker et al]: Incorporates user satisfaction surveys, glass box metrics Linear regression: relate user satisfaction, completion costs

87. 87 Controls flow of dialogue –Openings, Closings, Politeness, Clarification,Initiative –Link interface to backend systems Mechanisms: increasing flexibility, complexity –Finite-state –Template-based –Learning-based Acquisition –Hand-coding, probabilistic dialogue grammars, automata, HMMs Dialogue Management

88. 88 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty How do we integrate different information sources? Task plans and discourse plans Multi-modal cues: Multi-scale syntax, semantics, cue words, intonation, gaze, gesture How can we learn? Cues to discourse structure Dialogue strategies, models

89. 89 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty How do we integrate different information sources? Task plans and discourse plans Multi-modal cues: Multi-scale syntax, semantics, cue words, intonation, gaze, gesture How can we learn? Cues to discourse structure Dialogue strategies, models

90. 90 Broad Challenges How should we represent discourse?

91. 91 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty

92. 92 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty How do we integrate different information sources? Task plans and discourse plans Multi-modal cues: Multi-scale syntax, semantics, cue words, intonation, gaze, gesture How can we learn? Cues to discourse structure Dialogue strategies, models

93. 93 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty How do we integrate different information sources?

94. 94 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty How do we integrate different information sources? Task plans and discourse plans Multi-modal cues: Multi-scale syntax, semantics, cue words, intonation, gaze, gesture How can we learn?

95. 95 Broad Challenges How should we represent discourse? One general model? Fundamentally different? Text/Speech; Monologue/Multiparty How do we integrate different information sources? Task plans and discourse plans Multi-modal cues: Multi-scale syntax, semantics, cue words, intonation, gaze, gesture How can we learn? Cues to discourse structure Dialogue strategies, models

96. 96 Relation Recognition: Intention (Redux) Goals: Match utterance with 1+ dialogue acts, capture information Sample dialogue actions: –Maptask Acknowledgement Instruction/Explanation/Clarification Alignment/Check Question Yes-No/Other Question Affirmative/Negative Reply Other Reply Ready Unidentifiable

97. 97 Relation Recognition: Intention Knowledge sources: Overall dialogue goals Orthographic features, e.g.: punctuation cue words/phrases: “but”, “furthermore”, “so” transcribed words: “would you please”, “I want to” Dialogue history, i.e., previous dialogue act types Dialogue structure, e.g.: subdialogue boundaries, dialogue games dialogue topic changes Prosodic features of utterance: duration, pause, F0, speaking rate Empirical methods/ Manual rule construction: Probabilistic dialogue act classifiers: HMMs Rule-based dialogue act recognition: CART, Transformation-based learning

98. 98 Corpus Collection How would someone accomplish task? What would they say? Sample interaction collection: Wizard-of-Oz: Simulate all or part of a system Subjects interact Provides data for modeling, training, etc