Turn-taking and Disfluencies

Slides:

Advertisements

Similar presentations

Human Speech Recognition Julia Hirschberg CS4706 (thanks to John-Paul Hosum for some slides)

Advertisements

ANTLR in SSP Xingzhong Xu Hong Man Aug Outline ANTLR Abstract Syntax Tree Code Equivalence (Code Re-hosting) Future Work.

Progress update Lin Ziheng. System overview 2 Components – Connective classifier Features from Pitler and Nenkova (2009): – Connective: because – Self.

Eye Movements and Spoken Language Comprehension: effects of visual context on syntactic ambiguity resolution Spivey et al. (2002) Psych 526 Eun-Kyung Lee.

Catia Cucchiarini Quantitative assessment of second language learners’ fluency in read and spontaneous speech Radboud University Nijmegen.

Language and Cognition Colombo, June 2011 Day 8 Aphasia: disorders of comprehension.

Chapter 12 Speech Perception. Animals use sound to communicate in many ways Bird calls Bird calls Whale calls Whale calls Baboons shrieks Baboons shrieks.

5/10/20151 Evaluating Spoken Dialogue Systems Julia Hirschberg CS 4706.

® Towards Using Structural Events To Assess Non-Native Speech Lei Chen, Joel Tetreault, Xiaoming Xi Educational Testing Service (ETS) The 5th Workshop.

Automatic Prosodic Event Detection Using Acoustic, Lexical, and Syntactic Evidence Sankaranarayanan Ananthakrishnan, Shrikanth S. Narayanan IEEE 2007 Min-Hsuan.

Maurice Hermans.  Ontologies  Ontology Mapping  Research Question  String Similarities  Winkler Extension  Proposed Extension  Evaluation  Results.

Towards a model of speech production: Cognitive modeling and computational applications Michelle L. Gregory SNeRG 2003.

What can humans do when faced with ASR errors? Dan Bohus Dialogs on Dialogs Group, October 2003.

Extracting Social Meaning Identifying Interactional Style in Spoken Conversation Jurafsky et al ‘09 Presented by Laura Willson.

On the Correlation between Energy and Pitch Accent in Read English Speech Andrew Rosenberg, Julia Hirschberg Columbia University Interspeech /14/06.

On the Correlation between Energy and Pitch Accent in Read English Speech Andrew Rosenberg Weekly Speech Lab Talk 6/27/06.

Classification of Discourse Functions of Affirmative Words in Spoken Dialogue Julia Agustín Gravano, Stefan Benus, Julia Hirschberg Shira Mitchell, Ilia.

The Audio-Lingual Method

Writing on Formal and Informal Language

By Sarita Jondhale1 Pattern Comparison Techniques.

1 Computational Linguistics Ling 200 Spring 2006.

Teaching Productive Skills Which ones are they? Writing… and… Speaking They have similarities and Differences.

Speech Perception 4/4/00.

Acoustic Cues to Laryngeal Contrasts in Hindi Susan Jackson and Stephen Winters University of Calgary Acoustics Week in Canada October 14,

Pragmatics conversation. FloorFloor –The right to speak TurnTurn –The opportunity to speak Turn-takingTurn-taking –The change of speaker Local management.

1 Natural Language Processing Lecture Notes 14 Chapter 19.

Automatic Cue-Based Dialogue Act Tagging Discourse & Dialogue CMSC November 3, 2006.

Dirk Van CompernolleAtranos Workshop, Leuven 12 April 2002 Automatic Transcription of Natural Speech - A Broader Perspective – Dirk Van Compernolle ESAT.

Structural Metadata Annotation of Speech Corpora: Comparing Broadcast News and Broadcast Conversations Jáchym KolářJan Švec University of West Bohemia.

Recognizing Discourse Structure: Speech Discourse & Dialogue CMSC October 11, 2006.

Emilien Gorène. Automatic processing for pathologic speech : The case of schizophrenia Under direction of Maud Champagne-Lavau and Laurent Prévot. Recording.

Compiler Construction By: Muhammad Nadeem Edited By: M. Bilal Qureshi.

1/17/20161 Emotion in Meetings: Business and Personal Julia Hirschberg CS 4995/6998.

Finding document topics for improving topic segmentation Source: ACL2007 Authors: Olivier Ferret (18 route du Panorama, BP6) Reporter:Yong-Xiang Chen.

S1S1 S2S2 S3S3 8 October 2002 DARTS ATraNoS Automatic Transcription and Normalisation of Speech Jacques Duchateau, Patrick Wambacq, Johan Depoortere,

A STUDY ON PERCEPTUAL COMPENSATION FOR / /- FRONTING IN A MERICAN E NGLISH Reiko Kataoka February 14, 2009 BLS 35.

1 7-Speech Recognition Speech Recognition Concepts Speech Recognition Approaches Recognition Theories Bayse Rule Simple Language Model P(A|W) Network Types.

Natural Language Processing Vasile Rus

Sequence organisation in conversation: Sequence expansion

The Audio-Lingual Method

Approaches to Machine Translation

Linguistic knowledge for Speech recognition

Recognizing Disfluencies

Why Study Spoken Language?

Error Detection and Correction in SDS

Studying Intonation Julia Hirschberg CS /21/2018.

Issues in Spoken Dialogue Systems

Spoken Dialogue Systems

THE NATURE OF SPEAKING Joko Nurkamto UNS Solo.

Discourse How to analyse speech.

Why Study Spoken Language?

Turn-taking and Disfluencies

The Audio-Lingual Method

Recognizing Structure: Sentence, Speaker, andTopic Segmentation

Fadi Biadsy. , Andrew Rosenberg. , Rolf Carlson†, Julia Hirschberg

High Frequency Word Entrainment in Spoken Dialogue

Recognizing Disfluencies

Advanced NLP: Speech Research and Technologies

Spoken Dialogue Systems

Discourse Structure in Generation

Modules  Start! Bravo! Part 1: Basic Issues Module 10 CA Methodology

Approaches to Machine Translation

Towards Automatic Fluency Assessment

Recognizing Structure: Dialogue Acts and Segmentation

Behavioral Modeling with UML

Lexical selection: activates successive lexical items.

Artificial Intelligence 2004 Speech & Natural Language Processing

Correlation of EEG- and behavioral-based estimates of pairwise face similarity. Correlation of EEG- and behavioral-based estimates of pairwise face similarity.

Low Level Cues to Emotion

Presentation transcript:

Turn-taking and Disfluencies Julia Hirschberg CS 6998 11/29/2018

Today Turn-taking behaviors Conversational Analysis Importance in real systems Disfluencies How to model? Detect? Role in human-human interaction Importance in real systems? 11/29/2018

Turn-taking Expected patterns of behavior Deviation is significant How do we find the patterns? Ordinary conversation Telephone talk Meetings Email? Who looks for these? (Ilana) 11/29/2018

Telephone openings, closings Broadcasts Terminology Adjacency pairs Preference Pre-sequence Repair Examples: Telephone openings, closings Broadcasts 11/29/2018

Could this be useful when we build SDS? What do we expect to hear? What should we produce? 11/29/2018

Disfluencies and Self-Repairs Are these just ‘noise’? For people (Heba) For parsers For speech recognizers 11/29/2018

Hindle ’83: Finding the Edit Signal If we have it, can we ‘repair’ the self-repair automatically? Builds a correcting parser, Fidditch, for spontaneous speech Given a string with an edit signal marked, produces a ‘repaired’ version I was * I am really annoyed If X1 * X2 are similar linguistic elements separated by an edit signal, replace X1 w/X2 11/29/2018

What does it mean to be the same Same surface string Well if they’d * if they’d… Same category I was just that * the kind of guy… Same constituent I think that you get * it’s more strict in Catholic schools Restarts are completely different… I just think * Do you want something to eat? 11/29/2018

Bear et al ’92: Detecting and Correcting Self-Repairs Use multiple knowledge sources but not edit signal Lexical pattern matching Parsing failure + pattern matching + re-parsing Acoustic information: pause, peak F0, Cue words: well, no Fragments 11/29/2018

But…is there an edit signal? 11/29/2018

Does it identify self-repairs reliably? CART prediction: 86% precision, 91% recall Duration of interval, presence of fragment, pause filler, p.o.s., lexical matching across DI Are there edit signals? 11/29/2018

Next Week 11/29/2018