Real-Time Speech Recognition Subtitling in Education Respeaking 2009 Dr Mike Wald University of Southampton

Traditional supports create a reliance on intermediaries

Finding experienced or qualified interpreters, note-takers or re-speakers at university level is difficult

Speech Recognition: real time access to spoken language

Speech Recognition: also supports note taking

SR in classrooms is VERY DIFFICULT! Star Trek expectations Special vocabulary Spontaneous speech not writing Dialogue and interaction Andtherearenospacesbetweenwordswhenpeop letalksoitisunclearwherewordsbeginandend

How to Wreck a Nice Beach You Sing Calm Incense

This is a demonstration of the problem of the readability of text created by commercial speech recognition software used in lectures they were designed for the speaker to dictate grammatically complete sentences using punctuation by saying comma period new paragraph to provide phrase sentence and paragraph markers when people speak spontaneously they do not speak in what would be regarded as grammatically correct sentences as you can see you just see a continuous stream of text with no obvious beginnings and ends of sentences normal written text would break up this text by the use of punctuation such as commas and periods or new lines by getting the software to insert breaks in the text automatically by measuring the length of the silence between words we can improve the readability greatly

This is a demonstration of the problem of the readability of text created by commercial speech recognition software used in lectures they were designed for the speaker to dictate grammatically complete sentences using punctuation by saying comma period new paragraph to provide phrase sentence and paragraph markers when people speak spontaneously they do not speak in what would be regarded as grammatically correct sentences as you can see you just see a continuous stream of text with no obvious beginnings and ends of sentences normal written text would break up this text by the use of punctuation such as commas and period or new lines by getting the software to insert breaks in the text automatically by measuring the length of the silence between words we can improve the readability greatly

1998 LIBERATED LEARNING Pilot Project Today we will be discussing applied interactions in psychotherapy and how it related to Canadian law I will be covering chapters 7 8 and nine in preparation for next week;s midterm exam But first are they are questions about what we discussed yesterday lets move forward by asking the following question how does

It gives you something to compare your notes to and if you miss a class the notes are still accessible

It’s very helpful when the lecturer moves on while you’re still writing down a point as you can look at the screen

It’s forced me to be more reflective with my own teaching style and approach

It makes me ask myself what is teaching, why am I teaching this way, is there a better way?

Initial Research Summary Helps students access lectures Students thought it had great potential Improved teaching but gave teachers extra work Challenges: accuracy, readability, and ease of use

Accuracy & Understanding Stop / Proceed with Caution / GO

28 KB/s speech signal 320 b/s (240 words/minute) Real Time Editor Corrects 15 errors/minute 320 b/s (240 words/minute)

Speaker(s) uncorrected words Human intermediar(y/ies) can interact with a post processing system through the selection and correction of errors. SR System using voice & language models possible feedback? Post- processing system for automatic error identification and correction Corrected words SR Post-processing Enhancement

Text with Errors Text without Errors Post-processing Enhancement using statistical (including alternates lists and confidence levels), linguistic, context, phonemic, visual, signal and noise information

Text with Errors Text without Errors Post-processing Enhancement Topic Detection & Machine Translation

Text can reduce the memory demands of spoken language Speech can better express subtle emotions Images can communicate moods, relationships and complex information holistically. Multimedia

Easy to find WHOLE of recording but NOT a PART Analogy Text book with front cover but no contents page, index or page numbers

synchronised images (e.g. Slides) user created synchronised bookmarks, tags, notes with associated links to other resources audio / video synchronised text captions multimedia start time multimedia end time

Synchronised Multimedia Collaborate/Reflect Search Reason/Summarise Organise/Index Notes Tags Bookmarks Text Captions Images/Slides Video Audio & Links

Synote Supports learners to … search text and replay sections read transcript rather than listen to speech read text of slide images insert bookmark to continue later tag/highlight sections (e.g. not understood) add synchronised notes link to other web pages/resources

Synote Support teachers to… index and tag their recordings provide synchronised slides and captions respond to learner tags link to web pages/resources link to sections of existing multimedia

Synote Demo

Questions ?