Meeting Recorder Adam Janin

2 A Portable Meeting Recorder n Record meetings in natural settings n Support multiple speakers n Allowing correction and annotation n Support indexing and searching n Self-contained (using IRAM)

3 Vector IRAM n High absolute performance u Floating point and integer vector units u High bandwidth to memory n High performance per watt u Low instruction dispatch rate for vectors u On-chip memory n Well-suited for Meeting Recorder u ASR is (mostly) easy to vectorize u Supports floating point u Compiler

5 Why Meeting Recorder? n Why not written notes? u Unreadable handwriting u Detracts from listening u Incomplete record u Hard to search n Why not a tape recorder? u Hard to annotate u Playback can be disruptive u Too much data u Hard to search

6 Natural Settings n Record impromptu meetings n Robust to noise, reverberation, etc. n Large vocabulary n Spontaneous speech n Portable n Self-contained n Hands-free

7 Multiple Speakers n Robust to accents, speaking styles n Interrupts n Mark speaker change n Adapt to common participants

8 Correction and Annotation n Correcting – informing the system that it has made an error u If the system has a good idea of alternatives, it may be faster to correct than to edit u Recognizer can adapt to user and vocabulary n Annotation – Supplementing and modifying the output u “That’s not what I meant to say” u Pen vs. speech input u General ink and text annotations

9 Indexing and searching n Index by speaker and text content n Store and replay audio n Allow text and spoken queries n Perfect speech recognition is not required! u N-best lists u Function vs. content words u Audio record

10 Collaboration n Multiple Meeting Recorders in a meeting connected via a wireless network n Offline u Share notes (à la NotePals) u Improve transcript quality n Online u “Chat” u Microphone array

11 Data Collection Phase n Meeting room at ICSI n 5 wireless head-mounted microphones n 3 PZM omni-directional microphones n HandSpring PDA (probably wired network) n Primarily speech and NLP meetings n Collect and hand-transcribe about 40 hours of meetings

12 Issues n Will people object to being recorded? n Will their speaking style change? n Are the limitations of the data collection phase important? n Is cross-domain training feasible? n Lots of UI issues… n Lots of IR issues... n Lots of ASR issues…

13 Prototype of a Personal Dictation System n Records a single user dictating text n Allows correction and limited annotation Correct transcripts Insert annotations Search/playback Audio frontend Speech recognizer Correction server n Hosted system Palm Pilot Sun Workstation

Prototype Screen Shots Richard Posner, who heads the United States Court of Appeals...