Media Manager Mail Access Barbara Hohlt and Steve Czerwinski UC Berkeley Ericsson Presentation 2000

Project Overview Make messages more accessible –Enable faster browsing of many voic s –Access from different devices with different capabilities Media Mail services –Universal Inbox infrastructure –Voic is encoded in MIME Transcoding services –Enhance voic interaction –Includes: skimmed audio, transcript, text/audio summary, and outline

Related Work SpeechSkimmer, MIT’s MultiMedia Lab [Aaron95] Auto-Summarization, Microsoft Research CueVideo, IBM Other Universal Inboxes IBM’s universal inbox type project

Architecture Transcoder Service Voic ->Text Transcript Voic ->Text Summary Voic ->Text Outline ->Plain Audio Voic ->Audio Summary Voic ->Skimmed Audio Mail Access Interface NinjaMail Mail Access Interface POP Mail Access Interface IMAP Media Manager Interface Media Manager Service Client Folder Store

Bhaskar’s Cell-Phone Barbara’s Desktop Preference Registry Automatic Path Creation Service 800-MEDIA-MGR UID: Naming Service 1 1 mediamgr: Cluster locn. 2 2 MediaManager Mail Access Service 3 3 Bhaskar’s PSTN Phone Universal Inbox Clients

Messages and Content Objects Message Media Reference ID Queue of Content Objects Content Object Content ID Data Media Reference ID Message Part Content Type

MediaManagerServiceIF getFolders() getFoldersAs() getList() getListAs() getMessage() getMessageContent()

Audio Tools Speech Recognition/Synthesis –Transcribe voic to text –IBM ViaVoice SDK and custom audio libs Natural Language Processing –Directed word spotting by “understanding” content –ViaVoice SRCL Pitch –Detecting important words by emphasized pitch Pause –Compression through pause removal Spurts –Retrieve sentence structure of voic

Results Pause detection –Worked well for given applications –Playback speedup by 50-70% Pitch detection –Problems due to high pitch sounds and transitions Speech recognition –Performance decrease in conversational settings Natural Language Processing –Performed well with small grammar

Example: Adding GSM Adding GSMAudio and GSMSummary Add ContentObject definition to MediaManager –Add method createObject() –Add cases to Media Manager Add method toGSM() to Transcoder Edit.config file –External.transcoder.gsmrungsm Edit related transcoders –speechSynthesizer and audioSummary()

Implementing Other Mail Stores Implement MailAccessIF –MMgetFolders() –MMgetFoldersAs() –MMgetList() –MMgetListAs() –MMgetMessage() –MMgetMessageContent() Add new protocol to FolderStore

Conclusion