2/21/2018 10:54 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2/21/2018 10:54 AM B8092 Using Microsoft Cognitive Services to bring the power of speech recognition to your apps Panos Periorellis Program Manager © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Speech@Microsoft Focus on Custom Speech Service Announcements Demos

Speech @ Microsoft MSFT Product Portfolio – Cortana, Skype, Xbox 2/21/2018 10:54 AM Speech @ Microsoft MSFT Product Portfolio – Cortana, Skype, Xbox Technology Speech-to-Text, Text-to-Speech, Speaker Identification Delivered as part of our Cognitive Services Library – Speech API (a.k.a Bing Speech API) Custom Speech Service Speaker Recognition API © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Speech API Imagine you are building an App and you want to be able to take voice input from your users and provide voice output. Out of the box Getting Started Samples, SDKs and Protocol for doing Speech-To-Text and Text-To-Speech Pre-deployed general purpose dictation or conversational models Hosted on the cloud with easy to use client SDKs REST (One-Shot) & WebSocket (Continuous) Transcription Over 30+ STT and TTS languages/locales/voice fonts

Speech Client SDK C#.NET/ WebSockets <SubscriptionKey>); this.client = SpeechRecognitionServiceFactory.CreateMicrophoneClient( SpeechRecognitionMode.ShortPhrase, "en-US", <SubscriptionKey>); // Event handlers for speech recognition results this.client.OnMicrophoneStatus += this.OnMicrophoneStatusHandler; this.client.OnPartialResponseReceived += this.OnPartialResponseReceivedHandler; this.client.OnResponseReceived += this.OnResponseReceivedHandler; this.client.OnConversationError += this.OnConversationErrorHandler; private void OnResponseReceivedHandler (object sender, SpeechResponseEventArgs e) { Console.WriteLine("--- OnDataShortPhraseResponseReceivedHandler ---");   Console.WriteLine(e);  } C#.NET/ WebSockets

Custom Speech Consider the case where your App users may have very specific needs Acoustic: noise conditions, accents, age Language: vocabulary, terminology Same SDK as Speech Customized dictation and conversational models hosted in Azure. Support for 3 languages

How do you adapt a speech model? Custom Speech Service gives developers the power to customize Microsoft’s speech-to-text engine to their applications. Create custom language models to accurately recognize application-specific words and phrases. Create custom acoustic models to optimize performance for particular environments or user populations. Deploy custom models to a speech-to-text endpoint accessible from any device.

Custom Speech Service Portal

How to use it? C#.NET/ WebSockets this.client = SpeechRecognitionServiceFactory.CreateMicrophoneClient( SpeechRecognitionMode.ShortPhrase, "en-US", <NewSubscriptionKey>, <model_Uri>); this.client.AuthenticationUri = “https://westus.api.cognitive.microsoft.com/sts/v1.0/issuetoken”; // Event handlers for speech recognition results this.client.OnMicrophoneStatus += this.OnMicrophoneStatusHandler; this.client.OnPartialResponseReceived += this.OnPartialResponseReceivedHandler; this.client.OnResponseReceived += this.OnResponseReceivedHandler; this.client.OnConversationError += this.OnConversationErrorHandler; C#.NET/ WebSockets

2/21/2018 10:54 AM Announcements © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Announcements Speech API Microsoft Build 2017 2/21/2018 10:54 AM Announcements Speech API web socket protocol (public), reference implementation in Javascript and SDK published Support for conversational speech recognition        Support for up-to 10 min of continuous audio input         New languages and voices released for TTS © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Announcements cont. Custom Speech Service Language Support: German, Spanish soon Custom Pronunciation Model Accuracy Evaluation Pricing Model Simplification

Pricing Details Custom Speech Service More freedom to adapt models (no limit on data) More model deployments Control over scaling up

Demo Ivo Santos Principal Software Architect Microsoft 2/21/2018 10:54 AM Demo Ivo Santos Principal Software Architect Microsoft [Microsoft Translator PowerPoint Add-in] © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Translator PowerPoint Add-in Microsoft Build 2017 2/21/2018 10:54 AM Microsoft Translator PowerPoint Add-in Live Transcriptions and Translations directly from PowerPoint Participants can join from mobile Apps (iOS, Android, Windows UWP) or web browser and make questions in their own language Slide contents are used to customize the speech recognition models Service can learn from previous presentations from the same speaker, personalizing her/his acoustic model © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The impossible slide The biomimetic approach of the Fujianese frog is amazing LSTM is also important for ASR The AI+R division is a brand new group Plougastel is a nice little village in Brittany

Microsoft Build 2017 2/21/2018 10:54 AM Behind the scenes Built using technology available today in Cognitive Services: Custom Speech Translator Speaker ID (coming soon) Slides Custom Speech Service Language Model Custom SR Endpoint Translator © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Speaker Recognition API Microsoft Build 2017 2/21/2018 10:54 AM Speaker Recognition API What if that same App requires to recognize the user from his/hers voice input? Voice has unique characteristics that can be used to identify a person, just like a fingerprint. Scenarios  Transcribing phone calls Taking notes in Meeting © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Speech Enabled Bot & App 2/21/2018 10:54 AM Speech Enabled Bot & App Khuram Shahid Principal Software Engineer © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Things to remember Easy integration of speech capability to your App and connection with other cognitive services Ability to adapt speech recognition to your needs Use speech at scale as a Cognitive Service to fit your business needs Make use of latest research improvements of speech recognition

Documentation Microsoft Cognitive services Microsoft Build 2017 2/21/2018 10:54 AM Documentation Microsoft Cognitive services https://www.microsoft.com/cognitive-services Speech (Bing Speech API) https://www.microsoft.com/cognitive-services/en-us/speech-api Custom Speech https://www.microsoft.com/cognitive-services/en-us/custom-recognition-intelligent-service-cris Speaker Recognition https://www.microsoft.com/cognitive-services/en-us/speaker-recognition-api © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2/21/2018 10:54 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.