1. 2/21/ :54 AM
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. 2/21/ :54 AM
B8092
Using Microsoft Cognitive Services to bring the power of speech recognition to your apps
Panos Periorellis
Program Manager
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. Agenda Speech@Microsoft Focus on Custom Speech Service Announcements
Demos

4. Speech @ Microsoft MSFT Product Portfolio – Cortana, Skype, Xbox
2/21/ :54 AM
Microsoft
MSFT Product Portfolio – Cortana, Skype, Xbox
Technology
Speech-to-Text, Text-to-Speech, Speaker Identification
Delivered as part of our Cognitive Services Library –
Speech API (a.k.a Bing Speech API)
Custom Speech Service
Speaker Recognition API
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. Speech API
Imagine you are building an App and you want to be able to take voice input from your users and provide voice output.
Out of the box Getting Started Samples, SDKs and Protocol for doing Speech-To-Text and Text-To-Speech
Pre-deployed general purpose dictation or conversational models
Hosted on the cloud with easy to use client SDKs
REST (One-Shot) & WebSocket (Continuous) Transcription
Over 30+ STT and TTS languages/locales/voice fonts

6. Speech Client SDK C#.NET/ WebSockets <SubscriptionKey>);
this.client = SpeechRecognitionServiceFactory.CreateMicrophoneClient(
SpeechRecognitionMode.ShortPhrase,
"en-US",
<SubscriptionKey>);
// Event handlers for speech recognition results
this.client.OnMicrophoneStatus += this.OnMicrophoneStatusHandler;
this.client.OnPartialResponseReceived += this.OnPartialResponseReceivedHandler;
this.client.OnResponseReceived += this.OnResponseReceivedHandler;
this.client.OnConversationError += this.OnConversationErrorHandler;
private void OnResponseReceivedHandler (object sender, SpeechResponseEventArgs e) {
Console.WriteLine("--- OnDataShortPhraseResponseReceivedHandler ---");
  Console.WriteLine(e);  }
C#.NET/ WebSockets

7. Custom Speech
Consider the case where your App users may have very specific needs
Acoustic: noise conditions,
accents, age
Language:
vocabulary, terminology
Same SDK as Speech
Customized dictation and conversational models hosted in Azure.
Support for 3 languages

8. How do you adapt a speech model?
Custom Speech Service gives developers the power to customize Microsoft’s speech-to-text engine to their applications.
Create custom language models to accurately recognize application-specific words and phrases.
Create custom acoustic models to optimize performance for particular environments or user populations.
Deploy custom models to a speech-to-text endpoint accessible from any device.

9. Custom Speech Service Portal

10. How to use it? C#.NET/ WebSockets
this.client = SpeechRecognitionServiceFactory.CreateMicrophoneClient(
SpeechRecognitionMode.ShortPhrase,
"en-US",
<NewSubscriptionKey>,
<model_Uri>);
this.client.AuthenticationUri = “
// Event handlers for speech recognition results
this.client.OnMicrophoneStatus += this.OnMicrophoneStatusHandler;
this.client.OnPartialResponseReceived += this.OnPartialResponseReceivedHandler;
this.client.OnResponseReceived += this.OnResponseReceivedHandler;
this.client.OnConversationError += this.OnConversationErrorHandler;
C#.NET/ WebSockets

11. 2/21/ :54 AM
Announcements
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12. Announcements Speech API
Microsoft Build 2017
2/21/ :54 AM
Announcements
Speech API
web socket protocol (public), reference implementation
in Javascript and SDK published
Support for conversational speech recognition
       Support for up-to 10 min of continuous audio input
        New languages and voices released for TTS
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

13. Announcements cont. Custom Speech Service
Language Support: German, Spanish soon
Custom Pronunciation
Model Accuracy Evaluation
Pricing Model Simplification

14. Pricing Details Custom Speech Service
More freedom to adapt models (no limit on data)
More model deployments
Control over scaling up

15. Demo Ivo Santos Principal Software Architect Microsoft
2/21/ :54 AM
Demo
Ivo Santos
Principal Software Architect
Microsoft
[Microsoft Translator PowerPoint Add-in]
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

16. Microsoft Translator PowerPoint Add-in
Microsoft Build 2017
2/21/ :54 AM
Microsoft Translator PowerPoint Add-in
Live Transcriptions and Translations directly from PowerPoint
Participants can join from mobile Apps (iOS, Android, Windows UWP) or web browser and make questions in their own language
Slide contents are used to customize the speech recognition models
Service can learn from previous presentations from the same speaker, personalizing her/his acoustic model
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

17. The impossible slide
The biomimetic approach of the Fujianese frog is amazing
LSTM is also important for ASR
The AI+R division is a brand new group
Plougastel is a nice little village in Brittany

18. Microsoft Build 2017
2/21/ :54 AM
Behind the scenes
Built using technology available today in Cognitive Services:
Custom Speech
Translator
Speaker ID (coming soon)
Slides
Custom Speech Service
Language Model
Custom SR Endpoint
Translator
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19. Speaker Recognition API
Microsoft Build 2017
2/21/ :54 AM
Speaker Recognition API
What if that same App requires to recognize the user from his/hers voice input?
Voice has unique characteristics that can be used to identify a person, just like a fingerprint.
Scenarios 
Transcribing phone calls
Taking notes in Meeting
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20. Speech Enabled Bot & App
2/21/ :54 AM
Speech Enabled Bot & App
Khuram Shahid
Principal Software Engineer
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

21. Things to remember
Easy integration of speech capability to your App and connection with other cognitive services
Ability to adapt speech recognition to your needs
Use speech at scale as a Cognitive Service to fit your business needs
Make use of latest research improvements of speech recognition

22. Documentation Microsoft Cognitive services
Microsoft Build 2017
2/21/ :54 AM
Documentation
Microsoft Cognitive services
Speech (Bing Speech API)
Custom Speech
Speaker Recognition
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. 2/21/ :54 AM
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.