1. Reimagine what is possible with your videos using AI
6/17/2018 6:26 AM
BRK2175
Reimagine what is possible with your videos using AI
Martin Wahl
Principal Program Manager
@redsoxnutt
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Microsoft Build 2017
6/17/2018 6:26 AM
Video growth trends
Global IP video traffic will be 82 percent of all IP traffic by 2020
By 2020, internet video surveillance traffic will be more than 5 EBs per month
Amount of VOD traffic in will be equivalent to billion DVDs per month
Every second a million minutes of video will cross the network by 2020
Source – Cisco Zettabyte Era Whitepaper
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. Live & On Demand Streaming
Azure Media Services
Azure Media Services is an extensible, multi-tenant platform that enables end-to-end video workflows in the Azure public cloud. .
Cloud Upload & Storage
Encoding
Media Analytics
& Video AI
Content Protection
Live & On Demand Streaming
with Azure CDN
Player
Clients
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4. Imagine if you could…
Automatically create all of the metadata (spoken text, written text, faces, places, objects etc) for any video, including from their archives
As a call center operator, scan for frustrated customers or count time spent on hold
Collaboratively upload & edit videos in real-time, search to find key moments, and speed up time to publish new content
Enable your viewers to search by Name, Search by Keyword, Search by Shot Location, key objects
Distribute playable clips of live video content internally or externally without long download or encode times
Create automated summaries or highlight reels of their video content based on scene detection, specific people, and motion within the video
Translate your content into other languages for global and diverse audience reach

5. Media Analytics / Video AI
6/17/2018 6:26 AM
Media Analytics / Video AI
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6. Video AI is the key to solving these challenges….
6/17/2018
Video AI is the key to solving these challenges….
Improve Content Discoverability
Increase Content Value
Personalize the Viewing Experience
Uncover Hidden Content Insights
Reduce Manual Labor
Increase Revenue / Drive Viewing
Targeted Advertising
Predictive modeling & recommendations
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

7. Azure Media Analytics | Video AI Services
Microsoft Build 2017
6/17/2018 6:26 AM
Azure Media Analytics | AI Features
Azure Media Analytics | Video AI Services
Audio transcript
Convert audio to text based on acoustic language models
Face detection
Find when each face appears in the video
Video stabilization
Create smooth videos from videos captured by moving camera
Video OCR
Extract text that appears in videos as overlay, slides or background
Face redaction
Detect faces and choose which ones you want to redact
Motion detection
Detect when motion has occurred in videos
Face emotion detection
Recognize the emotion of a person or crowd based on facial expressions
Video summaries
Create summaries of long videos to enable quick previews
Content moderation
Detect and prevent explicitly visual content
Object detection
Detect objects based on a pre-defined object model
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8. Industries that can benefit with Video AI…
Microsoft Build 2017
6/17/2018 6:26 AM
Industries that can benefit with Video AI…
News
Entertainment
Sports
Retail
Government
Surveillance
Public Safety
More…
…they all have a lot of video content!
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9. Microsoft Cognitive Services
Microsoft Build 2016
6/17/2018 6:26 AM
Microsoft Cognitive Services
Give your apps a human side
Vision
From faces to feelings, allow your apps to understand images and video
Speech
Hear and speak to your users by filtering noise, identifying speakers, and understanding intent
Language
Process text and learn how to recognize what users want
Knowledge
Tap into rich knowledge amassed from the web, academia, or your own data
Search
Access billions of web pages, images, videos, and news with the power of Bing APIs
Labs
An early look at emerging Cognitive Services technologies: discover, try & give feedback on new technologies before general availability
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

10. Microsoft Cognitive Services
Microsoft Build 2016
6/17/2018 6:26 AM
Microsoft Cognitive Services
Give your apps a human side
Vision
Speech
Knowledge
Language
Labs
Search
Computer Vision
Content Moderator
Emotion
Face
Video Indexer
Custom Vision Service
Bing Speech
Custom Speech Service
Speaker Recognition
Bing Spell Check
Language Understanding
Linguistic Analysis
Text Analytics
Translator Text & Speech
Web Language Model
Academic Knowledge
Entity Linking
Knowledge Exploration
Recommendations
QnA Maker
Custom Decision Service
Bing Autosuggest
Bing Image Search
Bing News Search
Bing Video Search
Bing Web Search
Bing Custom Search
Project Prague (gesture)
Project Cuzco (events)
Project Johannesburg (routing)
Project Nanjing (isochrones)
Project Abu Dhabi (distance matrix)
Project Wollongong (location)
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11. Introducing… Video Indexer
6/17/2018 6:26 AM
Introducing… Video Indexer
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12. Video Indexer | Bundling Features Together
Microsoft Build 2017
Video Indexer | Bundling Features Together
6/17/2018 6:26 AM
Video Indexer | AI Features
Widgets
Search
Audio
transcript
Convert audio to text based on acoustic language models
Face
detection
Find when each face appears in the video
Visual text recognition
Extract text that appears in videos as overlay, slides or background
Object Tagging
Recognize common objects such as cat, table, car, ball etc. when they appear
Translation
Translate audio transcript to another language
Content moderation
Detect and prevent explicitly visual and textual content
Face identification
Identify who is the person that appears in the video
Speaker indexing
Map and understand who spoke what and when
Keywords extraction
Find out the keywords discussed in the video
Sentiment analysis
Understand the level of positive vs. negative spoken or written words
Scene
Detection
Detect when a scene changes based on visual analysis
Keyframe extraction
Automatically detect keyframes in the video
Scalable & Efficient Orchestration
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

13. Video Indexer Video AI | Products Video Indexer Customer app Extract
Microsoft Build 2017
6/17/2018 6:26 AM
Video AI | Products
Video Indexer
Video Indexer
Customer app
Extract
Extract a set of pre-defined cognitive metadata
Curate
Review and curate the metadata using a web portal
Publish
Embed player and insights widgets in your own apps
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. Video Indexer Demo 6/17/2018 6:26 AM
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

15. Video Indexer extracted metadata snippet
6/17/2018 6:26 AM
Video Indexer extracted metadata snippet
"faces": [ {
"id": 11775,
"shortId": "63c6d532ff",
"name": "Scott Hanselman",
"description": null,
"title": null,
"thumbnailUrl": "/api/Thumbnail/63c6d532ff/616468f efa-94e7-262f2e575059",
"thumbnailFullUrl": "
"appearances": [
"startTime": "00:00: ",
"endTime": "00:00: ",
"startSeconds": 8.2,
"endSeconds": 25.4 },
"startTime": "00:12: ",
"endTime": "00:13: ",
"startSeconds": 777.7,
"endSeconds": 787.1 } ],
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

16. Video Indexer extracted metadata snippet
6/17/2018 6:26 AM
Video Indexer extracted metadata snippet
"topics": [ {
"name": "days of history",
"appearances": [
"startTime": "00:04: ",
"endTime": "00:00:00",
"startSeconds": 282.1,
"endSeconds": 0.0 },
"startTime": "00:12: ",
"startSeconds": 727.6, } ],
"isTranscript": true ]
"sentiments": [ {
"sentimentKey": "Negative",
"appearances": [
"startTime": "00:00:00",
"endTime": "00:00: ",
"startSeconds": 0.0,
"endSeconds": 3.5 },
"startTime": "00:13: ",
"endTime": "00:13:14",
"startSeconds": 786.0,
"endSeconds": 794.0 } ],
"seenDurationRatio": ]
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

17. Video Indexer insights widget
6/17/2018 6:26 AM
Video Indexer insights widget
<iframe width="580" height="780" src=" frameborder="0" allowfullscreen></iframe>
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

18. Video AI | Integrated for Success
Microsoft Build 2017
6/17/2018 6:26 AM
Video AI | Integrated for Success
Partner Solutions
Video Indexer
Portal
APIs
Widgets
Additional video AI technologies
Orchestration engine
Azure Media Analytics
Other Cognitive services
Cosmos DB
Azure Search
& storage
Azure Media Encoding
Audio transcript
Face detection
Video stabilization
Translator
Text analytics
Celebrity identification
Video OCR
Object detection
Content moderator
Topic extraction
Face API
Bing entity search
More…
More…
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19. Video Indexer APIs Use cases: Index only Index and Translate Search
6/17/2018 6:26 AM
Video Indexer APIs
Use cases:
Index only
Index and Translate
Search
Visualize Insights
Streaming
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20. Video Indexer roadmap New improved Video AI algorithms
6/17/2018 6:26 AM
Video Indexer roadmap
New improved Video AI algorithms
Face emotions, Content moderator, audio event detection, more objects in object model, activity detection.
Customization of Models
Enable custom models for targeted output
Includes - Integration of CRIS, evolution of face recognition, IRIS for videos.
Content creation
Highlights based on metadata from multiple videos, Automated video summarization
Live video as Input
Integrate with live channels to provide near real time metadata from the video stream
Auto Transcription, Live Translation, Real-time face detection
Analytics
Key metrics, “Top 10 items”, etc.
Integration
Continued improvements to our Logic Apps/Flow Connector
Support for Triggers
Simplified integration with Cosmos DB and Azure Search
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

21. Putting it all together: Azure Logic Apps & Azure Flow
6/17/2018 6:26 AM
Putting it all together: Azure Logic Apps & Azure Flow
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. Azure Media Services, Azure Logic App & Azure Functions
6/17/2018 6:26 AM
Azure Media Services, Azure Logic App & Azure Functions
Example Logic App
Upload Video File to Blob Storage from OneDrive
Encode, Index & Summarize
Publish and Share streaming URL via SMS, , and Twitter
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. Azure Flow Example Flow App
Upload Video File to Blob Storage from OneDrive
Send to Video Indexer
Publish and Share streaming URL via
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

24. Getting creative with combining….
6/17/2018 6:26 AM
Getting creative with combining….
Auto Face & Object Tagging/ Live Captioning/ +
Live Translation
Logic Apps/Functions +
Live Streaming +
Live Subclipping +
Media Analytics +
AMP Plugin
+ Bing Translation
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25. Please evaluate this session
Tech Ready 15
6/17/2018
Please evaluate this session
From your
Please expand notes window at bottom of slide and read. Then Delete this text box.
PC or tablet: visit MyIgnite
Phone: download and use the Microsoft Ignite mobile app
Your input is important!
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

26. 6/17/2018 6:26 AM
© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.