1. Your Metro style app, video and audio, Part 1
4/23/2017 3:44 PM
PLAT-775T
Your Metro style app, video and audio, Part 1
Gabe Frost
Senior Program Manager Lead
Microsoft Corporation
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. demo
Music & Video apps
It’s easy to build tailored playback experiences with rich file metadata.

3. Agenda Delivering great tailored audio and video experiences
Media platform
HTML5 Media Elements
Programming Model
A word on formats
Recap
You’ll leave with examples of how to
Create apps with rich, integrated media experiences
Incorporate media into your apps in new ways

4. Building apps with great audio and video experiences is easy.

5. Use media to make apps more visually rich and engaging.
demo
Weather app
Use media to make apps more visually rich and engaging.

6. With media APIs + Media Elements
Take advantage of graphics hardware and devices with little domain knowledge
Easy to build tailored user experiences with built-in UI and controls
Simple to use media from sources other than web servers, like Local or HomeGroup PC libraries and network attached storage
Demanding scenarios like streaming, 3D and DRM are simplified because of the powerful and extensible underlying media platform

7. Windows Metro Style App <audio src=“…”> <video src=“…”>
Media platform
Windows Metro Style App
<audio src=“…”>
<video src=“…”>
WinRT (Windows.Media)
Playback/Preview
Capture
Transcode
Streaming
Media Foundation
Audio/Video
Source
Video
Decoder
Video
Effect 1
Video
Effect N
Video
Encoder
Audio/Video
Sink
Audio
Decoder
Audio
Effect 1
Audio
Effect N
Audio
Encoder

8. HTML5 Media Elements

9. Standard HTML5 <audio> and <video>
W3C HTML5 Spec
You can create rich media experiences by combining with CSS and JavaScript and DOM events
Fully hardware accelerated
Performance, battery life and reliability
Select methods and attributes
play()
pause()
load()
audioTracks
readyState
controls
currentTime
duration
error
src
playbackRate
preload
volume …

10. Standard HTML5 <audio> and <video>
HTML Video Tag Events
abort
canplay
canplaythrough
durationchange
emptied
ended
error
loadeddata
loadedmetadata
loadstart
pause
play
playing
progress
ratechange
seeking
seeked
suspend
stalled
timeupdate
volumechange
waiting

11. Example: HTML5 Audio and Video Elements
<!-- Attributes allowed on both tags: src, preload, autoplay,
loop, controls. Video tags also support poster, width, height, muted. Both support many events (e.g. volumechanged) -->
<audio id="myAudioTag" src="mySong.wma" preload="none" />
<audio src="mySong.m4a" autoplay loop controls />
<video id="myVideoTag" src="myVideo.mp4" preload="none" />
<video src="myVideo.wmv" autoplay loop controls />
<video src="myVideo.mp4" autoplay muted />
<video src="myAudio.mp3" poster="myImage.jpg" controls />

12. Example: HTML5 Audio and Video Elements
<audio id="myAudio" src=" autoplay />
<script type="text/javascript">
var audio = document.getElementById("myAudio");
// Handle when the audio track has reached the end
audio.addEventListener("ended", function() {
// Switch the audio src to a new track
audio.src = "
});
</script>

13. Example: Local files with Media Elements
<video id="player" controls autoplay />
<script type="text/javascript">
// Invoke a file picker for the user to select media from the file system
var picker = new Windows.Storage.FileOpenPicker();
picker.pickSingleFileAsync().
then(function(file) {
// Create a URL to represent the local file
var url = URL.createObjectURL(file);
document.getElementById("player").src = url;
});
<script>

14. Example: Accessing file metadata
// Programmatically select the file (could also be from home network storage)
var file = new
Windows.Storage.KnownFolders.musicLibrary.getFileAsync("track.mp3").
// From the file, extract media-specific properties
then(function(myFile) {
myFile.properties.getMusicPropertiesAsync().
then(function(musicProps) {
var album = musicProps.album;
var artist = musicProps.artist;
var title = musicProps.title;
// etc.
});

15. Windows 8 enhancements 3D video
Audio and video effects and extensibility
Audio output selection such as Bluetooth
Background audio
DRM such as PlayReady
Low-latency support for communications & games
Play To for streaming to TVs & audio systems
Zoom and mirror mode for video

16. Programming Model

17. Windows.Media APIs Much more than playing video and audio
Snap a picture, record video and audio
Capture
Devices
Input and output audio devices such as Bluetooth
Playlists
Work with audio playlists
Windows.Media.
Stream media to a television or audio system
PlayTo
Protection
Use PlayReady or other content protection systems
Convert music or video to other formats or resolutions, trim, or make effects permanent
Transcoding
VideoEffects
Insert or remove well-known video effects such as Stabilization

18. Streaming to devices with Play To
demo
Streaming to devices with Play To
HTML5 websites using IE10, Music, Video and Photo apps work today.
Simple to support in your Metro style apps

19. Example: Stream media with Play To (JS)
<video id="videoplayer" src=" controls autoplay />
<script type="text/javascript">
// Step 1: Obtain PlayToManager object for app’s current view.
var ptm = Windows.Media.PlayTo.PlayToManager.getForCurrentView();
// Step 2: Register for the sourcerequested event (user selects Devices button).
ptm.addEventListener("sourcerequested", function(e) {
var request = e.sourceRequest;
// Step 3: Specify the media to be streamed (to filter devices)
var deferral = request.getDeferral();
request.setSource(document.getElementById("videoplayer").msPlayToSource);
deferral.complete();
// The media will then be streamed to the device chosen by the user in the UI.
});
</script>

20. Example: Stream media with Play To (C#)
<MediaElement x:Name="videoplayer" Source="
AutoPlay="true" />
// Step 1: Obtain PlayToManager object for app’s current view.
PlayToManager ptm = Windows.Media.PlayTo.PlayToManager.GetForCurrentView();
// Step 2: Register for the SourceRequested event (user selects Devices button).
ptm.SourceRequested += (PlayToManager sender, PlayToSourceRequestedEventArgs e) => {
request = e.SourceRequest;
// Step 3: Specify the media to be streamed.
PlayToSourceDeferral deferral = request.GetDeferral();
request.SetSource(videoplayer.PlayToSource);
deferral.Complete(); }
// The media will then be streamed to the device chosen by the user in the UI.

21. demo
Camera Capture UI
It’s easy to snap a photo or record video by invoking the built-in capture UI.

22. Use the camera to snap a photo
// Application manifest capabilities required to access camera and microphone
<Capabilities>
    <DeviceCapability Name="webcam" />
<DeviceCapability Name="microphone" />
</Capabilities>

23. Use the camera to snap a photo
<img id="preview" />
<script type="text/javascript">
// Invoke the camera capture UI for taking a photo
var captureUI = new Windows.Media.Capture.CameraCaptureUI();
captureUI.captureFileAsync(Windows.Media.CameraCaptureUI.Mode.photo).
then(function(capturedItem) {
if (capturedItem) {
// Display the photo
document.getElementById("preview").src =
URL.createObjectURL(capturedItem); }
});
</script>

24. demo
Simple Encode app
Convert media files into different formats, resolutions or trim to a specific timeframe.

25. Transcode to a different format
var outputFile;
var vidLib = Windows.Storage.KnownFolders.videosLibrary;
var transcoder = new Windows.Media.Transcoding.MediaTranscoder();
// Choose a popular output format and quality
var profile = Windows.Media.Capture.MediaEncodingProfile.
createMp4(Windows.Media.Capture.VideoEncodingQuality.vga);
// Create the file to transcode into, and select the input file to read from
vidLib.createFileAsync("output.mp4").
then(function(result) {
outputFile = result;
return vidLib.getFileAsync("test.wmv");
// Transcode the file
}).then(function(inputFile) {
return transcoder.transcodeFileAsync(inputFile, outputFile, profile);
}).then(function() { // log "done" });

26. Simple trim var outputFile;
var vidLib = Windows.Storage.KnownFolders.videosLibrary;
var transcoder = new Windows.Media.Transcoding.MediaTranscoder();
// Specify the start and stop time defining the portion to keep
// Create the file to transcode into, and select the input file to read from
vidLib.createFileAsync("EditedVacation.mp4").
then(function(result) {
outputFile = result;
return vidLib.getFileAsync("vacation.mp4");
// Trim the file: profile isn’t changing so profile argument isn’t needed
}).then(function(inputFile) {
return transcoder.transcodeFileAsync(inputFile, outputFile);
}).then(function() { // log "done" });
transcoder.trimStartTime = 900;
transcoder.trimStopTime = 6000;

27. A word on formats

28. Media Formats and Codecs
System-wide
File format: MP4 (preferred) or ASF
Video: H.264 or VC-1
Audio: AAC, MP3 or WMA
Broad support across consumer hardware: great performance and battery life
Per-app
Your Metro style app can package private plug-ins for its own use (e.g. adaptive streaming)

29. Recap

30. With media APIs + Media Elements
Take advantage of graphics hardware and devices with little domain knowledge
Easy to build tailored user experiences with built-in UI and controls
Simple to use media from sources other than web servers, like Local or HomeGroup PC libraries and network attached storage
Demanding scenarios like streaming, 3D and DRM are simplified because of the powerful and extensible underlying media platform

31. Related sessions
[PLAT-776T] Your Metro style app, video and audio, Part 2
[APP-788T] Integrating stunning media experiences in XAML
[PLAT-777T] Capturing personal photos, video, and audio in Metro style apps
[HW-715T] Building a great Metro style device app for your camera
[PLAT-778T] Media fundamentals of a communications app
[PLAT-783T] Extending the media platform using input, output and processing plug-ins
[HW-177T] Building great networked media devices for Play To apps
[APP-408T] Integrating with the Windows device experience
[HW-329T] Understanding Wi-Fi Direct in Windows 8

32. Further reading and documentation
Design guidelines for Play To receivers
Streaming media to devices using Play To
Play To & media streaming for Metro style apps
Using audio hardware acceleration for ultra-low-power consumption.

33. thank you Feedback and questions http://forums.dev.windows.com
Session feedback

34. 4/23/2017 3:44 PM
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

35. BACKUP

36. Use the camera to snap a photo (C#)
// Define capture element in XAML
<captureElement x:Name="preview" Width="320" Height="240" Margin="10,5,10,5" />
// Initialize
mc = new MediaCapture();
Windows.Media.Capture.InitializeOperation opInitialize = await mc.initializeAsync();
// Assign the source
preview.Source = mc;
// Start the preview
Windows.Media.Capture.StartPreviewOperation opStartpreview = await
mc.StartPreviewAsync();

37. Transcode to a different format (C#)
// Select the input file to read from
StorageFile input = await
KnownFolders.VideosLibrary.GetFileAsync("input.wmv");
// Create the file to transcode into
StorageFile output = await
KnownFolders.VideosLibrary.CreateFileAsync("output.mp4");
// Choose a popular output format and quality
MediaEncodingProfile profile =
MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Vga);
// Transcode the file
MediaTranscoder transcoder = new MediaTranscoder();
await transcoder.TranscodeFileAsync(input, output, profile);

38. Simple trim (C#) // Select the input file to read from
StorageFile input = await
KnownFolders.VideosLibrary.GetFileAsync("input.mp4");
// Create the file to transcode into
StorageFile output = await
KnownFolders.VideosLibrary.CreateFileAsync("output.mp4");
MediaTranscoder transcoder = new MediaTranscoder();
// Specify the start and stop time defining the portion to keep
// Trim the file: profile isn’t changing so profile argument isn’t needed
await transcoder.TranscodeFileAsync(input, output);
transcoder.TrimStartTime = 900;
transcoder.TrimStopTime = 6000;