Your Metro style app, video and audio, Part 1 4/23/2017 3:44 PM PLAT-775T Your Metro style app, video and audio, Part 1 Gabe Frost Senior Program Manager Lead Microsoft Corporation © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

demo Music & Video apps It’s easy to build tailored playback experiences with rich file metadata.

Agenda Delivering great tailored audio and video experiences Media platform HTML5 Media Elements Programming Model A word on formats Recap You’ll leave with examples of how to Create apps with rich, integrated media experiences Incorporate media into your apps in new ways

Building apps with great audio and video experiences is easy.

Use media to make apps more visually rich and engaging. demo Weather app Use media to make apps more visually rich and engaging.

With media APIs + Media Elements Take advantage of graphics hardware and devices with little domain knowledge Easy to build tailored user experiences with built-in UI and controls Simple to use media from sources other than web servers, like Local or HomeGroup PC libraries and network attached storage Demanding scenarios like streaming, 3D and DRM are simplified because of the powerful and extensible underlying media platform

Windows Metro Style App <audio src=“…”> <video src=“…”> Media platform Windows Metro Style App <audio src=“…”> <video src=“…”> WinRT (Windows.Media) Playback/Preview Capture Transcode Streaming Media Foundation Audio/Video Source Video Decoder Video Effect 1 Video Effect N Video Encoder Audio/Video Sink Audio Decoder Audio Effect 1 Audio Effect N Audio Encoder

HTML5 Media Elements

Standard HTML5 <audio> and <video> W3C HTML5 Spec You can create rich media experiences by combining with CSS and JavaScript and DOM events Fully hardware accelerated Performance, battery life and reliability Select methods and attributes play() pause() load() audioTracks readyState controls currentTime duration error src playbackRate preload volume …

Standard HTML5 <audio> and <video> HTML Video Tag Events abort canplay canplaythrough durationchange emptied ended error loadeddata loadedmetadata loadstart pause play playing progress ratechange seeking seeked suspend stalled timeupdate volumechange waiting

Example: HTML5 Audio and Video Elements <!-- Attributes allowed on both tags: src, preload, autoplay, loop, controls. Video tags also support poster, width, height, muted. Both support many events (e.g. volumechanged) --> <audio id="myAudioTag" src="mySong.wma" preload="none" /> <audio src="mySong.m4a" autoplay loop controls /> <video id="myVideoTag" src="myVideo.mp4" preload="none" /> <video src="myVideo.wmv" autoplay loop controls /> <video src="myVideo.mp4" autoplay muted /> <video src="myAudio.mp3" poster="myImage.jpg" controls />

Example: HTML5 Audio and Video Elements <audio id="myAudio" src="http://www.contoso.com/track1.mp3" autoplay /> <script type="text/javascript"> var audio = document.getElementById("myAudio"); // Handle when the audio track has reached the end audio.addEventListener("ended", function() { // Switch the audio src to a new track audio.src = "http://www.contoso.com/track2.m4a"; }); </script>

Example: Local files with Media Elements <video id="player" controls autoplay /> <script type="text/javascript"> // Invoke a file picker for the user to select media from the file system var picker = new Windows.Storage.FileOpenPicker(); picker.pickSingleFileAsync(). then(function(file) { // Create a URL to represent the local file var url = URL.createObjectURL(file); document.getElementById("player").src = url; }); <script>

Example: Accessing file metadata // Programmatically select the file (could also be from home network storage) var file = new Windows.Storage.KnownFolders.musicLibrary.getFileAsync("track.mp3"). // From the file, extract media-specific properties then(function(myFile) { myFile.properties.getMusicPropertiesAsync(). then(function(musicProps) { var album = musicProps.album; var artist = musicProps.artist; var title = musicProps.title; // etc. });

Windows 8 enhancements 3D video Audio and video effects and extensibility Audio output selection such as Bluetooth Background audio DRM such as PlayReady Low-latency support for communications & games Play To for streaming to TVs & audio systems Zoom and mirror mode for video

Programming Model

Windows.Media APIs Much more than playing video and audio Snap a picture, record video and audio Capture Devices Input and output audio devices such as Bluetooth Playlists Work with audio playlists Windows.Media. Stream media to a television or audio system PlayTo Protection Use PlayReady or other content protection systems Convert music or video to other formats or resolutions, trim, or make effects permanent Transcoding VideoEffects Insert or remove well-known video effects such as Stabilization

Streaming to devices with Play To demo Streaming to devices with Play To HTML5 websites using IE10, Music, Video and Photo apps work today. Simple to support in your Metro style apps

Example: Stream media with Play To (JS) <video id="videoplayer" src="http://www.contoso.com/clip.mp4" controls autoplay /> <script type="text/javascript"> // Step 1: Obtain PlayToManager object for app’s current view. var ptm = Windows.Media.PlayTo.PlayToManager.getForCurrentView(); // Step 2: Register for the sourcerequested event (user selects Devices button). ptm.addEventListener("sourcerequested", function(e) { var request = e.sourceRequest; // Step 3: Specify the media to be streamed (to filter devices) var deferral = request.getDeferral(); request.setSource(document.getElementById("videoplayer").msPlayToSource); deferral.complete(); // The media will then be streamed to the device chosen by the user in the UI. }); </script>

Example: Stream media with Play To (C#) <MediaElement x:Name="videoplayer" Source="http://www.contoso.com/clip.mp4" AutoPlay="true" /> // Step 1: Obtain PlayToManager object for app’s current view. PlayToManager ptm = Windows.Media.PlayTo.PlayToManager.GetForCurrentView(); // Step 2: Register for the SourceRequested event (user selects Devices button). ptm.SourceRequested += (PlayToManager sender, PlayToSourceRequestedEventArgs e) => { request = e.SourceRequest; // Step 3: Specify the media to be streamed. PlayToSourceDeferral deferral = request.GetDeferral(); request.SetSource(videoplayer.PlayToSource); deferral.Complete(); } // The media will then be streamed to the device chosen by the user in the UI.

demo Camera Capture UI It’s easy to snap a photo or record video by invoking the built-in capture UI.

Use the camera to snap a photo // Application manifest capabilities required to access camera and microphone <Capabilities>     <DeviceCapability Name="webcam" /> <DeviceCapability Name="microphone" /> </Capabilities>

Use the camera to snap a photo <img id="preview" /> <script type="text/javascript">   // Invoke the camera capture UI for taking a photo var captureUI = new Windows.Media.Capture.CameraCaptureUI(); captureUI.captureFileAsync(Windows.Media.CameraCaptureUI.Mode.photo). then(function(capturedItem) { if (capturedItem) { // Display the photo document.getElementById("preview").src = URL.createObjectURL(capturedItem); } }); </script>

demo Simple Encode app Convert media files into different formats, resolutions or trim to a specific timeframe.

Transcode to a different format var outputFile; var vidLib = Windows.Storage.KnownFolders.videosLibrary; var transcoder = new Windows.Media.Transcoding.MediaTranscoder(); // Choose a popular output format and quality var profile = Windows.Media.Capture.MediaEncodingProfile. createMp4(Windows.Media.Capture.VideoEncodingQuality.vga); // Create the file to transcode into, and select the input file to read from vidLib.createFileAsync("output.mp4"). then(function(result) { outputFile = result; return vidLib.getFileAsync("test.wmv"); // Transcode the file }).then(function(inputFile) { return transcoder.transcodeFileAsync(inputFile, outputFile, profile); }).then(function() { // log "done" });

Simple trim var outputFile; var vidLib = Windows.Storage.KnownFolders.videosLibrary; var transcoder = new Windows.Media.Transcoding.MediaTranscoder(); // Specify the start and stop time defining the portion to keep // Create the file to transcode into, and select the input file to read from vidLib.createFileAsync("EditedVacation.mp4"). then(function(result) { outputFile = result; return vidLib.getFileAsync("vacation.mp4"); // Trim the file: profile isn’t changing so profile argument isn’t needed }).then(function(inputFile) { return transcoder.transcodeFileAsync(inputFile, outputFile); }).then(function() { // log "done" }); transcoder.trimStartTime = 900; transcoder.trimStopTime = 6000;

A word on formats

Media Formats and Codecs System-wide File format: MP4 (preferred) or ASF Video: H.264 or VC-1 Audio: AAC, MP3 or WMA Broad support across consumer hardware: great performance and battery life Per-app Your Metro style app can package private plug-ins for its own use (e.g. adaptive streaming)

Recap

With media APIs + Media Elements Take advantage of graphics hardware and devices with little domain knowledge Easy to build tailored user experiences with built-in UI and controls Simple to use media from sources other than web servers, like Local or HomeGroup PC libraries and network attached storage Demanding scenarios like streaming, 3D and DRM are simplified because of the powerful and extensible underlying media platform

Related sessions [PLAT-776T] Your Metro style app, video and audio, Part 2 [APP-788T] Integrating stunning media experiences in XAML [PLAT-777T] Capturing personal photos, video, and audio in Metro style apps [HW-715T] Building a great Metro style device app for your camera [PLAT-778T] Media fundamentals of a communications app [PLAT-783T] Extending the media platform using input, output and processing plug-ins [HW-177T] Building great networked media devices for Play To apps [APP-408T] Integrating with the Windows device experience [HW-329T] Understanding Wi-Fi Direct in Windows 8

Further reading and documentation Design guidelines for Play To receivers Streaming media to devices using Play To Play To & media streaming for Metro style apps Using audio hardware acceleration for ultra-low-power consumption.

thank you Feedback and questions http://forums.dev.windows.com Session feedback http://bldw.in/SessionFeedback

4/23/2017 3:44 PM © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

BACKUP

Use the camera to snap a photo (C#) // Define capture element in XAML <captureElement x:Name="preview" Width="320" Height="240" Margin="10,5,10,5" /> // Initialize mc = new MediaCapture(); Windows.Media.Capture.InitializeOperation opInitialize = await mc.initializeAsync(); // Assign the source preview.Source = mc; // Start the preview Windows.Media.Capture.StartPreviewOperation opStartpreview = await mc.StartPreviewAsync();

Transcode to a different format (C#) // Select the input file to read from StorageFile input = await KnownFolders.VideosLibrary.GetFileAsync("input.wmv"); // Create the file to transcode into StorageFile output = await KnownFolders.VideosLibrary.CreateFileAsync("output.mp4");   // Choose a popular output format and quality MediaEncodingProfile profile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Vga); // Transcode the file MediaTranscoder transcoder = new MediaTranscoder(); await transcoder.TranscodeFileAsync(input, output, profile);

Simple trim (C#) // Select the input file to read from StorageFile input = await KnownFolders.VideosLibrary.GetFileAsync("input.mp4"); // Create the file to transcode into StorageFile output = await KnownFolders.VideosLibrary.CreateFileAsync("output.mp4");   MediaTranscoder transcoder = new MediaTranscoder(); // Specify the start and stop time defining the portion to keep // Trim the file: profile isn’t changing so profile argument isn’t needed await transcoder.TranscodeFileAsync(input, output); transcoder.TrimStartTime = 900; transcoder.TrimStopTime = 6000;