Developing Windows ® CE Multimedia Solutions Using Windows Media Rod Deyo Program Manager Windows CE Platforms Microsoft Corporation 8-307

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

Windows CE DirectX Designed for OEMs to build DirectX and Windows Media enabled platforms using Windows CE Designed for OEMs to build DirectX and Windows Media enabled platforms using Windows CE Adds to Windows CE Platform Builder 2.12 and 3.0 Adds to Windows CE Platform Builder 2.12 and 3.0

Example Applications Alpha-blended displays for advanced set top boxes and automotive systems Alpha-blended displays for advanced set top boxes and automotive systems Viewing Internet multimedia content with Web browser Viewing Internet multimedia content with Web browser WM Audio and MP3 juke boxes and players WM Audio and MP3 juke boxes and players Video time-shift Video time-shift Advertising signage Advertising signage

Products WebTV ™ Microsoft ® TV (MSTV) client WebTV ™ Microsoft ® TV (MSTV) client MSN ™ Web Companion MSN ™ Web Companion Microsoft AutoPC Automotive Adaptation Kit (AAK) Microsoft AutoPC Automotive Adaptation Kit (AAK) Windows-based Terminal (WBT) Windows-based Terminal (WBT) Portable audio player Portable audio player Digital music systems Digital music systems Multimedia internet appliances Multimedia internet appliances

Windows CE Multimedia Architecture Graphics and Audio Drivers DirectShow Filter Graph WM Player Controls Application/Browser DirectDraw/GDI DirectSound/WaveDev Windows Media Components DirectX Components Hardware Sample Drivers Content/HTML Page

Windows CE DirectX DirectDraw ® Graphics DirectDraw ® Graphics  Hardware acceleration  Surface management, tear-free fast blts, overlays, clippers, palettes, video ports DirectSound ® Audio DirectSound ® Audio  Buffer management, software mixing, rate and format conversion, capture DirectShow Multimedia DirectShow Multimedia  Flexible multimedia playback  Multiple file formats and codecs

Windows CE Multimedia Windows Media Player control Windows Media Player control  Run, pause, stop media control  Uses DirectShow Windows Media technologies Windows Media technologies  Streaming with DirectShow filters  Windows Media formats  Audio, video, speech codecs Author and encode once, play anywhere! Author and encode once, play anywhere!

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

Windows CE DirectX DirectX Platform Adaptation Kit (DXPAK) for Platform Builder DirectX Platform Adaptation Kit (DXPAK) for Platform Builder DXPAK 1.0 released DXPAK 1.0 released  Enables hardware accelerated multimedia  DirectDraw 6.1  DirectSound 6.1  DirectShow 6.1  Sample drivers – DDGPE, DSHAL, WaveDev  Platform Builder 2.12  Supports x86, MIPS FP, and SH4 CPUs

Windows CE DirectX DXPAK 1.1 released 5-00 DXPAK 1.1 released 5-00  Enables Windows Media 4.1 client  DirectShow WM streaming filter  WM ASF/ASX Formats  Microsoft WMA and MPEG-4, Sipro ACELP.net, Fraunhofer MP3 codecs  Microsoft Windows Media Player 6.4 ActiveX ® Controls  Platform Builder 2.12 Platform Builder 3.0 version soon! Platform Builder 3.0 version soon!  Additional CPUs  Further performance optimizations

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

DXPAK And Pocket PC Pocket PC Pocket PC  Product SDK  WM Player app, audio codecs only  Local file-based playback only Windows CE DXPAK Windows CE DXPAK  Platform Builder add-on  WM Player control  Audio, video, speech codecs  DirectShow-based Windows Media streaming

DXPAK And Pocket PC Feature DXPAK 1.1 Pocket PC Local File Playback YesYes* WMT Streaming YesNo WMP application NoYes* WMP control YesNo DirectXYesNo WMA codec YesYes* MP3 codec YesYes* MPEG-4 codec YesNo Speech codec YesNo * As part of application

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

Effective Streaming Data is delivered only as needed Data is delivered only as needed Audio and video stream display synchronization Audio and video stream display synchronization Data compression Data compression  Audio (44.1 kHz 16 bit stereo)  1.41 mbps  64 kbps is 22:1  Video (320x bpp 30 fps)  36.9 mbps  300 kbps is 122:1

Effective Streaming Coordinated application, network, and physical layer protocols Coordinated application, network, and physical layer protocols  Network conditions and bandwidth taken into consideration  Packet loss mitigated Client and server work together automatically Client and server work together automatically Simple application controls Simple application controls

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

WM Player Control Windows Media Player Control v6.4 for CE Windows Media Player Control v6.4 for CE  Scriptable from JScript ™  VCR-Like Controls: Play, Stop, Pause, FF, FR  Audio Controls: Balance, Volume, Mute  Information Retrieval: Title, Author, Copyright  Monitoring: Stream status, network reception  Seamless stream switching  Playlists, URL flipping, Markers  Componentized: Toolbar, Statbar, Bkgnbmp

<Object ID="MediaPlayer" CLASSID="CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95" CLASSID="CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95" TYPE="application/x-oleobject" TYPE="application/x-oleobject" WIDTH="320" WIDTH="320" HEIGHT="240" HEIGHT="240" STYLE="position:absolute; STYLE="position:absolute; left:50px; left:50px; top:50px;“ top:50px;“> </Object> No CODEBASE=url attribute – WMP control must be part of WinCE device image Make controls visible WMP Control Embedding

WMP Control HTML Tags Tags have identical behavior to desktop Tags have identical behavior to desktop Tag Tag  Provides full WMP control functionality in IE Tag Tag  Netscape compatibility, limited on IE Tag Tag  Not supported by DXPAK WMP control  Use Windows CE WMPHLPR sample application Tag Tag  MPEG-1, AVI, and QT v2 formats only

WMP Control Host Hosted by OCX 96 compatible container Hosted by OCX 96 compatible container  Internet Explorer compatible browser Control can be either Control can be either  Windowed  Windowless

WMP Control Differences WMP control is NOT the WMP application WMP control is NOT the WMP application  No app, no menus or dialog boxes Subset of desktop WMP v6.4 control Subset of desktop WMP v6.4 control  Smaller, robust control for Embedded devices  Compatibility with existing media content Features not supported Features not supported  PowerPoint ® (PPT) streaming  Hotspots ASF authoring

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

WM Technologies Windows Media Technologies v4.1 for CE Windows Media Technologies v4.1 for CE  Componentized DirectShow source and transform filters  WM Format container and metadata files (ASF, ASX, NSC)  ASF streaming (UDP, TCP, HTTP)  Codecs (WMA, MPEG-4, Sipro ACELP, MP3)  Authentication  Intelligent multiple bit-rate streaming  Event driven stream switching

Header AudioStream Index Video Stream #1 Video Stream #n Script Stream Time WM Technology Formats Advanced Streaming Format (ASF) Advanced Streaming Format (ASF)  Application-level transport and program stream  Stores media and metadata in one file  Contains multiple bit-rate video streams synchronized with audio stream

WM Technology Formats Advanced Streaming Format Advanced Streaming Format  Time based synchronization  Interleaved data optimized for HTTP packets Advanced Stream Redirector (ASX) v3 Advanced Stream Redirector (ASX) v3  Hyperlinks to ASF files  Server and protocol rollover rules  Playlists Windows Media Services Station (NSC) Windows Media Services Station (NSC)  Multicast channels

WM Technology Protocols Multicast Multicast  One packet is broadcast to multiple clients  Requires use of Windows Media Station files (NSC) MMS MMS  TCP connection for command, UDP or TCP for data  Protocol rollover (UDP -> TCP -> HTTP) HTTP HTTP  HTTP packets can be received through firewalls  Does not support protocol rollover Local File Local File

WM Technology Codecs Windows Media Audio v2 Windows Media Audio v2  8 – 48 kHz stereo at 56 – 128 kbps  Near FM quality at 28.8 kbps, near CD quality at 64 kbps Microsoft MPEG-4 Video v2, v3 Microsoft MPEG-4 Video v2, v3  28.8 – 1000 kbps  320x240 resolution video 30 fps at 300 kbps Sipro ACELP.net Sipro ACELP.net  Speech quality 8-16 KHz mono at 5-16 kbps Fraunhofer MPEG-1 Layer 3 Fraunhofer MPEG-1 Layer 3  Near CD quality at 128 kbps

WMT Componentization WMT for CE is componentized into the following modules WMT for CE is componentized into the following modules  MMS streaming  HTTP streaming  File streaming  Broadcast and multi-bit rate streaming  ASX  Individual codecs

WMT Differences ASX v3 ASX v3  PREVIEWDURATION, BANNER, LOGO elements not supported  v1 and v2 formats not supported

Agenda What is Windows CE DirectX ® ? What is Windows CE DirectX ® ? Software availability Software availability DXPAK and Pocket PC DXPAK and Pocket PC Streaming multimedia Streaming multimedia Windows Media Player control Windows Media Player control Windows Media technologies Windows Media technologies DirectShow ® DirectShow ®

DirectShow Powerful, configurable connection architecture for streaming media Powerful, configurable connection architecture for streaming media Automatically handles audio and video synchronization Automatically handles audio and video synchronization Source, transform, and rendering filters Source, transform, and rendering filters Filter graph: a collection of connected filters Filter graph: a collection of connected filters Every filter is pluggable/replaceable Every filter is pluggable/replaceable WM ASF streamer, parser, and decoder filters WM ASF streamer, parser, and decoder filters ASFStreamerASFParser WMAdecoder MPEG-4decoder Audiorenderer Videorenderer Typical Streaming Filter Graph

DirectShow Architecture Hardware DirectDraw/DirectSound File sys ActiveXControls Filter graph manager SourcefilterTransformfilter Clock COMinterfaces Browser/Application Rendererfilter Internet

DirectShow Filters Allocate buffers Allocate buffers Negotiate media types and connections to other filters Negotiate media types and connections to other filters Push data through the filter graph Push data through the filter graph Send events to the filter graph manager Send events to the filter graph manager Synchronize multiple threads Synchronize multiple threads State - run, pause, stopped State - run, pause, stopped

DirectShow Interfaces Filter Graph Manager Filter Graph Manager  Create filter graph - IGraphBuilder  Run, pause, stop - IMediaControl  Seek to position or time – IMediaSeeking  Events (rendering complete) – IMediaEvents  Audio and video control – IBasicAudio, IBasicVideo Filters – usually hidden by FGM Filters – usually hidden by FGM  Control, pins, info – IBaseFilter  Media-related data – IMediaSample  Position, seeking – IMediaSeeking

DirectShow Renderer Filters Audio Audio  Volume and balance controls - IBasicAudio  Uses DirectSound to allow multiple audio streams to be played and mixed at once  Fallback to WaveOut if DirectSound not present Video Video  Video window controls – IBasicVideo  Uses DirectDraw for optimal video playback performance  Will use hardware overlay surfaces if supported  Uses hardware accelerated YUV to RGB color space conversion  Fallback to GDI if DirectDraw not present

DirectShow Differences Not supported Not supported  WDM KS or CSA  IKsPropetySet  Audio or video capture  WM encoding

DirectShow Media Playback – It’s Easy! Declare interface pointers and event code storage Declare interface pointers and event code storage // playmovie.cpp HRESULT PlayMovie(LPTSTR lpszmovie) { // interface pointers we’ll need IGraphBuilder *pGB = NULL; IMediaControl *pMC = NULL; IMediaEvent *pME – NULL; // something to hold event code LONG evCode;

DirectShow Playback CoCreateinstance of FilterGraph object and get IGraphBuilder interface CoCreateinstance of FilterGraph object and get IGraphBuilder interface // instantiate a filter graph as in-proc server if (FAIL(CoCreateInstance(CLSID_FilterGraph, NULL,CLSCTX_INPROC,IID_GraphBuilder, (void**) &pGB))) return E_FAIL; No filter graph exists yet since media type and format are unknown to GraphBuilder No filter graph exists yet since media type and format are unknown to GraphBuilder

DirectShow Playback Get IMediaControl to run filter graph and IMediaEvent to wait for end of stream Get IMediaControl to run filter graph and IMediaEvent to wait for end of stream // get interface to control graph if(FAIL(pGB->QueryInterface(IID_MediaControl, (void **) &pMC))) return E_FAIL; // get interface for done event if(FAIL(pMC->QueryInterface(IID_IMediaEvent, (void **) &pME))) return E_FAIL; Build the filter graph from the media and format Build the filter graph from the media and format // build the filter graph if(FAIL(pGB->RenderFile(lpszMovie, NULL))) return E_FAIL;

DirectShow Playback Start playing the media and block application waiting for EC_COMPLETE from renderer filter Start playing the media and block application waiting for EC_COMPLETE from renderer filter // play the file pMC->Run())) // and block indefinitely until completed pME->WaitForCompletion(INFINITE,&evCode))) // release interfaces pGB->Release(); pMC->Release(); pME->Release(); } Release interfaces Release interfaces

Summary OEMs can now build DirectX and Windows Media enabled platforms using Windows CE and Platform Builder OEMs can now build DirectX and Windows Media enabled platforms using Windows CE and Platform Builder Componentized Windows CE versions of Componentized Windows CE versions of  Windows Media Technologies v4.1  Windows Media Player Control v6.4  DirectShow 6.1

Further Information White papers White papers  Multimedia Streaming on Windows CE  DirectX and Windows Media Technologies on CE Windows CE DX Website Windows CE DX Website  eatures/directx.asp eatures/directx.asp eatures/directx.asp MSDN Developer Center MSDN Developer Center  

Questions?