1. Diepold - Multimedia at Your Fingertips 1 Open Standards – the Fuel for Ubiquitous Multimedia Deployment Klaus Diepold Technische Universität München Munich, Germany Multimedia at Your Fingertips Delft, November 2006

2. Diepold - Multimedia at Your Fingertips 2 Open Standards Open Standards vs. Proprietary (Closed) Standards I like Standards – there are some many to choose from Standards for Mulitmedia  MPEG Standards (ISO/IEC)

3. Diepold - Multimedia at Your Fingertips 3 Standardisation – the MPEG Way Standard: The process by which a group of individuals recognise the advantage of all doing certain things in an agreed way and codify that agreement.

4. Diepold - Multimedia at Your Fingertips 4 Standards Help Markets Mature Compatibility over device generations Interoperability between vendors Minimize dependency on a single vendor No one company can service all the needs No one implementation is best for every use Standards facilitate competition on the product or service level Benefit for the Consumer

5. Diepold - Multimedia at Your Fingertips 5 Standards Achieve High Quality Specification captures best core technologies Standards provide many degrees of freedom Law of large numbers will prevail Cast of thousands working in the open to improve quality Economy of scale

6. Diepold - Multimedia at Your Fingertips 6 Standards Survive the Test of Time NTSC – 1953 PAL – 1963 VHS -1976 (industry standard though) Audio CD - 1980 TCP/IP - 1981 MPEG-1, MP3 - 1992 MPEG-2 – 1994 DVB – 1994 DVD – 1995

7. Diepold - Multimedia at Your Fingertips 7 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 JPEG“Still images” MPEG-1 “Digital Storage Media” MPEG-2 “Digital TV and DVD” MPEG-4 “Interactive Multimedia Objects” MPEG-7 “Interface for Content Description” MPEG-21 “Multimedia Framework” JPEG2000 “Still images” H.262 H.262 “Digital TV and DVD” H.261 H.261 “p  64” H.263 H.263 “Low Bitrate Coding (LBC)” H.264 H.264 “Digital TV and DVD”MPEG-4 Part 10 (AVC) H.263+ H.263+ “Very LBC” ISO/IEC Standards ITU-T Standards H.120 H.120 1984/1988 T.800 T.81 JPEG-LS “Lossless Coding of Still Images” T.87 MPEG-A “Multimedia Application Formats” New !

8. Diepold - Multimedia at Your Fingertips 8 MPEG-A: Multimedia Application Formats Large Number of diverse MPEG Standards difficult to oversee all the technical options evaluate all their relative benefits. Collection of tools is no longer enough Complete solutions are needed Industry consorita define application specific combinations of standardized tools DVB, DVD,… 3GPP… Etc. New !

9. Diepold - Multimedia at Your Fingertips 9 Multimedia Application Formats: The Super-Format Challenge Objective for MAFs Provide combinations of MPEG tools Provide a complete multimedia solutions Satisfy the needs of certain classes of applications Consider the need for fast standardisation Application specific super-formats New !

10. Diepold - Multimedia at Your Fingertips 10 Multimedia Application Formats: Concept

11. Diepold - Multimedia at Your Fingertips 11 The MAF Story So Far … (ISO/IEC 23000) 1: Purpose of Multimedia Application Formats (TR) 2: Music Player - includes Music Player and Protected Music Player (FCD in October 2006) 3: Photo Player (FDIS in October 2006) 4: Musical Slide Show (FCD in July 2006) 5: Media Streaming (CD in October 2006) 6: Professional Archival (WD in October 2006) 7: Open Release (CD in October 2006) 8: Portable Video Player (WD in October 2006) 9: Digital Multimedia Broadcasting (WD in October 2006)

12. Diepold - Multimedia at Your Fingertips 12 Music Player MAF

13. Diepold - Multimedia at Your Fingertips 13 Music Player MAF Digital music libraries Combination of audio, metadata, and images Handling audio data, metadata, and images for individual pieces of music collections of music complete albums or playlists.

14. Diepold - Multimedia at Your Fingertips 14 Music Player Format: Main Tools MP3 audio coding (MPEG-1/2 Audio Layer III) MPEG-7 metadata (corresponding to ID3-Tags) MPEG-21 Digital Item Declaration (DID) ISO Media Base, MPEG-4 and MPEG-21 File Formats JPEG images

15. Diepold - Multimedia at Your Fingertips 15 Creating Music Player Files …

16. Diepold - Multimedia at Your Fingertips 16 Playing Music Player Files …

17. Diepold - Multimedia at Your Fingertips 17 Photo Player MAF FDIS in October 2006

18. Diepold - Multimedia at Your Fingertips 18 Photo Player MAF Digital photos as personal media; Amount of digital photos is difficult to be managed and accessed … Difficult to search and find A standard file format for easy management organization of digital photos

19. Diepold - Multimedia at Your Fingertips 19 Photo Player MAF: Main Tools MPEG ISO Base Media and MPEG-4 File Formats MPEG-7 Systems (Binary Format for Metadata - BIM) MPEG-7 Multimedia Description Schema metadata MPEG-7 Visual metadata Dominant Color Descriptor, Scalable Color Descriptor Color Layout Descriptor, Color Structure Descriptor Edge Histogram Descriptor, Homogeneous Texture Descriptor JPEG images EXIF (Exchangeable Image File Format) acquisition metadata

20. Diepold - Multimedia at Your Fingertips 20 Photo Player MAF Capabilities … Photo Categorization Situation-based Photo Clustering

21. Diepold - Multimedia at Your Fingertips 21 Protected Music Player MAFs FCD in October 2006

22. Diepold - Multimedia at Your Fingertips 22 Protected Music Player MAFs Content protection for Music Different approaches to protection different understandings of interoperability. Different protection approaches signaled using MPEG-21 Intellectual Property Management and Protection (IPMP) Components tools.

23. Diepold - Multimedia at Your Fingertips 23 Music Player Format: Main Tools MP3 audio coding (MPEG-1/2 Audio Layer III) MPEG-7 metadata (corresponding to ID3-Tags) MPEG-21 Digital Item Declaration (DID) ISO Media Base, MPEG-4 and MPEG-21 File Formats JPEG images MPEG-21 Rights Expression Language (REL) MPEG-21 IPMP Components Outside MPEG: AES128 encryption

24. Diepold - Multimedia at Your Fingertips 24 Musical Slide Show MAF FCD in October 2006

25. Diepold - Multimedia at Your Fingertips 25 Musical Slide Show MAF Slide shows, karaoke, story telling, photo-music albuns, practice materials of foreign languages, etc. Based the Music Player MAF and the Photo Player MAFs + text and animation + synchronization capabilities.

26. Diepold - Multimedia at Your Fingertips 26 Musical Slide Show Content …

27. Diepold - Multimedia at Your Fingertips 27 Musical Slide Show MAF: Main Tools MPEG-1/2 Layer III (MP3) audio ISO Base Media and MPEG-4 File Formats MPEG-4 Streaming Text Format for timed text MPEG-4 LASeR, Mini profile for animation and composition Lightweight Animated Scene Representation MPEG-7 metadata MPEG-21 DID and File Format JPEG images

28. Diepold - Multimedia at Your Fingertips 28 Media Streaming MAF CD in October 2006

29. Diepold - Multimedia at Your Fingertips 29 Media Streaming MAF Proposed by the Digital Media Project (DMP) Consortium Public broadcasting in the digital world Commercial broadcasting in the digital world Internet streaming via Multicast Pay TV IPTV Internet Television Storage and subsequent play of content in a domain

30. Diepold - Multimedia at Your Fingertips 30 Media Streaming MAF: Main Tools MPEG-2 and MPEG-4 IPMP-X MPEG-21 DID and Digital Item Identification (DII) MPEG-21 IPMP Components MPEG-21 REL ISO Base Media and MPEG-21 File Formats MPEG-21 Binary Format MPEG-21 Digital Item Streaming (DIS) Supplemental media technologies, e.g. domain management

31. Diepold - Multimedia at Your Fingertips 31 Open Release MAF CD in October 2006

32. Diepold - Multimedia at Your Fingertips 32 Open Release MAF “Public Domain” Content / Non-Commercial Contene e.g., collections of photos, teaching lectures, teasers, advertizing material, etc., Distribution of content for free – without loosing track of the usage e.g. automatically sending email back to the file provider … Express intention of Creative Commons (CC) licenses No enforcement

33. Diepold - Multimedia at Your Fingertips 33 Open Release MAF: Main Tools Subset of MPEG-21 DID and MPEG-21 DII Subset of MPEG-21 REL Subset of MPEG-21 Rights Data Dictionary (RDD) MPEG-21 File Format MPEG-21 Binary Format MPEG-21 Event Reporting

34. Diepold - Multimedia at Your Fingertips 34 Professional Archival MAF WD in October 2006

35. Diepold - Multimedia at Your Fingertips 35 Professional Archical MAF Digital backup and direct delivery of recorded music Compression and archiving intermediate data generated by sound editing tools Preservation & archiving Packing file format for non-audio waveforms

36. Diepold - Multimedia at Your Fingertips 36 Professional Archical MAF: Main Tools MPEG-4 Part-3: Audio Lossless Coding (ALS) MPEG-4 Part-3: Lossless coding of oversampled audio (DST) ISO Base Media File Format MPEG-4 File Format MPEG-7 MDS metadata MPEG-21 DID MPEG-21 DII MPEG-21 File Format

37. Diepold - Multimedia at Your Fingertips 37 Portable Video Player MAF WD in October 2006

38. Diepold - Multimedia at Your Fingertips 38 Portable Video Player MAF Standardised “Video iPod” Playback of personal recordings or home videos Playback of content sold on disk media (mini-DVD) Playback of content offered via the Internet

39. Diepold - Multimedia at Your Fingertips 39 Portable Video Player MAF: Main Tools ISO Base Media, MPEG-4 Advanced Video Coding (AVC) Baseline Profile MPEG-4 HE-AAC Profile, Level 2, Stereo MPEG-4 BSAC Audio MPEG-4 Streaming Text Format for subtitles MPEG-4 LASeR (e.g. for menu screens) MPEG-7 Metadata JPEG still images of e.g. movie posters MPEG-21 File Format and DID MPEG-21 based Content Protection Mechanisms

40. Diepold - Multimedia at Your Fingertips 40 Digital Multimedia Broadcasting MAF WD in October 2006

41. Diepold - Multimedia at Your Fingertips 41 Digital Multimedia Broadcasting MAF Storage of mobile broadcast content IP media services User-creative mobile broadcasting content Mobile TV Services in Korea (DMB)

42. Diepold - Multimedia at Your Fingertips 42 Digital Multimedia Broadcasting MAF: Main Tools ISO Base Media File Format Video: Advanced Video Coding (AVC) Audio: MPEG-1 Layer-2, MPEG-2 AAC+, MPEG-4 ER-BSAC MPEG-4 HE-AAC Systems: IOD/OD, BIFS, MPEG-4 File format, AVC File format DMB Data Services Content description metadata: MPEG-7, TV-Anytime etc. User metadata: MPEG-7, TV-Anytime, etc MPEG-21 IPMP, REL, etc.

43. Diepold - Multimedia at Your Fingertips 43 Surveillance MAF Follows significant interest on surveillance applications by many companies. Currently, this MAF proposes to include: MPEG-2 Video, MPEG-4 Visual, AVC,SVC MPEG-4 Audio (scalable coding) MPEG-2/MPEG-4 Systems (media streaming) MPEG-7 Simple profile MPEG-7 Visual and Audio MPEG-21 DID, DII, DIA MPEG-21 IPMP Components and REL MPEG-X File Format Cryptographic functions from outside

44. Diepold - Multimedia at Your Fingertips 44 Digital Video/Cinema MAF Addresses digital entertainment, especially digital cinema. Currently, this MAF proposes to include: MPEG-2 Video (main profile) MPEG-4 AVC/SVC MPEG Audio MPEG-7 Visual MPEG-21 File format MPEG-21 DID/ DII MPEG-21 IPMP/REL/RDD MPEG-21 Digital Item Adaptation

45. Diepold - Multimedia at Your Fingertips 45 Final Remarks A number of MAFs are ready for application development  ubiquitous multimedia deployment MPEG-A represents a new layer for interoperability. MPEG-A provides complete solutions MPEG-A teaches the usage of MPEG standards e.g. MPEG-7 and MPEG-21. However, the use of Standards is challenged by Patent Issues

46. Diepold - Multimedia at Your Fingertips 46 Useful References MAFs Overview Document available at http://www.chiariglione.org/mpeg/ To probe further: K. Diepold, F. Pereira, W. Chang. MPEG-A: Multimedia Application Formats. IEEE Multimedia, Volume 12, Issue 4, Page(s): 34 – 41, Oct.-Dec. 2005

47. Diepold - Multimedia at Your Fingertips 47 Thanks for your attention ! Thanks to Prof. Munchurl Kim and Prof. Fernando Pereira for some slides.