3-D virtual worlds for broadcast television IST project MADISON Thierry Keutgen Senior Application Engineer DELTATEC

Brussels, 28 th November 2003 Television ? Internet ?

Brussels, 28 th November 2003 Television ? Internet on TV …… TV on Internet Is it the issue ? Television is evolving in a new framework New business models: pay TV, pay-per-view, games, T-commerce, … Content to multiple types of supports

Brussels, 28 th November 2003 How do we make it happen ? Deltatec is a High-tech explorer for its customers Specialist in broadcast This revolution is an opportunity It’s far from trivial … R&D work needed

Brussels, 28 th November 2003 European IST Project With real-world objectives it is a nice framework to Meet technical experts Meet industrial end-users Explore new technical fields Get a technical edge for our customers

Brussels, 28 th November 2003 Idea – Issues - Solutions 3-D Multi-user Client-to-client interaction Broadcast Latency Distributed application No demonstrator for the technology IEEE DISMPEG-4

Brussels, 28 th November 2003 MPEG-4 MPEG-4 allows to wrap 3-D synthesis information into more general mulitmedia structures where video, audio, 3-D and image objects coexist and interact NOT low bit rate video NOT encoding / decoding NOT internet BROADCAST

Brussels, 28 th November 2003 Architecture ContentProvider Set-Top Box Satellite/ Cable receiver Operating system Application Software Network Layer MPEG 4 Subsystem DataInjectorDistributor SimulationServer Internet TV set UserInteractiondevices

Brussels, 28 th November 2003 Satellite/ Cable receiver Operating system Application Software Network Layer Set-Top Box ContentProvider MPEG 4 Subsystem DataInjectorDistributor SimulationServer Internet TV set UserInteractiondevices Architecture UserInteractiondevices Application Software MPEG 4 Subsystem TV set

Brussels, 28 th November 2003 Satellite/ Cable receiver Operating system Application Software Network Layer Set-Top Box ContentProviderDataInjectorDistributor SimulationServer Internet UserInteractiondevices Architecture MPEG 4 Subsystem TV set MPEG 4 Subsystem TV set Application Software Internet SimulationServer DataInjectorDistributor Satellite/ Cable receiver UserInteractiondevices Network Layer

Brussels, 28 th November 2003 Partners Archetypon Boostworks Canal+ Belgique Deltatec EPFL Memondo Norsk Regnesentral

Brussels, 28 th November 2003 Results produced Simulation server – IEEE DIS-like message protocol MPEG-4 over MPEG-2 injector Network acceleration layer Prototype STB

Brussels, 28 th November 2003 Channel selection

Brussels, 28 th November D virtual world

Brussels, 28 th November 2003 Interaction with context or other users

Brussels, 28 th November 2003 Data Injector ContentProvider Set-Top Box Satellite/ Cable receiver Operating system Application Software Network Layer MPEG 4 Subsystem DataInjectorDistributor SimulationServer Internet TV set UserInteractiondevices DataInjector

Brussels, 28 th November 2003 MPEG-4 over MPEG-2 injector Different options to inject data over a MPEG-2 TS : Raw data over a MPEG-2 TS IP over MPEG-2 TS MP4 file hinting and injection «True» MPEG-4 SL-Packets embedding through a DMIF interface

Brussels, 28 th November 2003 MPEG-4 over MPEG-2 injector Based on a PCI board with: 2 DVB ASI transmitters and 2 receivers Reconfigurable I/O controller to process MPEG-2 TS Applications: PID filtering and TS processing DVB MPEG-2 video servers Interactive Television (iTV) IP data insertion/Data broadcasting Video over IP (VoIP) Video-on-Demand (VoD)

Brussels, 28 th November 2003 SL-Packets injector Data Injector TS Packet Builder PSI Monitoring and Update Simulation Server DVB Server DVB Multiplexer FlexMux Core Embedder MPEG-2 TS ASI MPEG-4 descriptors MPEG-4SL-Packets TCP/IP

Brussels, 28 th November 2003 MP4 file hinter and injector Data Injector TS Packet Builder Simulation Server Hinter PSI Monitoring and Update DVB Server MPEG-2 TS ASI MP4file MP4 fileTCP/IP DVB Multiplexer Core Embedder Local MP4 file

Brussels, 28 th November 2003 Prototype Set-Top Box ContentProvider Set-Top Box Satellite/ Cable receiver Operating system Application Software Network Layer MPEG 4 Subsystem DataInjectorDistributor SimulationServer Internet TV set UserInteractiondevices Set-Top Box

Brussels, 28 th November 2003 Prototype STB - MadBox Based on a PC running Windows 2000 QAM receiver board and demultiplexer MPEG-4 DMIF for Local files Internet Broadcast Customised housing Everything managed through a TV remote control

Brussels, 28 th November 2003 Broadcast DMIF

Brussels, 28 th November 2003 Broadcast DMIF Instance BroadcastInstance Class Callback Methods DAI Primitives Top-level terminal layers Sync Layer Application Layers (Decoders + Compositor) Delivery Layer Delivery Medium (MPEG-2 Transport Stream) FlexMux DMIF Filter Local Files DMIF Network DMIF Broadcast DMIF Low-level TS access functions MPEG-2 Parsing functions MPEG-4 Parsing functions

Brussels, 28 th November 2003 MPEG-4 for broadcast… for now MPEG-4 is not just low bit-rate video Authoring tools are missing 3-D is really a step forward MPEG-4 and 3-D are very demanding for the STB Enhancements are good, content is key

Brussels, 28 th November 2003 Next R&D step We are ready for MPEG-4 in broadcast We want to go further IST ENAMORADO project: Production of multimedia content for live events: AR, metadata, transmission, … Delivery to multiple types of mobile clients Focus on MPEG-7, R&D on MPEG-21 Deltatec, Canal+, Siemens Mobile, …

Brussels, 28 th November 2003 Get concrete

Brussels, 28 th November 2003 DELTA-asiDELTA-sdi From a MADISON prototype … to a range of products

Brussels, 28 th November 2003 Get concrete STB processing power is market-driven … Manufacturers still to endorse the technology Let’s demonstrate the technology on programs that pay In football: ad insertion, synthetic replay, new camera views, T-commerce, virtual after-match pub, …

Brussels, 28 th November 2003 Thank you Thierry Keutgen