Multimedia System Dave Chung 9/94

Technology Trends Multimedia workstations with audio and video processing capability Multimedia workstations with audio and video processing capability –Tremendous improvements in CPU processing power –Special purpose compression engines for video and audio High capacity, high performance storage devices High capacity, high performance storage devices –Availability of magnetic disks with several gigabyte capacity, tremendous increase in data transfer rate due to disk array technology –Technology for large optical storages servers evolving rapidly

Technology Trends (cont.) High speed fiber optic networks and fast packet switching technology High speed fiber optic networks and fast packet switching technology –Networks with hundreds of Mbps already in place, gigabit networks also being deployed –Broadband to the home (DSL, cable modem)

Deployment of Multimedia Application Content creation Content creation –Creation of actual video-audio clips and images that viewed by users –Artist’s job Application creation Application creation –Integrate audio and video clips into applications Designing and building the systems for deploying multimedia applications and delivering multimedia content. Designing and building the systems for deploying multimedia applications and delivering multimedia content.

Service Vision and Applications Residential services: Residential services: –Switching access television, video-on-demand –Video phone/conferencing systems –Video advertisement and shopping –Self-paced education Business services: Business services: – Corporate education – Desktop multimedia conferencing and multimedia

Service Vision and Applications -Continued Education: Education: –Multimedia repository of available classes, videos, books, … –Access to digital multimedia library over high speed networks Science and technology: Science and technology: – computational prototyping and scientific visualization –Astronomy and environmental science studies: efficient access to large number of satellite images.

Service Vision and Applications - Continued Medicine: Medicine: –Diagnosis and treatment – support queries like where was the tumor found (I.e., detection and location), what happened to the tumor and assessment of therapeutic response (I.e., temporal analysis), etc.

Hierarchy of Distributed Applications Class 1: Information retrieval Class 1: Information retrieval –Distance learning Class 2: Information retrieval and processing Class 2: Information retrieval and processing – digital library of satellite imagery, subsurface modeling Class 3: Interactive systems Class 3: Interactive systems – Education and training for fire fighters, command and control.

Design Difficulties Real-time performance requirements Real-time performance requirements Large sizes of information objects Large sizes of information objects –Speech 8000 samples/s8 K Bytes/s –CD Audio44100 samples/sec 176 K Bytes/s 2 bytes/sample –Satellite180*180 km 2, 600 MB/image imagery30 m 2 resolution (60 MB if compressed) imagery30 m 2 resolution (60 MB if compressed) –NTSC video30 frames/s30 Mbytes/s 640*480 pixels, (2-8 Mbps if compressed) 3 bytes/pixel

The medium is the message. Themes of the course: Themes of the course: –In multimedia communication, how the message arrives is often as important as what the message is. –The two are entangled. –Emphasize more on networked multimedia. How the message arrives: Networking How the message arrives: Networking What the message is: Multimedia Encoding What the message is: Multimedia Encoding

Example of Video Streaming

Six Key Areas of Streaming Media 1. Video and voice compression –Scalable and non-scalable 2. Application-layer QoS control Cope with –varying network conditions and –Different presentation quality –Techniques: congestion control and error control 3. Continuous media distribution service –Network support to reduce transport delay and packet loss ratio. –Techniques: network filter, application-level multicast, and content replication.

Six Key Areas of Streaming Media (cont’d) 4. Streaming servers –Process media data under timing constraints –Support interactive control operations. –Retrieve media components in a synchronous fashion. –Consists of »Communicator (e.g.,transport protocol) »Operating system »Storage system

Six Key Areas of Streaming Media (cont’d) 5. Media synchronization mechanisms –Present various media streams in the same way as they were originally captured, e.g., lip-sync. 6. Protocols for streaming media –Provide services such as network addressing, transport, and session control. –E.g., IP, UDP, RTSP

Course Objectives Overview of multimedia systems Overview of multimedia systems Overview of Media Compression Technologies Overview of Media Compression Technologies Focus on current multimedia services Focus on current multimedia services –Video on demand –Voice over IP Focus on current streaming technologies Focus on current streaming technologies –Overlay streaming –Wireless multimedia

Grading Class attendance 15% Class attendance 15% Presentation 20% Presentation 20% Term Report 20% Term Report 20% Homework 20% Homework 20% Final: Take home examination 25% Final: Take home examination 25%

Textbook Class handout Class handout Papers Papers