Chapter 2 Enabling Technologies Multimedia Systems

Key Points All types of media are represented digitally as patterns of bits. Analogue signals must be digitized, which comprises sampling and quantization. The Sampling Theorem states that, if the highest frequency component of a signal is at f h, the signal can be properly reconstructed if it has been sampled at a frequency greater than 2f h. Contemporary personal computers are capable of playing back most types of media. Multimedia production places much greater demands on computer systems.

Key Points Multimedia production requires the use of many different software applications, including authoring systems. Networks, and the Internet in particular, offer valuable opportunities for distributing multimedia, but they also present formidable technical difficulties. The most popular form of online multimedia delivery is the World Wide Web, which uses the Hypertext Transfer Protocol (HTTP) to organize its communications. Various formal and informal standards govern multimedia data, especially in the context of networks.

Production and consumption of digital multimedia depends on ability of digital computer. –Media data in digital form Program can be used to change, combine, store and display media of all types. –Distribution Networks Removable storage: CDROM, DVD

Computers are not only devices that can manipulate digital data. –DVD, set-top box, CD player Computer –Full interactivity Future: multimedia player –Cheaper than computer

Digital Representations Bits, bytes, words. –Data structures –Programs Digitization –Analogue and digital representations –Analogue to digital converters –Sampling and quantization –Digital signals are much more robust than analogue ones, and do not suffer degradation when they are copied, or transmitted over noisy media.

Sampling and Quantization Sampling –Measure the signal’s value at discrete intervals –Sampling rate Quantization –Restrict the value to a fixed set of levels –Quantization levels Analogue to digital converter (ADC)

DAC Sample and hold Fig. 2.4 –Not a good approximation Such a signal is passed to an output device such CRT or speaker, these discontinuities can be smoothed out.

Frequency –Cycles per second (cps) –Hertz (Hz): KHz, MHz –Velocity = frequency * wavelength –FFT Any periodic waveform can be decomposed into a collection of different frequency components. –Frequency Temporal, spatial Fig. 2.6, vary periodically in space –Fig. 2.7, Frequency components of a square wave –DC component, f=0 –Higher frequency: abrupt transitions –Fig. 2.8, square wave in frequency domain

Filter Remove certain frequencies High pass filter Low pass filter

Sampling Theorem If the highest frequency component of a signal is at f, the signal can be properly reconstructed at the sampling rate 2*f. This limiting value is known as the Nyquist rate. Fig. 2.9 The hearing range of the average person is approximately 20 Hz to 17kHz. (Sounds above this range is called ultrasonic sounds) Audio cards use 44.1kHz with sample sizes 8 and 16 bits

Undersampling and Quantization noise Undersampling –Sampling rate is less than the Nyquist rate. Aliasing –In images, it is usually seen in the form of jagged edges. Quantization noise –Number of quantization levels Image: grey levels Sound

Hardware Requirements Multimedia PC (MPC): Level 3

Hardware PDA –Less powerful devices –Difficult to achieve the same performance of PC –Limitation on the amount of storage –Often based on cellular telephone network, less bandwidth –Challenge of web designers Accessible to powerful PC and to PDA Multimedia architecture must be scaleable –Transparently to devices with different capabilities

High quality multimedia production are demanding than those for its consumption –Special-purpose graphics acceleration hardware and input devices for video and audio –High-speed disks, video capture card, specialized peripherals –High-powered, SGI: 3D animation and video effects, common in film and TV –Special-purpose array processing in Pentium and PowerPC (MMX, AltiVec) Blur the distinction between high-end PC and workstations

Producing multimedia content –Processing power –High-speed data buses –Large memories –Powerful graphics boards Content preparation –Substantial more storage than finished product E.g. image layers up to 100 High quality audio Compression only for final delivery

Speed of data transfer to and from disks is a serious limiting factor Digital video: 5Mbytes to 30 Mbytes IDE, SCSI-1 cannot cope with SCSI-2 supports 40Mbytes Ultra SCSI-2: 80Mbytes Firewire, IEEE 1934: 50 Mbytes, DV cameras Current disk speed for a dedicated computer: adequate performance Higher performance of disk for central server –RAID array: Redundant array of inexpensive disks –Improved performance in parallel AV disk

RAID 8 levels for different degrees of performance and fault-tolerance RAID 0: no protection against disk failure –If one disk fails, whole array fails RAID 1: mirrors disk

Specialized Peripherals Graphics tablet with a pressure-sensitive pen Large high-resolution monitor Two monitors: preparing images and display images Scanners Digital cameras Video camera and sound recording –The equipment may itself be digital and produce digital output that can be sent to computer over Firewire. –Video and audio capture from analogue equipment

Software The essence of multimedia is the combination of different media elements through computer programs. Different applications for different media –Graphics: Image editing, painting, drawing –Text: editors, layout –Video: capture, editing, postproduction –Motion graphics and animation –Sound: recording, editing, effects –Music synthesizer and sequencers Writing program for combination: advanced programming skills An authoring system a software that permits a user to combine and organize these media elements (example, Director)

Authoring Systems May be based on –Layout model with markup language –Timeline Scripting language

Multimedia Production A host of software tools and skills Require a team Low end of multimedia software, web page design –Non-specialists –Hide technical details such as HTML tags and JavaScript code –Wizards, assistants guide the production process

Multimedia Presentation A unified interface and manipulated a single program –Production: different software tools and team input WWW –Define a format (in markup language, HTML, XML) that can accommodate different media and view it using a dedicated browser Define an architecture (format) and API provides a rich set of functions to manipulate data in that format –QuickTime Deliver in a stand alone form, needs no additional software –Director Three approaches are not necessarily separate or incompatible –QuickTime movies in WWW pages –Director movies in Shockwave and embedded in WWW pages Fig. 2.11

Networks TCP/IP ISP, Internet Service Providers Dial-up connection, 56 kbps –V90 standard: downstream=56k, 33.6k in opposite direction –34-48 kbps are more realistic –Maximum bandwidth using analogue connections Digital telephone, ISDN –A digital channel=64 kbps –Basic rate= two connections=128kbps –Mid-1980

ADSL –Downstream: up to 6.1 Mbps –Upstream: up to 640 kbps Cable modem –500 kbps to 10 Mbps T1=1.544 Mbps, T3= Mbps LAN –10 base T Ethernet, 10 Mbps –100 base T, 100Mbps Table 2.1

Networks

Clients and Servers Server listen on a communication channel for requests from clients. Server receives a request, it sends a response. The requests and responses conform to a protocol. HTTP: HyperText Transfer Protocol HTML: markup language Browser: Netscape, IE FTP Real-time data streaming for audio and video

Dynamically Generate Web Pages Dynamically generate Web pages from database CGI, common gateway interface Microsoft, Active Server Pages, ASP Apple, WebObjects

MIME Types OS –Extension of file’s name Multipurpose Internet Mail Extension –Content-type: type/subtype Type: broad indication of the sort of data –text, image, audio, video –model for 3D model data such as VRML –message: –application: binary data, e.g. application/postscript Subtype: precise format –HTML, GIF, AIFF –x-, experimental, video/x-msvideo

Standards Define interfaces, file formats, markup languages, network protocols, and so on Three organization –ISO –International Electrotechnical Commission (IEC) –International Telecommunication Union (ITU) ISO, IEC: commercial companies Countries –ANSI: USA –BSI: UK –DIN: Germany

Rapidly changing environment of computers, networks, multimedia –Standard often obsolete before they have passed through all the stages. Internet standard –Semi-formal –Arpanet + NSFNET = TCP/IP Company production –Postscript –Quicktime –SVG (W 3 C) < Flash