1. Judith Molka-Danielsen, Oct. 02, 2000
[Class 6: Multimedia Management] IN350: Document Handling and Information Management
Judith Molka-Danielsen, Oct. 02, 2000
1/14/2019

2. Multimedia Services: Processes of Interest
1. Capture
Synchronize audio, video, and other media streams
Compression in real-time
2. Presentation
Decompression in real-time
Low end-to-end delay (< 150ms )
3. Storage
Quick access for forward and backward retrieval
Random access (< 1/2 s)
1/14/2019

3. Multimedia Services need Compression
Typical Example of a Video Sequence
25 images/sec
3 byte/pixel
Image resolution 640*480 pixel
Data rate= 640*480*3*25= byte/s = ~23Mbytes/s
Therefore, Compression is required
1/14/2019

4. General Requirements for Real Time Multimedia Services
Low Delay
Low Complexity of compression algorithm
Efficient Implementation of compression algorithm
High quality output after decompression
1/14/2019

5. Compression Coding Categories
Entropy Coding - Run Length Coding, Huffman Coding,
Adaptive Algorithms, Arithmetic Coding (some examples discussed in previous lectures.)
Source Coding -
Prediction, DCT (using FFT-fourier frequency transform), subband coding, quantization
Hybrid Coding - JPEG, MPEG, H.261, DVI RTV,
DVI PLV
1/14/2019

6. Source Coding
This type of coding tries to take into account the specific data characteristics (in a photo for example) and the human sensitiveness (vision) to that type of data.
Luminance: is the overall response of the total light energy dependant on the wavelength of the light. The human eye is more sensative to certain wavelengths than to others. (more to:yellow, green; less to: red, violet).
Lightness: is the “whiteness” of a specific object
1/14/2019

7. Discrete Cosine Transformation (DCT)
Uses source coding: going from the spacial domain to the frequency domain
typical pictures have minimal changes in colour or luminance between two adjacent pixels
the frequency representation describes the amount of variation. There are small coefficients for high frequencies. The coefficients of the low frequencies contain most of the information.
1/14/2019

8. Discrete Cosine Transformation (DCT)
Assumption: changes of colour and luminance between adjacent pixels is low
DCT can be used for photos
DCT can not be used for vector graphics or two coloured pictures (black text, white background)
DCT is a lossless technique: But information can be lost at the quantization step. The divisor selected for the matrix can reduce precision.
1/14/2019

9. Hybrid Coding for images, video and audio
Basic Encoding Steps:
Image Preparation - resolution, frame rate. Video:lossy, Audio:lossless
Image Processing - DCT. Lossy or lossless.
Quantization - analog to digital conversion of data flow.
Entropy Encoding - Run Length, Huffman. Lossless.
1/14/2019

10. JPEG (Joint Photographics Experts Group)
Coding/Decoding of continuous-tone still images
JPEG allows Different resolution of individual components Interleaved / non-interleaved coding of components Lossless / lossy modes
loss of information by DCT is minimal
loss of information by quantisation may be significant, but amount of lost data is adjustable
Quantisation:
the JPEG standard defines two default quantization tables, one for luminance and one for chrominance
1/14/2019

11. MPEG (Moving Pictures Experts Group)
Algorithms for Compression of Audio and Video:
A movie is presented as a sequence of frames, the frames typically have temporal correlations
If parts of frame #1 do not change then they do not need to be encoded in Frame #2 and Frame #3 again.
Moving vectors describe the re-use of a 16x16 pixel area in other frames.
New areas are described as differences
MP3 is just the 3rd Layer of the MPEG standard used for audio. Layer 1 and 2 are for video.
1/14/2019

12. MPEG frame types
MPEG frame types with different temporal correlations:
(I-Frames) Intracoded full frames are transmitted periodically.
(P-Frames) predictive frames and (B-Frames) bidirectional frames are used for temporal correlations. They can look to the “past” using (P-Frames) and to the “future” using (B-Frames).
1/14/2019

13. Other Video Compression standards
H.261 (Px64):
Video compression for video conferences.
Compression in real time for use with ISDN.
Compressed data stream= px64kbps where p=1 to 30. There are 2 resolutions:
CIF (Common Intermediate Format)
QCIF (Quarter Intermediate Format)
Digital Video Interactive (DVI):
Intel/IBM technology for digital video.
DVI PLV (Production Level Video, for VCR),
DVI RTV (Real Time Video, lower quality).
1/14/2019

14. Requirements for real time systems
Required characteristics for Real time systems:
The processing time for all system components must be predictable
Reservation of all types of resources must be possible
Short response time for time critical processes
The system must be stable at high load
1/14/2019

15. Problems with continuous media
Computers usually handle data as discrete, so continuous media, like video and audio must be viewed as periodical and discrete by the computer.
There are few file system services for continuous media applications.
Kernel, and hardware availability varies over time. It is difficult to shedule processes. Many interupts are to be handled by the system.
Reservation of resources, like CPU time, is not possible on a standard desktop PC.
Delivery delays in real time applications make scheduling periodical processing of the media difficult.
1/14/2019

16. Summary
Raw digitized data is much too large. Compression can be used to reduce the size of data.
Entropy coding: loss less compression techniques take statistical occurrence of symbols within a stream
Source coding: often lossy compression, that take into account specific data characteristics and the human perceptive sensitivities to data
Hybrid coding: combination of the two types.
1/14/2019

17. Summary continued
The hardware architecture and the system software in desktop computers are not adapted for handling continuous media.
Hardware support for handling continuous media would increase quality, such as replacement of the single asynchronous bus.
Standard operating system software has not changed to better schedule multimedia or reserve resources. But application level languages such as SMIL can help.
SMIL (Synchronized Multimedia Integration Language) is a XML-based language to mix media presentations over low speed connections (>=28.8 kbps). See
1/14/2019