1. Multimedia- and Web-based Information Systems Lecture 5

2. Multimedia: Color- and Video- technology

3. Video-Technology Television- and Video-Technology form the basis of the medium motion picture Generation – Recording from the real world – Synthesis on the basis of a description Analogous and digital technology

4. Representation of the video signal Representation of the video signal contains – Visual representation – Transmission – Digitalization

5. Visual Representation Presentation of the video signal trough a CRT (Cathode Ray Tube) – In television and computer screens Representation of a scene as realistic as possible – Delivery of the space and time content of a scene

6. Fundamentals of visual representation Resolution – Width W – Height H – E.g. W=833, H=625 Width/height-relation – 4:3 or 16:9 Perception of depth – In the natural preception trough the use of both eyes (different view angles onto one scene) – Focus-depth of the camera, appearance of the material of an object

7. Fundamentals of visual representation Luminance / Chrominance Motion picture resolution / continuity – Discreet sequence of single pictures can be perceived as a continually sequence – Boundary of motion picture resolution – 15 pictures/sec (video used 30 pictures/sec) – No boundary with acoustic signals

8. Fundamentals of visual representation Flicker – With small refresh rate – Eg. 50 or 60 Hz – Full and half pictures (interlacing)

9. RGB Color Coding RGB (Red Green Blue) Additive color blend Normalization of values (R+G+B=1)

10. YUV Color Coding For the human eye, brightness is more important than color information Brightnessinformation (Luminance) – 1 channel of luminance (Y) Color Information (Chrominance) – 2 channels of chrominance (U and V)

11. Component Coding YUV Y = 0.30 R + 0.59 G + 0.11 B U = 0.493 (B-Y) V = 0.877 (R-Y) Errors in Y are more severe – Y to be encoded with high bandwidth YUV Coding often specified with a raito of the channels (4:2:2)

12. Component Coding YUV YIQ (similar to YUV) Derived from NTSC Y = 0.30 R + 0.59 G + 0.11 B I = 0.60 R + 0.28 G + 0.32 B Q = 0.21 R + 0.52 G + 0.31 B

13. Shared Signal Individual components (RGB, YUV, YIQ) need to be combined to one signal Methods of modulation to avoid interference

14. Video formats Resolution of a picture (frame) Quantisation Framerate Video controller – Dedicated video memory

15. Video formats CGA (Color Graphics Adapter) – 320x200, 4 colors, 16.000 bytes EGA (Enhanced Graphic Adapter) – 640x350, 16 colors, 112.000 bytes VGA (Video Graphic Array) – 640x480, 256 colors, 307.200 bytes XVGA (eXtended Video Graphic Array) – 1024x768, 256 colors, 768.423 bytes XGA (eXtended Graphic Array) – 1024x768, 16M colors, 2304 kbytes Many more

16. Conventional Systems NTSC (National Television Systems Commitee) – From the USA, oldest standard, widely used, 30 Hz, 525 lines SECAM (Sequential Coleur avec Memoire) – France, Eastern Europe, 25 Hz, 625 lines PAL (Phase Alternating Line) – Western Europe, 25 Hz, 625 lines

17. High-Definition Television (HDTV) Resolution – 1440x1152 / 1920x1152 Frame rate – 50 or 60 Hz No longer interlaced

18. Digitalisation of video signals Conversion into a digital representation Nyquist-Theorem (bandwidth = half the sampling rate) – Of the components Quantisation 2 Alternatives – Shared Coding – Component Coding

19. Shared Coding Scanning of the whole of the analogue video signal (e.g. composite video) Dependant on the standard Bandwidth the same for all components Disadvantage: low in contrast

20. Component Coding Separate digitalisation of the components (e.g. YUV) Ratio 4:2:2 – 864 scan values for luminance – 432 scan values for chrominancy

21. Digital Television Digital Television Broadcasting (DTVB) – Digital Video Broadcasting (DVB) – DVB-T (terrestric broadcast) – System description Implementation of HDTV Employs MPEG-2 – Coding of Audio and Video

22. Advantages of DVB Increase in the number of TV-channels Adaptable picture and sound quality Encryption possible for Pay-TV New Services: Data broadcast, Multimedia broadcast, Video-on-Demand Convergence of PC and TV

23. Multimedia: Data Compression

24. Data Compression Audio and Video require lots of storage space – Increasing Demand Text – Single Pictures – Audio – Motion Picture Data rates influence – Transmission – Processing Efficient Compression – Theory – Standards

25. Storage Space / Bandwidth Considerable storage capacity for uncompressed pictures, audio and video data – For uncompressed Video, even a DVD is not sufficient Uncompressed Audio-/Videodata requires very high bandwidth

26. Required Storage Space Text – 80 x 60 * 2 bytes = 9600 bytes = 9,4 KByte Figures – 500 primitives * 5 Bytes for properties = 2500 bytes Voice – 8 kHz, 8 bit quantisation = 8 kByte / s Audio – 2 x 44100*16 bit / 8 bit * 1 byte = 172 Kbyte / s Video – 640 x 480 * 3 x 25 frames = 22,500 Kbyte /s

27. Important Methods JPEG (JPEG 2000) – For single pictures H.261 and H.263 – Video sequences of small resolution MPEG 1,2 and 4 – Motion Picture and Audio (MPEG Layer 3)

28. Demands on Methods Good quality Small complexity – Effective implementation Time boundaries with decompression (and compression) – MPEG-1: high effort with compression

29. Demands in Dialogue mode End-to-End latency – Part of the (De-)Compression < 150 ms – 50 ms -> natural dialogue – Additionally all latencies of the network, communication protocols and of the in- and output devices

30. Demands in Query mode Fast Forward / Rewind with simoultaneuos display of the data Random access to single frames – < 0.5 s – Decompression of single pictures without interpretation of all the frames before them

31. Demands in Dialogue and Query mode Format independent of screen size and refresh rate Audio and video in different qualities (to adapt to the respective circumstances) Synchronisation of Audio and Video Implementation in software

32. Classification of compression methods Entropy coding – Lossless methods Source coding – Often lossy Hybrid coding – Combined application of both of the methods above for a specific scenario

33. Entropy coding Independent of media specific properties Data to compress is a sequence of digital data values Losslessness – Data before and after the compression/decompression are identical

34. Source coding Usage of the semantics of the information Compression ratio depends on the specific medium Data before and after the compressen/decompression are very similar to each other but no longer identical

35. Hybrid coding Combination of entroy and souce coding, used e.g. In – JPEG – MPEG – H.263

36. Decompression Inverse function of the compression Decompression possible in real time? Symmetric methods – Similar effort for coding and decoding Assymetric method – Decoding possible with smaller effort

37. Run length encoding Sequence of identical bytes Number of repeating bytes Mark M (e.g. „!“) Stuffing if M is in the data space Example 1: 0, „!“, 256 Example 2: „!“, „!“ (Stuffing) In what cases does it help? Maximum saving?

38. Suppression of null values Special case of run length encoding Selection of a single character that is repeated often (e.g. „0“) Mark M, after that number of repetitions In what cases does it help? Maximum saving?

39. Vector quantisation Splitting of the data stream into blocks of n bytes Table with patterns for blocks Index into the table to the entry most similar to the block Multi-dimensional table -> vector Approximation of the original data stream Example

40. Pattern Substitution Patterns of frequent occurence replaced by one byte Mark M, then index into a table Well suited for text e.g. keywords in programming languages

41. Diatomic Encoding Putting together of two bytes of data at a time Determination of the byte-pairs occuring most frequently e.g. in the English language – „E“, „T“, „TH“, „RE“, „IN“,... (8 in total) Special bytes not occuring in the text used to represent 2 letters Reduction in data of ca. 10%

42. Static encoding Frequency of occurence of a character Different coding length for characters Basis of the Morse code Important: unambigous decompression

43. Huffmann coding Regards the probability of occurence Minimum number of bits for given probability of occurence Characters occuring most often get the shortest code words Binary tree (Nodes contain probabilities, edges bit 0 or 1)

44. Huffmann coding P(A)=0.16, P(B)=0.51, P(C)=0.09, P(D)=0.13 and P(E)=0.11

45. Huffmann Coding w(A)=001, w(B)=1, w(C)=011, w(D)=000, w(E)=010 P(ADCEB)=1.0 P(B)=0.51P(ADCE) P(CE)=0.20 P(AD)=0.29 P(C)=0.09P(E)=0.11P(D)=0.13P(A)=0.16 0 1 1 0 1 0 0 1

46. Transformation coding Data transformed into a better suited mathematical space Inverse Transformation needs to be possible Discrete Cosine-Transformation (DCT) Fast-Fourier-Transformation (FFT) See example in the JPEG lecture

47. Prediction or relative encoding Forming the difference to the previous value Data do not differ much Combination of methods – e.g. homogenous areas in pictures DPCM, DM and ADPCM

48. Further Methods Color tables – with pictures (video) Muting – Threshold for sound volume