1. Comp 294-9 :: Fall 2003 Video As A Datatype Ketan Mayer-Patel

2. Comp 294-9 :: Fall 2003 The mothers of all video... NTSC - American standard PAL - European standard SECAM - French standard (ugly stepchild) For better or worse, this is where it all starts.

3. Comp 294-9 :: Fall 2003 Color Spaces One luminance component Two chrominance components –Original TV was black and white. –Adding color had to be done in a compatible way. NTSC: YIQ PAL: YUV In general: YUV and YCrCb used as terms.

4. Comp 294-9 :: Fall 2003 YUV vs. RGB Simple linear transform from one cube to the other. Specific transform coefficients vary for specific systems, but a common one: –Y = 0.299 R + 0.587 G + 0.114 B –Cr = -0.147 R - 0.289 G + 0.437 B + 0.5 –Cb = 0.615 R - 0.515 G - 0.100 B + 0.5

5. Comp 294-9 :: Fall 2003 YUV Challenges YUV is like taking RGB cube, standing it on a corner with (0,0,0) on bottom and (1,1,1) at top, rotating slightly, and taking the bounding box. What problems might this incur? –Some RGB colors are illegal. –A lot of YUV colors are illegal.

6. Comp 294-9 :: Fall 2003 A Note On Color Previous view of color is EXTREMELY simplified. –Color is complicated Frequency dependent response for contrast, lightness, etc. –Gamma correction.`

7. Comp 294-9 :: Fall 2003 Scanning Analog video signal is continuous voltage signal that gets scanned along the screen. The electron stream controlled by two orthogonal sets of magnets. –Horizontal: Beam is moved from left to right and then quickly back. –Vertically: Beam is moved from top to bottom and then quickly back.

8. Comp 294-9 :: Fall 2003 Scanning illustrated Time Voltage

9. Comp 294-9 :: Fall 2003 Structure of Video Blinn’s article. –Macrostructure Frequency peaks at frame rate. –Microstructure Frequency peaks at line rate Adding color is a good first example of source- aware channel coding. –What was the problem? –What was the solution? –Why does it work? –What are its drawbacks (i.e., when does it fail)?

10. Comp 294-9 :: Fall 2003 Interlacing Progressive = every scan line done in turn. Interlaced = every other scan line done. –Creates two fields: odd field and even field. –NTSC: 262.5 lines per field at 60 fields per sec. –PAL: 312.5 lines per field at 50 fields per sec. Fields are separated in time.

11. Comp 294-9 :: Fall 2003 Analog Bandwidths True meaning of the bandwidth. Y, U, and V signals are all continuous along a scan line. –A bit of a hybrid signal: discrete vertically, continuous horizontally. NTSC: –Y is 4.2 MHz wide, I is 2 MHz, Q is 1 MHz PAL: –Y is 6 MHz wide, U is 3 MHz, V is 3 MHz

12. Comp 294-9 :: Fall 2003 Corresponding Data Rates How much data can you put in 6 MHz band? –Depends on S/N ratio. –Depends on modulation scheme. Typical: 27 - 36 Mbs How many cable channels do you get? –If 50, then 1.3 - 1.8 Gbs coming into your home. The real question for multimedia is: –Why haven’t we found a better use for 1.3 Gbs than continuous broadcast of Real World.

13. Comp 294-9 :: Fall 2003 Digital Video Frames Almost always progressive 3 planes of pixel values (Y, U, and V) Pixel depth Geometry of each plane: width x height –Chrominance is generally subsampled. How the planes relate to each other. Frame rate.

14. Comp 294-9 :: Fall 2003 CCIR-601 Standard established for digitizing NTSC and PAL signals.

15. Comp 294-9 :: Fall 2003 Why 8-bits for chrominance? What’s another way to cut chrominance bandwidth in half? –Use 4-bits per pixel. Why won’t that work? –Need the dynamic range for color. But what about when Y is either small or large? –Don’t need the range, but lots of YUV combinations that won’t ever be used.

16. Comp 294-9 :: Fall 2003 4:2:2 For every 4 luminance samples, take 2 chrominance samples from odd lines and 2 from even lines. Chrom. planes just as tall, half as wide. JPEG does this.

17. Comp 294-9 :: Fall 2003 4:2:0 2 chrominance samples for every 4 luminance samples, odd lines only. Chrominance halved in both directions. MPEG generally does this.

18. Comp 294-9 :: Fall 2003 4:1:1 What should this be? –1 chrominance for every 4 luminance for both odd and even lines. –And that is what it is. But sometimes used to refer to this:

19. Comp 294-9 :: Fall 2003 De-interlacing Since analog video fields are separated in time by 1/2 the frame rate, at least half of a digital frame is missing no matter where you sample from. Normal solution: linear interpolation of even fields to produce matching samples for odd fields. Even better: linear interpolation of both fields to produce matching samples and thus digital frame rate will equal field rate.

20. Comp 294-9 :: Fall 2003 Film Frame Rates Film is a different beast altogether. –Continuous both vertically and horizontally –Projection is simultaneous for all parts of the picture. –Expensive medium. –Combination of all of this motivates 24 fps. Film to digital is easier than video to digital. –No interlacing, sample where you want to. Film to video is harder.

21. Comp 294-9 :: Fall 2003 3:2 Pulldown Converts 24 frames to 60 fields. 12345 1e1o2e2o3e2o3e3o4e4o5e5o

22. Comp 294-9 :: Fall 2003 Overall sampling lesson Can’t recover what you don’t have. Conversion between representations requires estimation of missing samples. Interpolation causes errors: –Spatially: at the edges. –Temporally: when moving.

23. Comp 294-9 :: Fall 2003 Common Digital Video Sizes CCIR-601720x4804:2:2, 4:2:0 SIF360x2404:2:0 CIF360x2884:2:0 4:3 HDTV1440x11524:2:2, 4:2:0 9:16 HDTV1920x11524:2:2, 4:2:0 4CIF, 16CIF, QCIF

24. Comp 294-9 :: Fall 2003 Digital Bitrates Current television: –30 fps * 720 * 480 * 1.5 * 8 = 124 Mb/s 9:16 HDTV –30 fps * 1920 * 1152 * 1.5 * 8 = 796Mb/s This motivates compression.