1. Digital Media Dr. Jim Rowan ITEC 2110 Video Part 2

2. Roll call Sanchez-Casas, Jon F. Simson, Davis Sinnock, Grant A. Swaim, Mark S. Tran, Dung Q. Vyas, Anand A. Woldeyohannes, Tesfamichael Barton, Paul H. Bois, Lauren C. Bonds, Allison E. Duncan, Jarred T. Lawson, Joseph I. Mulongo, Julio B. Pennison, Heather L. Reilly, Daniel J.

3. Roll call Jones, Crystal L. Marsh, Kerreen A. Thompson, Daniel G. Tran, Christopher V.

4. Digital Video Standards Even though digital video COULD be much less complicated... it isn’t because... Backward compatibility requirement –new equipment must create signals that can be handled by older equipment Originally... TV signals (analog) needed to be converted to digital format

5. Digital Video Standards... Looking at NTSC and PAL are ANALOG standards –Each has a screen size and frame (or refresh) rate –Each define a number of lines on the screen (that can be easily used for one the Y dimension) –But what about the other, the X?... –Each line is a continuous (analog) signal which has to be converted to digital... How do you do that? –SAMPLE the analog data! –But directly sampling for each pixel results in a data stream of 20 Mbytes/ second... HUGE!

6. Sampling for the Y component To reduce the data stream you can consider human vision again Human eyes are less sensitive to color changes than luminance Fewer samples taken for color than luminance –Sub-sample for color –4:2:2 (Y’CbCr) –4:1:1 used for many DV formats

7. Color representation in video based on the de-facto TV standard for CRTs To show a color pixel you need Intensity & R, G & B To reduce the data stream (for video size reduction –designers realized that Green contributes the most to intensity, Red is next and Blue hardly contributes anything to intensity –decided to use a formula for luminance Y = 0.2125R+0.7154G+0.0721B

8. Color representation in video based on the de-facto TV standard for CRTs Y = 0.2125R+0.7154G+0.0721B There is a use for algebra! –With this formula you can derive any color (RGB) if given Y and two of the colors then calculate the Green –Why transmit Y and Red and Green and Blue if you can achieve the same thing with Y’ & R & G? This is the Y’CbCr model for CRTs

9. NTSC & PAL weirdness NTSC & PAL –Define different screen sizes –Define different frame rates –Both have the same aspect ratio of 4:3 –BUT... they each are digitized (through sampling) to the same screen size The result? –The pixels are not square –PAL is taller than it is wide –NTSC is wider than it is tall

10. DV and MPEG DV and its different forms (DVCAM & DVPRO) –use the same compression algorithm (5:1) –use the same data stream (25Mbits) –use 4:2:2 sampling where DV uses 4:1:1 MPEG-1 originally meant for Video CD –never got very popular –developed into a family of standards MPEG-4 rose from the ashes –used for iTunes video

11. and now for something completely different! Video Compression! everything we have done so far results in a smaller video stream... but it is still too big so what do you do?

12. Video Compression Two ways to do this –1) Individual images can be compressed using the techniques discussed in the bitmapped section Spatial compression –2) Sequences or subsequences of frames can be compressed... Temporal compression

13. Computational Irony Digital has been touted as a way to create exact copies while analog (VCR) cannot... Analog VCR suffers from generational loss BUT only if you use video compression that is... LOSSLESS AND... you guessed it, a lossless video compression technique is not used because the lossless ones don’t compress enough

14. Lossless compression compressed original compression routine Original decompress routine Original Exact duplicate

15. Lossy compression compressed original compression routine Original decompress routine Changed Original Changed Original 2

16. Video Compression Coder/Decoder - Codec encodes and decodes video –can be symmetric –can be asymmetric PAL DV does 4:2:0 (p210) NTSC DV does 4:1:1

17. Temporal Compression 1 Use the Difference in two frames –naive approach can result in good compression –works well for a small amount of movement –A Tarantino film? not so much...

18. Temporal Compression 2 Use Motion Compensation When an object moves –compute its trajectory –fill in the resulting exposed background Works well for a small amount of movement A Tarantino film? not so much... BUT... why isn’t this an easy thing to do?

19. Temporal Compression 2 Use Motion Compensation When an object moves –compute its trajectory –fill in the resulting exposed background Works well for a small amount of movement A Tarantino film? not so much... BUT... why isn’t this an easy thing to do? Bitmapped images do not have defined objects! What to do?

20. Temporal Compression 2 Define blocks of 16 x 16 pixels –called a macroblock Compute all possible movements of the block within a short range Compute a vector to define that movement Store the movement vectors Compress the vectors

21. Temporal Compression Need some place to start from Can be forward or backward prediction Called KeyFrames –pick a keyframe –compute next image from that –What happens when the scene completely changes? Pick a new key frame... But HOW? Requires powerful AI

22. Why worry about size? When we know that computers are always getting faster and larger... why not just wait? First one on the market usually wins the market (if it succeeds!) Putting video on some small device with limited space and limited speed –iPhone –Spacecraft? Gotta be light weight and small!

23. A final worry... We have been talking about making video smaller There are a variety of techniques to do this Which to choose? –It is a tradeoff between compression technique and its computational complexity

24. Questions?