1. Digital Media Lecture 10: Video & Compression Georgia Gwinnett College School of Science and Technology Dr. Jim Rowan

2. Coping with Video Size Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 2) Spatial 3) Temporal differencing vectoring

3. Coping with Video Size Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 3) spatial 4) Temporal differencing vectoring

4. Chrominance sub-sampling Humans can’t distinguish changes in color as well as they can distinguish luminance changes –http://en.wikipedia.org/wiki/Chroma_subsa mplinghttp://en.wikipedia.org/wiki/Chroma_subsa mpling In our cameras… –Of every 4 frames –store the luminance for each frame –only store a proportion of the color info –4:2:0

5. Chrominance sub-sampling http://dougkerr.net/pumpkin/articles/Subsampling.pdf

6. http://en.wikipedia.org/wiki/Chroma_subsampling#Sampling_systems_and_ratios Chrominance sub-sampling Luminance, Cr, Cb

7. Coping with Video Size Aside from screen size and frame rate... Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 2) spatial 3) Temporal differencing vectoring

8. Coping with Video Size Spatial compression Individual images can be compressed using the techniques discussed in the bitmapped section Doesn’t result in very much compression for video Doesn’t take into consideration the other frames that come before or after it Our video cameras do this, compressing each frame to jpeg

9. Coping with Video Size Aside from screen size and frame rate... Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 2) spatial 3) Temporal differencing vectoring

10. Temporal Compression differencing Use the Difference in two frames –A naive approach can result in good compression –Works well for a small amount of movement Security cameras spend most of their time “seeing” the same thing all night long –A Tarantino film? not so much… Most pixels change with nearly every frame

11. Image Differencing To subtract one image from the next Do it one pixel at a time –red minus red –green minus green –blue minus blue Store the difference To play it back –Play frame one 1 –Add frame 2 to frame 1 Next, an example in black & white

12. Temporal Compression differencing

13. Example 1, the difference of two identical images

14. The result ===>

15. Example 2, the difference of two similar images

16. The result ===>

17. Temporal Compression vectoring When an OBJECT moves –compute its trajectory –fill in the resulting exposed background –BUT there’s a problem... –why isn’t this an easy thing to do? Captured Frame 1 Captured Frame 2 Stored Object Movement Vector Stored Background

18. More on differencing… The differencing can happen in a forward manner and a backward manner It might be more economical (in data size) to create a frame from a frame that follows it...

19. MPEG-2 iFrame (interframe prediction) pFrame (forward predicted) bFrame (backward predicted) GOP (group of pictures) http://en.wikipedia.org/wiki/Inter_fram e http://en.wikipedia.org/wiki/I- frames#Intra_coded_frames_.28or_slic es_or_I-frames_or_Key_frames.29 MPEG terminology

20. iFrame: -a keyframe -spatially compressed (a fully specified image) pFrame -predicted frame -contains only the difference between the current frame and the previous iFrame (smaller in size than iFrame) bFrame -bi-predicted frame -contains difference between current and both the preceding and following iFrames -even smaller in size than iFrame MPEG terminology

21. All spatially (intra-frame) compressed Spatially compressed and predictive (difference) Spatially compressed forward and backward predictive (difference) The largest Smaller… but more computation Smallest… but more computation and it is transported out of order! Play sequence: 1 2 3 4 5 6 7 8 9 10 Transmit sequence: 1 4 2 3 7 5 6 10 8 9 Group(s) Of Pictures (GOPs)

22. So… How does this ===> happen?

23. http://en.wikipedia.org/wiki/Inter_frame

24. Video Compression What does it? Coder/Decoder - Codec –encodes and decodes video Can be symmetric it takes as long to compress as decompress Can be asymmetric it takes longer to compress or decompress than it does to decompress to compress

25. Video Compression: Which codec to choose? It’s a tradeoff… compression technique sorenson DV Cinepak Intel Indeo MPEG4 the compression result asymmetric or symmetric –satellite provider’s choice? larger or smaller final size computational complexity the artifacts generated

26. http://www.100fps.com/codec_quality_comparison.htm Which is the original? Notice the artifacts?

27. http://www.100fps.com/codec_quality_comparison.htm Center one is the original Left is “sharpened” Right is “blurred”

28. Retrograde motion http://wiki.ggc.edu/images/8/85/Jrowan Spring2012CroppedRetrogradeMotionClip. mov http://wiki.ggc.edu/images/3/35/Retrogr adeGMCarcadiaBroadbandHigh.mov

30. Two ways to make Moving Pictures: Video & Animation In this class: –Video shot with a camera captures images from the world then play them back –Animation create frames individually using inkscape and blender play them back

31. Reading in the supplemental text: Moving images: Video