1. Digital Video Heejune AHN Embedded Communications Laboratory Seoul National Univ. of Technology Fall 2008 Last updated 2008. 8. 31

2. Heejune AHN: Image and Video Compressionp. 2 1. Video & Image Video Projection of 3-D scene onto a 2-D Plane Illumination & Color information but No depth info Cf) 3-D Video, 2 D Stereo Vision Analog/continuous function: f (x, y, t) Image Still Image f(x, y, t = t0)

3. Heejune AHN: Image and Video Compressionp. 3 Digital Video Processing Digital Video System Capture Sampling in Time Domain and Spatial Domain Photo sensor (e.g. CCD arrays) Analog vs Digital Camcoder/Camera Question: Why Digital ? Capture digitization Processing, Storage, Transmission sensor DSP/ASIC display

4. Heejune AHN: Image and Video Compressionp. 4 2. Video Sampling Background Theory Nyquist Theorem We loose information over sampling rate (f s /2) Human Visual System Spatial Domain Sampling Resolution: N 1 X N 2 pel: Picture Element

5. Heejune AHN: Image and Video Compressionp. 5 Time Domain Sampling Frame rate Human Visual System (~30 Hz) Flickering Interlaced against Progressive Reduce Time Resolution without loss of spatial resolution Bad effect on Today’s Computer Displays t + ts/2 (odd) t + ts (even)

6. Heejune AHN: Image and Video Compressionp. 6 Typical Examples for Resolution CIF 352x288 = 101376 pixels VHS qualitity SD 704x576 = 405504 pixels Broadcast quality HD 1440x1152 = 1313280 pixels HD quality Typical Examples for Appearance Below 10 fps: Jerky 10 ~ 20 fps: Slow movement (video phone) is OK 20 ~ 30 fps: Movement is quite smooth 50 ~ 60 fps: Very Smooth/Fantastic

7. Heejune AHN: Image and Video Compressionp. 7 3. Color spaces RGB (red, green and blue) R, G, B has equal sensitivity for HVS e.g. 8 bits for R/G/B, 6/6/5 bits for R/G/B 24bits/pel B G R

8. Heejune AHN: Image and Video Compressionp. 8 YCrCb YCrCb (YUV) HVS is more acute to Luminance (Y) Reduce Data by Reducing Coloring Resolution (e.g. YCrCb420) Color Transform

9. Heejune AHN: Image and Video Compressionp. 9 YCbCr Transform Luminance as a weighted sum of R, G, B Y = 0.299R + 0.587G + 0.114B Cr, Cb, and Cg are different from Y. Cb = 0.564 (B-Y), Cr = 0.713 (R-Y), Cg = ?? Note that Cr + Cb + Cg = constant

10. Heejune AHN: Image and Video Compressionp. 10 Digital YCbCr For easy signal processing Non-negative Color and 128 center value 16<Y<235, 16<U, V< 240, Center = 128 Y U(Cb) V (Cr)

11. Heejune AHN: Image and Video Compressionp. 11 Color Sampling (4:x:y) How many/where to sample Color components. x = the number of color samples in odd number line y = the number of color samples in even number line

12. Heejune AHN: Image and Video Compressionp. 12 Raw Image File Format Packed: pixel base packed easy for pixel unit processing (eg. display and capture) Planar: merged for each components easy for compression processing rgb rgb rgb rgb y y y y rgb rgb rgb rgb y y y y u u v v

13. Heejune AHN: Image and Video Compressionp. 13 4. HVS Human Visual System Note that ultimate performance is evaluated by Human Signal Flow Anatomy Eye Lens => Retina => Optical nerve => Brain For HVS details, See Jain’s Fundamental of Image Processing

14. Heejune AHN: Image and Video Compressionp. 14 동공의 광학특성 광파 주파수 인식특성 휘도 인식특성 공간주파수 인식특성 빛에너지휘도 빛세기 luminance light intensity light energy Perceived Luminance (contrast) brightness 인식휘도 ( 물체의 ) 밝기 LPF 특성 ( 망막착상 ) 가시광 대역 violet red 파장 Logarithmic 인식특성 ( 휘도의 크기에 대한 ) LPF/BPF 특성 ( 휘도의 공간측변화에 대한 )

15. Heejune AHN: Image and Video Compressionp. 15 Important Features of HVS More sensitive to Luminance than Color More sensitive to High contrast than low Contrast (e.g. Edges) More sensitive to Low frequency than High More sensitive to Long duration than short Sampling time is around 20 ~ 30 Hz HVS response varies from individuals to individuals

16. Heejune AHN: Image and Video Compressionp. 16 5. Video Quality No Measure No Engineering Performance Measure quality: how good similar to original signal? Applied for lossy compression only compression ratio : how much data volume is reduced? Applied for both lossy and lossless compression Engineering Trade-off between quality and compression ratio Given bandwidth (compression ratio), and optimize quality Given required quality, and minimize the bandwidth

17. Heejune AHN: Image and Video Compressionp. 17 Subjective Quality Measure Principle By human vision (like Olympic gymnastics) Individual Difference E.g. Professional and non-Professional has different MOS (mean Opinion Score) DSCQS (Dual Stimulus Continuous Quality Scale) Two Image/Video is given at the same time, score them. Original and Processed are compared SSCQS (Single Stimulus Continuous Quality Scale) orignal processing display

18. Heejune AHN: Image and Video Compressionp. 18 Objective Quality Measure By Numerical Value SNR (Signal to Noise Ratio) and MSE (Mean Squared Error) PSNR (Peak Signal to Noise Ratio) E.g. HQ: 40 db = 20log10 (255/ESTD) => 255/ESTD = 100 => ESTD = 2.5 LQ : 20 db => 255/ESTD = 10 => ESTD = 25

19. Heejune AHN: Image and Video Compressionp. 19 What PSNR for Left Image?

20. Heejune AHN: Image and Video Compressionp. 20 6. Standard Image Format Standard Resolution size

21. Heejune AHN: Image and Video Compressionp. 21 7. Applications Broadcasting Digital TV : Sattlite, Terrestrial, Cable Mobile TV: DMB, DVB-H etc Telecommunication Video Phone Video Conference MMS (multimedia Messaging Service) Game and Entertainment Video Storage : CD, DVD, Blue-Ray Natural Image Games ETC E-Learning UCC Remote Medicine Surveillance Monitoring

22. Heejune AHN: Image and Video Compressionp. 22 System Type Dedicated System STB, DVD player, PMP Player etc PC Mobile Embedded PDA, Handsets Platform type Dedicated Hardware (ASIC) DSP Powerful General Purpose Processor

23. Heejune AHN: Image and Video Compressionp. 23 PVR Hard disk drive MPEG-2 Qualities Best7.7 Mbps High5.4 Mbps Medium3.6 Mbps Basic2.2 Mbps