Last update on June 15, 2010 Doug Young Suh suh@khu.ac.kr Transformation Last update on June 15, 2010 Doug Young Suh suh@khu.ac.kr 7/30/2018

Entropy and compression amount of information = degree of surprise Entropy and average code length Information source and coding Memoryless source : no correlation ∙∙∙∙∙ Red blue yellow yellow red black red ∙∙∙ 00011010001100 ∙∙∙ 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University Entropy Entropy What if {0.99,0.003,0.003,0.004}? H(X)≈0 Reduce H(X)!! We need narrower pdf. 1/4 1 2 3 4 1/2 1/4 1/8 1 2 3 4 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University How to get small H(X)? Transformation Signals in the frequency domain or mapping into more probable vectors Predict to reduce uncertainty H(X) = “degree of uncertainty” Prediction by using known information 7/30/2018 Media Lab. Kyung Hee University

Set of ortho-normal vectors Inner product of two orthogonal vectors is 0. Inner product Add multiplications at the same positions. Normal Normal 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University Mapping to { , } Any vector in the 2 dimensional space can be represented by weighted sum of 2 ortho-normal vectors.                                                                        In this case, c0 is stronger. What does it mean “a weight is large”? weights 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University How to get the weights? Inner product for weight More generally, 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University 2 point DCT/IDCT 2 point DCT/IDCT DCT IDCT 7/30/2018 Media Lab. Kyung Hee University

4-pt transform 4 basis vectors (i.e. code) possible C1 =[½ ½ ½ ½ ] Matrix representation Networked Video

Media Lab. Kyung Hee University 8-pt DCT with 8D vectors Any set of 8 data can be represented by weighted sum of 8 ortho-normal vectors 8 weights for 8 ortho-normal vectors in the 8 dimensional spaces. Frequency 0  DC, constant (not varying) 7/30/2018 Media Lab. Kyung Hee University

8X8D ortho-normal vectors is calculated for u=1 and x=0~7 (cosine 1/2 period) is calculated for u=2 and x=0~7 (cosine 1 period) is calculated for u=3 and x=0~7 (cosine 1.5 period) is calculated for u=7 and x=0~7 (cosine 3.5 period) 7/30/2018 Media Lab. Kyung Hee University

8 8D ortho-normal vectors DC frequency 0 u=0 From low frequency u=1 u=2 u=3 To high frequency n=0  n=3  n=7 7/30/2018 Media Lab. Kyung Hee University

8 8D ortho-normal vectors From low frequency u=4 u=5 u=6 u=7 To high frequency n=0  n=3  n=7 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University 8-pt DCT/IDCT IDCT DCT 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University 8-pt DCT of DC DC : constant signal 255 255 255 255 255 255 255 255 722 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University 8-pt DCT of step signal Step signal 255 255 255 255 360 326 -114 77 65 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University 8-pt DCT of step signal High frequency signal 255 255 255 255 360 65 77 114 327 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University 8x8 DCT 2D DCT of the 8x8 block, 64 pixels 8 point DCT for 8 rows of 8 pixels  8 point DCT for 8 columns 8x8 DCT 8x8 IDCT 7/30/2018 Media Lab. Kyung Hee University

8x8 DCT 8 pt DCT is to calculate weights for 8 1D basic patterns, while 8x8 2D DCT is to calculate weights for 64 2D basic patterns. For example,

8x8 DCT 64 8x8 patterns U=0 U=7 v=0 v=7

Media Lab. Kyung Hee University 2D 8x8 DCT By using 8X8 DCT, 64 weights are calculated and stored, respectively. These are 2D 8x8 DCT coefficients. 7/30/2018 Media Lab. Kyung Hee University

2D 8x8 DCT at horizontal edge F00 and F01 are large. For u>0, they are almost 0. DCT Block with Horizontal edge DCT of Horizontal edge 7/30/2018 Media Lab. Kyung Hee University

Block with vertical edge Media Lab. Kyung Hee University 2D 8x8 DCT at vertical edge F00 and F10 are large. For v>0, the values are small. F10 < 0 ? DCT Block with vertical edge DCT of vertical edge 7/30/2018 Media Lab. Kyung Hee University

Low pass filtering (LPF) Remove weights of higher frequency LPF 7/30/2018 Media Lab. Kyung Hee University

Media Lab. Kyung Hee University Energy compaction More compression when energy distribution is focused to a direction. The simpler an image, the more compression. DCT is better than DFT for image.  KL transform! Ortho-normal vector patterns of DCT are better suited to image. 1 2 3 4 5 6 7 Simple image Complex image DCT DFT 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 7/30/2018 Media Lab. Kyung Hee University

Matrix representation Matrix representation of 4x4-pt transform where , then Complexity ~ O(N3)  needs fast algorithm Networked Video

Media Lab. Kyung Hee University Summary Transform for compression Energy compaction transform = inner product to ortho-normal vectors weighted sum Weights = frequency coefficients No information loss at all !! 7/30/2018 Media Lab. Kyung Hee University

Video encoding 2-1 2-3 2-2 DCT Q VLC IQ IDCT Original Video Encoded + DCT Q VLC Encoded Bitstream IQ Motion Estimation Motion vector Frame Memory IDCT Networked Video