JPEG implementation and improvement Regular scan, Huffman : 0.72602 Zigzag scan, Huffman: 0.72445 Zigzag scan, Huffman_greedy: 0.72219 Zigzag scan, adaptive Arithmetic coding(AAC): 0.71271 R06942072 林俊宏

Follow JPEG standard published in 1992 Input : Lena color image Size = 512 * 512 Lena512c.bmp Environment : matlab 1972年Lena圖

Coarse block diagram of Compression Source data (Text, Image, Video….) Preprocessing Entropy encoder Compressed data (Bit Stream 0101….) PSNR（Peak Signal to Noise Ratio） SSIM （structural similarity） Reconstruction data (Text, Image, Video….) Bit per pixel (b.p.p) Bit per character (b.p.c.) Compression ratio (CR) Postprocessing Entropy decoder

Evaluation, B.p.p. & PSNR Reduce↓ Keep the same Lena 512*512 Color image Reduce↓ Keep the same

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Conclusion Future work

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Conclusion Future work

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) 64*64 cell 1*4096 cell 512*512 Stream_Y Chrominance channel (Cb & Cr) 64*64 1*4096 Huffman decoder fun1Dto2D Inverse Quantization Inverse DCT Up sampling Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC frequency domain DCT Ac Run length coding 利用DCT後的特性做紀錄的小技巧 Dc 差分編碼 space domain space domain 4:2:0 Stream_Cb 256*256 Stream_Cr

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

左上數值小的Q table

DCT & Quantization

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

AC, Run length coding 999 = End of block(EOB) Input : 8*8 block →[ZRL]→ output : 1*1cell

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

base code -- table

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

Inside Huffman.m : 4 Table DC (Category 0~11) AC (Run/Size = 0~F/0~A) Luminance (Y channel) 12 elements 16*10+2 (EOB & ZRL) = 162 elements Chrominance (Cb & Cr channel) 依照資料數值大小可以用多少bits表示來分類

How to categorize？ Range DC difference Category (from 0 to 11) Size of AC (from 1 to A) EOB || ZRL -1,1 1 -3,-2,2,3 2 -7~-4, 4~7 3 -15~-8, 8~15 4 ….. -1023~-512, 512~1023 10 A

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

Stream structure Include 5 parts : Category DC base code Content DC Run/Size AC base code Content AC ………………repeat End of Block(EOB) Take the first block of luminance channel for example : DC & AC: 16, [0 1 0 1 999] After encoding : 11010000 001 001 1010 DC : Category 5 , need 5 bits to represent content DC

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Conclusion Future work

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

Proposed method 1.「之」字形紀錄 The scanning style from Regular-scan to Zigzag-scan. 1 2 3 …… 63 64 65 66 67 127 128 4033 4034 4035 4095 4096 1 2 3 …… 63 64 128 127 126 66 65 4096 4095 4094 4034 4033

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Future work Conclusion

Proposed method 2. apply greedy algorithm on luminance DC. Create own luminance DC table. Why I want to create own table ？ To realize the core concept of Huffman : less bits for symbol with high frequency.

Calculate each Category appears frequency. Does not make sense !!! Category 2 should have the shortest Code length in Lena image →but it did not.

greedy algorithm luminance DC

tree{2}{1}{2}{2}{2}{2}{2}{2}{2} = 12 ← stages nine How to implement? tree{1}{1} = 3 ← stages two tree{1}{2} = 4 ← stages two tree{2}{1}{2}{2}{2}{2}{2}{2}{2} = 12 ← stages nine Cell的結構，分1/0 = tree的左邊右邊 Category2, Codelength = 2

How about apply Greedy algorithm on chrominance DC?

Calculate each Category appears frequency. The table is already fit the distribution of datas!!! We cannot get benefit, from greedy algorithm.

Conclusion of Proposed method 2. only apply greedy method on luminance DC (Y channel). Future work : To apply on AC channel maybe can further improve.

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Conclusion Future work

(Symbol-based) v.s. (Stream-based) An optional alternative to Huffman coding in several video coding standards (ex. H.263, MPEG-4, H.26L)

Example of Arithmetic coding To encode “aaabaa” [Lower,upper] = 0.4096, 0.475136 Use b bits to represent C b = 5, C = 14 ‘aaabaa’→ ‘01110’ 重點提及： 事前機率0.8 , 0.2 Lower和upper調整

adaptive arithmetic coding (A.A.C.) Add two mechanisms to fit the application on JPEG : 1. adapt the frequency table dynamically during the coding process. 2. adapt for long data length 直接套進來是行不通 : 1.不知道每個值發生的機率 2. 無法跑很長long Length [講稿1] the frequency table can truely fit the data distribution property. Beneifit from a larger numerical interval compared to fix frequency table. Consequently, we use less bit and make a good compression. [講稿2] 剛剛的資料很短aaabaa，要適應道很長資料上，電腦的數值沒辦法紀錄這麼細 微 所以我設0.5為threshold，如果越界了就必須輸出0或1來標記，這樣在解碼端才 解的回來，penalty就是Stream length會上升

adaptive arithmetic coding (A.A.C.) Stream structure In A.A.C. After encoding : 10001011……01010 long data length + Range encode Take the first block of luminance channel for example : DC & AC: 16, [0 1 0 1 999] In huffman After encoding : 11010000 001 001 1010

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding Huffman encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

Lena Image Luminance channel (Y) Chrominance channel (Cb & Cr) 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding AAC encoder AAC encoder fun2Dto1D Table L, AC Table L, DC Luminance channel (Y) Stream_Y Chrominance channel (Cb & Cr) Down Sampling 88 DCT Quantization 8*8 Qtable AC Run length coding DC differential coding AAC encoder fun2Dto1D Table C, AC Table C, DC Stream_Cb Stream_Cr

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Conclusion Future work

Conclusion Implement the lossy compression standard JPEG. modified the model to reduce b.p.p: Regular scan, Huffman : 0.72602 Zigzag scan, Huffman: 0.72445 Zigzag scan, Huffman_greedy: 0.72219 Zigzag scan, adaptive Arithmetic coding(AAC): 0.71271

Outline JPEG overview proposed method 1 - scanning style proposed method 2 – Huffman greedy algorithm proposed method 3 – adaptive arithmetic coding Conclusion Future work

Future work shape adaptive : the block size is not always 8*8. Coarse area : large block ; Fine area : small block edge detection image segmentation

Reference http://djj.ee.ntu.edu.tw/ADSP.htm http://chu246.blogspot.tw/2013/01/huffman-coding-by- matlab.html http://twins.ee.nctu.edu.tw/courses/soclab_04/lab_hw_pdf /proj1_jpeg_introduction.pdf