Image Compression Standards (JPEG) Chapter 9

outline 9.1 The JPEG Standard Appendix 9.2 The JPEG2000 Standard 9.3 The JPEG-LS Standard 9.4 The JBIG Standard

9.1 The JPEG Standard JPEG  “Joint Photographic Experts Group". Formally accepted as an international standard in 1992. lossy image compression method. Transform coding method using the DCT (Discrete CosineTransform). For each 8x8 (Y,U,V) or (Y,I,Q) plane, Transform f(i,j)  F(u,v) look choppy (“blocky") when a high compression ratio is specified by the user.

Observations for JPEG Image Compression Visual acuity is much greater for gray than for color. Chroma sub-sampling (4:2:0) is used in JPEG. Observation 2: Useful contents change relatively slowly across the image. It is unusual for intensity values to vary widely several times in a small area, Observation 3: Loss of high spatial frequency components is much less noticed then lower frequency components. Spatial redundancy can be reduced by reducing the high spatial frequency contents.

Observation 2. Useful image contents change relatively slowly across the image. Observation 3. reducing the high spatial frequency contents Observation 1. sub-sampling (4:2:0)

Review(1) RGB  YUV 4.3.1/ 4.3.2 Definition U=B-Y V=R-Y

Review (2) DCT vs. FFT DCT can approximate linear signals well with few coefficients.

Review (3) 8x8 Frequency Basis 左圖各小圖之位置只是彰顯(u,v)之值 小圖應還原為大圖( 8x8)才能看出貼頻之意

M=N=8

9.1.1 Main Steps in JPEG Image Compression (1) YIQ or YUV + Subsampling (2) DCT on 8 x 8 blocks (3) Quantization (4) Zig-zag ordering + Run-length (5) Entropy coding

Quantization F (u, v) : quantized DCT coefficients Q(u,v) 與 “Quality factor” 呈反比設計 ^ F (u, v) : quantized DCT coefficients F (u, v) : DCT coefficient, Q(u, v) : “quantization matrix"

Quantization Matrices (Tables)

Shifted by -128 再進行DCT (Shift 只對DC有作用) Quick Proof: Avg≒192 (1/8)*(192-128)*64=512≒515

IDCT +128 ~

左圖各小圖之位置只是彰顯(u,v)之值 小圖應還原為大圖( 8x8)才能看出貼頻之意

Shifted by -128 再進行DCT (Shift 只對DC有作用) 高頻的 Quantization 級距大造成誤差，但可忍受

Encoder 內部開兩座工廠 (1) DC encoder (2) AC encoder Encoder Decoder 每個 8x8 block 的 F^ DC先進 DC encoder  output bit stream AC's 隨後進 AC encoder Decoder 讀取 bit stream 次序不亂 block by block 正確 分送到 DC/AC decoder 即可還原!! Block by block

DPCM on DC coefficients The DC coefficients are coded Separately from the AC ones. Block by block (DC0,DC1,.. Goes to DC encoder) Differential Pulse Code Modulation (DPCM) is the coding method. Example: DC coeff. for the first 5 blocks 150,155, 149, 152, 144 150, 5,-6, 3, -8 Assuming d i = DC i +1 – DC i and d 0=DC 0.

DPCM on DC Coefficients (cont.) Step 1:  DCi  (Size, Amplitude) Step 2: Encoding size: indicating size of VLI of amplitude, VLC (huffman coding): input to encoder, application specific amplitude VLI (Variable Length Integer): hardwired into proposal

In the example we're using, codes 150, 5, -6, 3, -8 will be turned into (8, 10010110), (3, 101), (3, 001), (2, 11), (4, 0111) . SIZE is Huffman coded since smaller SIZEs occur much more often. AMPLITUDE is not Human coded, its value can change widely so Human coding has no appreciable benefit. (VLI)

Size & Amplitude 負數取 1’s Complements 見 0 首引就識別為負數 然後 0  1 (bit by bit) ------------------------------------ SIZE Value ----0----------------0---------(f)-- 1 -1, 1 (0, 1) 2 -3, -2, 2, 3 (00, 01, 10, 11) 3 -7..-4, 4..7 (000, 001, 010, 011,100,…) 4 -15..-8, 8..15 . . 10 -1023..-512, 512..1023 Categorize DC values into SIZE (number of bits needed to represent) and actual bits. DC 之極值為 -(1/8)(128*64), (1/8)(127*64) DPCM 最大差值為 Amp = ± 2040 /step 假設 step=16 則 Amp= ± 128 假設 step=1 則 Amp= ± 2040 2040  (11, 11111111000)

Encoder 內部開兩座工廠 (1) DC encoder (2) AC encoder Encoder Decoder 每個 8x8 block 的 F^ DC先進 DC encoder  output bit stream AC's 隨後進 AC encoder Decoder 讀取 bit stream 次序不亂 block by block 正確 分送到 DC/AC decoder 即可還原!! Block by block

Run-length Coding (RLC) on AC coefficients Zig-Zag Scan turns 8x8 Matrix to 64-Vector RLC turns the F (u, v) values into sets {#-zeros-to-skip , next non-zero value} ^

RLC on AC Coefficients 1 x 64 vector has lots of zeros in it RLE: Run Length Encoding Processing Steps Step 1: AC  (RunLength, Size), (Amplitude) Step 2 (RunLength, Size): VLC (Huffman coding) Amplitude: VLI

Entropy Coding The DC and AC coefficients DC: (SIZE, AMPLITUDE) finally undergo an entropy coding step to gain a possible further compression. DC: (SIZE, AMPLITUDE) each DPCM coded DC coefficient is represented SIZE indicates how many bits are needed for representing the coefficient AMPLITUDE contains the actual bits. AC: ((Runlength, SIZE), AMPLITUDE) (Runlength, Size) is the Symbol for VLC Coding. Zig-Zag Ordering: Matrix-to-Vector DC … DPCM  (SIZE, AMP) AC … 使用 Runlength 不用 DPCM  ((Runlength, SIZE), AMP)

Example: AC: (0, 3) (4) Insert 0 zeros before using 3 bits to represent a 4 (Note: Shift 只對 DC有作用) (Note: AC 不使用 DPCM) DC: 61=(111101) needs 6 bits VLC(110)是碰巧

Useful image contents change relatively slowly across the image. Observation 2. Useful image contents change relatively slowly across the image. Observation 3. reducing the high spatial frequency contents Block by block Observation 1. sub-sampling (4:2:0) DC encoder DC0, DC1, DC2… AC's AC's AC encoder

break

9.1.2 Four Commonly Used JPEG Modes Lossless Mode -- discussed in Ch 7, and Sect 9.3 JPEG-LS Sequential Mode -- the default JPEG mode, implicitly assumed Each gray level image or color image component is encoded in a single left-to-right, top-to-bottom scan. Progressive Mode. Hierarchical Mode. DC sequence 嵌在每個 8x8 Block串列， 第一次出現時，要記得以DPCM解碼。

Lossless mode Predictive coding

Sequential Mode

Progressive Mode

Hierarchical Mode

Progressive Mode (I) Progressive JPEG delivers low quality versions of the image quickly, followed by higher quality passes. 1. Spectral selection: Takes advantage of the “spectral" (spatial frequency spectrum) characteristics of the DCT coefficients: --- -- higher AC components provide detail information. Note: (1) 表頭都先傳送過去 or 載入 (2) 利用DCT的線性，先還原部份基頻

Spectrum selection Scan 1: Scan 2: ... Scan k: Encode DC and first few AC components, e.g., AC1, AC2. Scan 2: Encode a few more AC components, e.g., AC3, AC4, AC5. ... Scan k: Encode the last few ACs, e.g., AC61, AC62, AC63. 以RLC解出ACk, if k≧K, then 切換下一個 8x8 block

Reconstructing AC Spectrums Recall: + ||

Recall:

Recall:

Progressive Mode (II) 2. Successive approximation: Instead of gradually encoding spectral bands, all DCT coefficients are encoded simultaneously but with their most significant bits (MSBs) first. Note: (1) 表頭都先傳送過去 or 載入 (2) 利用DCT的線性，先還原整數之部份高位元 (8+4+2+1)

Successive approximation Scan 1: Encode the first few MSBs, e.g., Bits 7, 6, 5, 4. Scan 2: Encode a few more less significant bits, e.g., Bit 3. ... Scan m: Encode the least significant bit (LSB), Bit 0. 以2r 為乘數優先編高位元組合, 這樣在使用Runlength 時，亦時常 可以壓縮大量 “0”的記錄而達到效果

Hierarchical Mode The encoded image at the lowest resolution is basically a compressed low-pass filtered image, whereas the images at successively higher resolutions provide additional details (differences from the lower resolution images). Similar to Progressive JPEG, the Hierarchical JPEG images can be transmitted in multiple passes progressively improving quality.

同一張影像要編成 "小"，"中差小"，"大差中"三張影像， 所幸三者的成本總合不大於原影像，只是編解碼變複雜。

Encoder for a Three-level Hierarchical JPEG 1. Reduction of image resolution: Reduce resolution of the input image by a factor of 2 in each dimension f (512 x 512)  f 2 (e.g., 256 x 256). Repeat this  f 4 (e.g., 128 x 128). 2. Compress low-resolution image f 4: Encode f 4 using any other JPEG method (e.g., Sequential, Progressive) to obtain F 4.

3. Compress difference image d 2: (a) Decode F 4 to obtain f 4. Use any interpolation method to expand f 4  to E(f4) ≈ f2 of the same resolution as f 2 (b) Encode difference d 2 = f 2 – E (f 4) using any other JPEG method (e.g., Sequential, Progressive) to generate D2. (c) Repeat this to generate f2, d1, D1 ~ ~ ~ ~ ~

Decoder for a 3 l-level Hierarchical JPEG

Useful image contents change relatively slowly across the image. Observation 2. Useful image contents change relatively slowly across the image. Observation 3. reducing the high spatial frequency contents (1) YIQ or YUV + Subsampling (2) DCT on 8 x 8 blocks (3) Quantization (4) Zig-zag ordering + Run-length (5) Entropy coding Observation 1. sub-sampling (4:2:0)

JPEG Bitstream Q-Tables Y, U, V planes H-Tables Progressive Modes

Recall: Quantization F (u, v) : quantized DCT coefficients Q(u,v) 與 “Quality factor” 呈反比設計 ^ F (u, v) : quantized DCT coefficients F (u, v) : DCT coefficient, Q(u, v) : “quantization matrix"

Different quality of JPEDG, different data rate Factor 5 JPEG size=9438 bytes

Factor 10 JPEG size= 15325 bytes

Factor 50 JPEG size = 46295 bytes

Factor 75 JPEG size = 70586 bytes

Factor 100 JPEG size 326321 bytes Original image TIFF processed by Adobe size 781300 bytes

outline 9.1 The JPEG Standard Appendix 9.2 The JPEG2000 Standard 9.3 The JPEG-LS Standard 9.4 The JBIG Standard

9.2 The JPEG2000 Standard Acceptable Low-Bitrate Compression (不切割 8x8) Design Goals: 修正 JPEG 的不足 (ISO15444) Acceptable Low-Bitrate Compression (不切割 8x8) Coexisting Lossless and Lossy Compression Super Large Image Support (232 pixels 遠大於 216=65536) Single Decompression Architecture (44 into 1) Transmission in Noisy Environments (Arithmetic Coding) Seamless Quality and Resolution Scalability (不限3階) Region-of Interest Coding Computer-Generated Imagery Applicability Compound Document Applicability ( 使用 Wavelet 轉換)

Wavelet Transform Harr Transform: X={10,13, 25,26, 29,21, 7,15} {11.5, 25.5, 25, 11, -1.5, -0.5, 4, -4 } {18.5, 18, -7, 7, -1.5, -0.5, 4, -4} {18.25, 0.25, -7, 7, -1.5, -0.5, 4, -4} (a+b)/2 a b (a-b)/2 Average images d3 d2 d

2D Discrete Wavelet Transform 512 x 512 (a+b+c+d)/4 (a+b-c-d)/4 128 x 128 256 x 256 (a-b+c-d)/4 (a-b-c+d)/4

Performance of JPEG2000

9.3 The JPEG-LS Standard JPEG-LS is in the current ISO/ITU standard for lossless or “near lossless" compression of continuous tone images. It is part of a larger ISO effort aimed at better compression of medical images. Uses the LOCO-I (LOw COmplexity LOssless Compression for Images) algorithm proposed by Hewlett-Packard. (Predictive Method) Main Advantage: Low complexity!

Recall: Lossless JPEG The Predictive method 1. Forming a differential prediction: A predictor combines the values of up to three neighboring pixels as the predicted value for the current pixel, indicated by `X' in Fig. 7.10. The predictor can use any one of the seven schemes listed in Table 7.6. 2. Encoding: The encoder compares the prediction with the actual pixel value at the position `X' and encodes the difference using one of the lossless compression techniques we have discussed, e.g., the Huffman coding scheme.

Neighboring Pixels for Predictor Note: Any of A, B, or C has already been decoded before it is used in the predictor, on the decoder side of an encode-decode cycle. C A X B

C B A X

9.4 JBIG and JBIG-2: Bi-level Image Compression Standards Main Goal: Enables the handing of documents in electronic form. Primarily used to code scanned images of printed or handwritten text, computer generated text, and facsimile transmissions. (1) 主要功能：向量字、圖文合併格式 (2) 被 Adobe’s PDF 取代

JBIG is a lossless compression standard JBIG is a lossless compression standard. It also offers progressive encoding/decoding capability, the resulting bitstream contains a set of progressively higher resolution images. JBIG-2 introduces model-based coding  similar to context based coding. It supports lossy compressions well.

End of Chapter 9