4C8 Dr. David Corrigan Jpeg and the DCT

2D DCT

Q step = 15

Each band is the same size and there are 64 bands in total so the entropy is

Optimum Block Size is 8!

Slow DCT Sledgehammer implementation for 8 point DCT Each row multiply requires 8 MADDs (approx) So for all 8 rows requires 64 MADDs (approx)

Fast DCT Exploit Symmetry

Fast DCT So split Matrix T into two parts...

Fast DCT split Matrix T into two parts, change y...

Fast DCT 4 “adds”, 16 MADDS for each operation = 8 adds and 32 MADDS = 40 ops Compare with 64 MADDS from before.

Fast DCT This sub-matrix can be simplified with symmetry again! 4 “adds”, 8 MADDS in total = 12 ops (down from 20) So now we are at 20 (for the first sub matrix) + 12 (for these two) = 32 ops So we have saved about x2!

JPEG and Colour Images JPEG uses YC B C R colourspace. The chrominance channels are usually downsampled. There are 3 commonly used modes – 4:4:4 – no chrominance subsampling – 4:2:2 – Every 2 nd column in the chrominance channels are dropped. – 4:2:0 – Every 2 nd column and row is dropped.

Subjectively Weighted Quantisation In JPEG it is standard to apply different thresholds to different bands

Subjectively Weighted Quantisation

Lower Frequency Bands are assigned lower step sizes. There is a slight drop of in step size from the DC coefficient to low frequency coefficients. The step sizes for the chrominance channels increase faster than for luminance.

We have seen this before

Comparing Different Quantisations Q step = Q lum Uncompressed JPEG

Comparing Different Quantisations Q step = Q lum PSNR = 32.9 dB

Comparing Different Quantisations Q step = 2 * Q lum PSNR = 30.6 dB Uncompressed JPEG

Comparing Different Quantisations Q step = 15 PSNR = 37.6 dB Q step = Q lum Q step = 15

Comparing Different Quantisations Q step = 30 PSNR = 33.4 dB Q step = Q lum Q step = 30

Comparing Different Quantisations Q step = 30PSNR = 33.4 dB Q step = Q lum Q step = 30 PSNR indicates better quality for Q step = 30 over Q step = Q lum but this clearly is not true from a subjective analysis.

Comparing Different Quantisations QuantisationPSNR (dB) Subjective Ranking Entropy (bits/pel) * Q lum Q lum *Q lum Using the subjectively weighted Quantisation achieves much higher levels of compression for equivalents levels of quality.

JPEG Coding The most obvious way might seem to code each band separately – ie. Huffman with RLC like we suggested with the Haar Transform. – We could get close to the entropy This is not the way it is coded because – It would require 64 different codes. High cost in computation and storage of codebooks. – It ignores the fact that the zero coefficients occur at the same positions in multiple bands.

JPEG Coding Instead we code each block separately – A block contains 64 coefficients, one from each band. Each block contains 1 DC coefficient (from the top left band) and 63 AC coefficients Two codebooks are used in total for all the blocks, one for the DC coefficients and the other for the AC coefficients. At the end of each Block we insert an End Of Block (EOB) symbol in the datastream

Data Ordering Each block covers is a 8x8 grid of coeffs – A Zig-Zag scan converts them into a 1D stream. – As most non-zero values occur in the top left corner using a Zig-Zag scan maximises the lengths zero runs so improves efficiency of RLC

Zig-Zag Scan Example -13, -3, 6, 0, 0, 2, 0, 0, 0, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 36 more zeros, the end Typical DCT Block Coefficients Non-Zero values are at the top left corner of the block Zig-Zag scan concentrates the non-zero coefficients at the start of the stream

Coding the DC Coefficients Differential Coding

Coding the DC Coefficients This value is actually the difference between the dc coefficient of the current and previous blocks Typical DCT Block Coefficients

Coding DC Coefficients There is potentially a large number of levels to encode. – Up to 4096 depending on the quantization step size. We break down the symbol value into a size index pair

Coding DC Coefficients So if the DC value is -13 – The size is 4 – The index is 0010 In JPEG only the size is encoded using Huffman – The index is uncoded, efficiency is not dramatically affected. – Only 12 codes required in huffman table – Table size is = 28 bytes

ValueSizeIndex More examples of Coefficient to size/index pair conversions

Coding the AC Coefficients , -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end Typical DCT Block Coefficients The block usually ends with a long run of zeros The length of the run and the value of the coeff after it are strongly correlated Size/Index Pair for DC coefficient

Coding the AC Coefficients Code/Size Correlations – High coeffs follow short runs and low coeffs follow long runs Final run of zeros – These don’t need to be coded – Just tell the encoder that there are no more non- zero coefficients and move onto the next block.

Symbols Run/Coefficient Symbols eg. 0, 0, 9 is a run of 2 zeros followed by a 9 However we represent 9 using the size/index format from the dc coeffs 9 has a size of 4 and an index 1001 So we code the run/size pair (2,4) and the index 1001 is appended to the stream

Symbols Run/Size Symbols – All possible combinations of runs from 0->15 and size from 1->10 – 160 total symbols – Huffman Codes are used for each symbol – Index values are not coded further

Special Symbols ZRL – Used to represent a run of 16 zeros – Used when the run of zeros is greater than 15 – Eg. 17 zeros, 14 - is coded as (ZRL) (1,4) 1110 EOB – Inserted when a block ends with a run of zeros In total there are 160 run/size symbols and 2 special symbols 162 symbols to 2 encode codetable is = 178 bytes

Coding Example Typical DCT Block Coefficients -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end DC Coefficient is -13. The size is 4 and the index is 0010 Current Stream State:

Coding Example -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end The first ac value is -3. That is a run of 0 zeros followed by has size 2 and index 0000 Therefore the run/size pair is (0,2) Current Stream State: (0,2) 00

Coding Example -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end The next ac value is 6. That is a run of 0 zeros followed by 6. 6 has size 3 and index 110 Therefore the run/size pair is (0,3) Current Stream State: (0,2) 00 (0,3) 110

Coding Example -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end The next ac value to encode is a run of 2 zeros followed by a ac coefficient 2. 2 has size 2 and index 10 Therefore the run/size pair is (2,2) Current Stream State: (0,2) 00 (0,3) 110 (2,2) 10

Coding Example The next ac value to encode is a run of 3 zeros followed by a ac coefficient has size 1 and index 0 Therefore the run/size pair is (3,1) Current Stream State: (0,2) 00 (0,3) 110 (2,2) 10 (3,1) 0 -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end

Coding Example The next ac value to encode is a run of 17 zeros followed by a ac coefficient 1. As the run is > 15 zeros we have to use the ZRL symbol to code the first 16 zeros. The remaining run length consists of ( ) = 1 zero. An ac coefficient of 1 has size 1 and index 1 Therefore we insert the run/size pair (1,1) after the ZRL marker Current Stream State: (0,2) 00 (0,3) 110 (2,2) 10 (3,1) 0 ZRL (1,1) 1 -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end

Coding Example The remaining coeffs are all 0. Therefore the EOB marker is used. If the last ac coeff is non-zero, then the EOB marker is not used. Current Stream State: (0,2) 00 (0,3) 110 (2,2) 10 (3,1) 0 ZRL (1,1) 1 EOB -13, -3, 6, 2 zeros, 2, 3 zeros, -1, 17 zeros, 1, 36 more zeros, the end

Huffman Coding Best Solution is to define the 2 Huffman codes for each image during compression However a default Huffman codetable is defined in the JPEG standard. Final Stream: (0,2) 00 (0,3) 110 (2,2) 10 (3,1) 0 ZRL (1,1) 1 EOB Encoded using dc codetable Encoded using ac codetable No further encoding

Default Codetables AC table DC table Final Stream: (0,2) 0000 (0,3) 110 (2,2) 10 (3,1) 0 ZRL (1,1) 1 EOB Fully Encoded Stream: bits to encode 64 coefficients = bits/coefficient

How good is this scheme?

Should we use default codetables? Even though doubling the quantisation sizes reduces the number of events the distribution of those events doesn’t change much. Only the EOB probability changes significantly. Therefore using the same codetable for both cases is reasonable

How good is this scheme? In fact using the same codetable for multiple images doesn’t reduce the efficiency of the code much. Efficiency when the default codetable is used 97.35% 95.74%

Special Markers

Synchronisation Markers There are 8 synch markers FFD0 ->FFD7 They can be placed at intervals which can be specified by using the DRI (FFDD) marker Each marker is sent sequentially so if any marker is corrupted its absence can be easily detected.

Summary We have covered the basics of JPEG standard The standard specifies a syntax rather than specifying exactly how it is implemented Most implementations use the recommended settings provided by the JPEG community.