JPEG (Joint Photographic Expert Group) Prepared By: Md. Abedul Haque Std No: 100505010P CSE, BUET

JPEG JPEG is the image compression standard developed by the Joint Photographic Experts Group. In 1992, JPEG became an ISO international Standard. JPEG applies to color and gray-scaled still images.

JPEG requirements The JPEG implementation should be independent of image size. applicable to any image and pixel aspect ratio. Color representation should be independent of the special implementation. Image content may be of any complexity. The JPEG standard specification should be state-of-the-art regarding the compression factor and achieved image quality. Sequential and progressive decoding should be possible.

Steps of JPEG compression The following figure outlines the steps of JPEG compression. Picture Preparation Pixel Block, MCU Picture Processing Predictor FDCT Quantization Entropy Coding Run-Length Huffman Arithmetic

JPEG Modes Four different variants of image compression lead to four modes. The lossy sequential DCT-based mode(baseline process) must be supported by every JPEG implementation The expanded lossy DCT-based mode provides a set of further enhancements to the baseline process. The lossless mode has a low compression ratio but allows perfect reconstruction of the original image. The hierarchical mode accommodates images of different resolutions and selects its algorithms from the three modes of above.

Lossy Sequential DCT-based Mode Image Preparation

Lossy Sequential DCT-based Mode Step: Image Preparation Image Component/Plane: An image consists of at least 1 and at most 255 planes (or component). An RGB image consist of three components(or planes) i.e. RED, GREEN, BLUE RED Plane GREEN Plane BLUE Plane

Image Preparation Image Component/Plane: a gray-scale image consist of a single component i.e. Intensity. An YUV image consist of three planes namely Luminance plane/Y plane Chrominance plane U Chrominance plane V C1: Luminance Y C2: Chrominance U C3: Chrominance V Each small rectangle represents a pixel

Image Preparation Image Component/Plane: Now, unlike RGB image, Y, U, and V plane may be of different resolution. That is the following figure is also possible. In this case, we see that 4 pixels share same chrominance value. Each pixel is represented by p bits. All pixels of all components within the same image are coded with the same number of bits. C1: Luminance Y C2: Chrominance U C2: Chrominance V

Image Preparation Data Unit/Block: Each block is made of 8x8 pixels. This definition of block/data unit comes from DCT. These blocks of 8x8 pixels are transferred to the next step as a unit for processing. There are two ways these data units are passed to the next step. Non-interleaved data ordering: In this case, data units are passed component by component. All the data units from first component are passed and then from the second component and so on. Data units from the each component are passed from left to right, top to bottom order.

To the Image Processing Step Data Unit/Block Non-interleaved data ordering: Using this mode for an RGB-encoded image with very high resolution, the display would initially present only the red component, then green and then blue resulting the original image. Represents data unit of 8x8 pixels To the Image Processing Step

To the Image Processing Step Data Unit/Block Interleaved data ordering: In this approach data units from different components are combined into Minimum Coded Unit (MCU). If all components have the same resolution, an MCU consists of exactly one data unit from each component. MCU1 MCU2 MCU3 To the Image Processing Step MCU4 MCU5 And so on

Data Unit/Block Interleaved data ordering: Example 1: If all components don’t have the same resolution, Different number of data units from each component comprises the MCU. The number of data units from each component is calculated from relative horizontal and vertical sampling ratios. Data units from each component are taken from left to right, top to bottom order. MCUs are also constructed from left to right, top to bottom order. Example 1: Let, three different plane of an image has resolutions as follows: Plane 0: X0 = 512, Y0 = 256 Plane 1: X1= 256, Y1 = 512 Plane 2: X2 = 128, Y2 = 256

Data Unit/Block Example 1: ……………………………………………… 256 512 ………………………………………………………………………………………………………….. 512 256 128 256 …………... Plane 2(128x256 pixels) Plane 0(512x256 pixels) Plane1 (256x512 pixels) Now, if we see data units of each component we find **************************************************************** 32 64 ******************************************************************************** 64 32 Each * represents a data unit i.e. 8x8 pixels 16 32 ********

Data Unit/Block Example 1: Now Hi, and Vi are called relative sampling ratios and calculated for each plane. Hi = Xi / Xmin Vi= Yi / Ymin So, we get Plane 0: H0 = 4, V0 = 1 Plane 1: H1 = 2, V1 = 2 Plane 2: H2 = 1, V2 = 1 Hi, and Vi must be integer values between 1 and 4. Now, a MCU is built by taking H0xV0 data units from plane 0 H1xV1 units from plane 1 H2xV2 units from plane 2

Data Unit/Block Example 1: Example 2: **** * MCU Example 2: Let, an image has four planes with the following dimensions Plane 0: X0 = 48 pixels, Y0 = 32 pixels Plane 1: X1= 48 pixels, Y1 = 16 pixels Plane 2: X2 = 24 pixels, Y2 = 32 pixels Plane 3: X3 = 24 pixels, Y3 = 16 pixels

Data Unit/Block Example 2: Represents a data unit or block i.e. 8x8 pixels H1 = 2, V1 = 1 H3 = 1, V3 = 1 H0 = 2, V0 = 2 H2 = 1, V2 = 2 If we calculate Hi and Vi like previous example we will find MCUs like below MCU1 MCU4 MCU2 MCU5 MCU6 MCU3 Blocks of these MCUs are transferred to the next step.

Lossy Sequential DCT-based Mode Image Processing

Lossy Sequential DCT-based Mode Step: Image Processing In this baseline mode, blocks of 8x8 pixels come to the image processing unit in particular order from the previous step. Each pixel is encoded using 8 bits. First step of image processing is a transformation performed by DCT. FDCT: FDCT transforms an image from spatial to frequency domain. Transforming a 8x8 pixel block through FDCT we find 64 coefficients which can be regarded as a two-dimensional frequency.

Image Processing FDCT: The formula for FDCT is given below Svu represents the DCT coefficients. The co-efficient S00 corresponds to the lowest frequency in both dimensions. It is known as DC-coefficient, which determines the fundamental color of the data units of 64 pixels. The other coefficients are called AC-coefficients.

Image Processing FDCT: Steps of FDCT: The function of the formula Svu is called basis function. The 64 basis functions can be illustrated by the following image. The FDCT transforms the 64 pixels to a linear combination of these 64 squares. Steps of FDCT: At the beginning of FDCT, the pixel values are shifted into the range [-128, 127], with zero as the center. These shifted pixel values are the Syx, used in the formula.

Image Processing Steps of FDCT: Example 3: If one of the blocks is like below, Which after shifting results in,

Image Processing Steps of FDCT: Example 3: And then taking DCT and rounding to the nearest integer results

Lossy Sequential DCT-based Mode Quantization

Lossy Sequential DCT-based Mode Step: Quantization The human eye is fairly good at seeing small differences in brightness over a relatively large area. But not so good at distinguishing the exact strength of a high frequency brightness variation. This fact allows reducing the amount of information in the high frequency components. This is done by simply dividing each coefficient by a constant for that component, and then rounding to the nearest integer.

Quantization This is the main lossy operation in the whole process. As a result of this, it is typically the case that many of the higher frequency components are rounded to zero. A common quantization matrix is(i.e. the numbers by which the coefficients are divided)

Quantization Using this quantization matrix with the DCT coefficient matrix of example 3 results in Here we see, most of the high frequency components are zero. This matrix is sent to the next step for entropy encoding.

Lossy Sequential DCT-based Mode Entropy Encoding

Lossy Sequential DCT-based Mode Step: Entropy Encoding Encoding of DC-coefficient: During the initial step of entropy encoding, a DC-coefficient is encoded as the difference between the current and the previous one. Only the differences are subsequently processed. Blocki-1 Blocki DCi-1 DCi Diff = DCi – DCi-1

Entropy Encoding Encoding of DC-coefficient: The DC coefficients are usually highly correlated, this reduces the entropy of the DC data stream. The result is a series of numbers. Example 4: This may result something like 1,2,-1,0,2,3,-3,..., one number for each block in the image, which is to be compressed with a lossless entropy encoding method. Now these numbers are encoded according to the following table. … -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 -15 -14 -13 -12 -11 -10 -9 8 9 10 11 12 13 14 15

Entropy Encoding Encoding of DC-coefficient: Example 4: The rows are to be extended up to the number 11. Any positive or negative integer in the range -2047 to +2047 can be specified using its row and column index in this table 1 is at (1, 0) in binary (0001,00000000000) represented as (0001, 0) 3 is at (2, 1) in binary (0010,00000000001) represented as (0010, 01) -3 is at (2,-2) in binary (0010,11111111110) represented as (0010, 10) Under the baseline mode the first number( i.e. 0001 of (0001,0) ) is additionally compressed using either a default Huffman table, or optionally one provided with the image.

Entropy Encoding Encoding of DC-coefficient: Example 4: The default table (for luminance DC coefficients) has the mapping 0000 00 0001 010 0010 011 0011 100 0100 101 0101 110 0110 1110 0111 11110 1000 111110 1001 1111110 1010 11111110 1011 111111110 So, the seven coefficients 1,2,-1,0,2,3,-3,..., are represented as 1 010 0 2 011 00 -1 010 1 00 3 011 01 -3 011 10

Entropy Encoding Encoding of AC-coefficient: Zig-Zag ordering of AC-coefficients: Zig-Zag ordering of AC-coefficients: The AC-coefficients are taken in the order shown below. The Zig-Zag sequence actually collects the low frequency coefficients before high frequency coefficients, thereby grouping large numbers at the beginning of the sequence. The Zig-Zag sequence for the example 3 after quantization will be −3, 0, −3, −2, −6, 2, −4, 1, −4, 1, 1, 5, 1, 2, −1, 1, −1, 2, 0, 0, 0, 0, 0, −1, −1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

Entropy Encoding Encoding of AC-coefficient: JPEG has a special Huffman code word for ending the sequence prematurely when the remaining coefficients are zero. Using this special code word, EOB, the sequence becomes −26, −3, 0, −3, −2, −6, 2, −4, 1, −4, 1, 1, 5, 1, 2, −1, 1, −1, 2, 0, 0, 0, 0, 0, −1, −1, EOB Now, this sequence is first encoded using RLE for 0. All sequences of 0s are replaced by a special symbol and then the number of consecutive zeros. Then this sequence is processed in the similar fashion, i.e. each coefficient is transformed to two numbers using the table and then the first number is Huffman coded.

Lossy Sequential DCT-based Mode In this case of lossy sequential encoding, the whole image is coded and decoded in a single run. The figure below shows an example of decoding with immediate presentation; the picture is presented from top to bottom. Sequential picture presentation used in the lossy DCT-based mode

Expanded Lossy DCT-based Mode

Expanded Lossy DCT-based Mode Step: Image Preparation The difference in this step from the previous mode is, a sample precision of 12 bits per sample, in addition to 8 bits per sample, can be used. Step: Image processing: Image Processing is DCT-based and follows rules analogous to be baseline DCT mode.

Expanded Lossy DCT-based Mode Step:Quantization and Encoding In this mode, JPEG specifies progressive encoding in addition to sequential encoding. In the first run, a very rough representation of the image appears which looks out of focus and is refined during successive steps. Progressive picture presentation used in the expanded DCT-based mode

Expanded Lossy DCT-based Mode Step:Quantization and Encoding Progressive image representation is achieved by expansion of quantization. A buffer is added at the output of the quantizer that temporarily stores all coefficients of the quantized DCT. Progressiveness is achieved in two different ways By using a spectral selection. DC component and first few AC coefficients are sent first, then gradually some more AC components.

Expanded Lossy DCT-based Mode Spectral selection: In the following figure we see how selection of more coefficients refines the image. In the figure, original image and reconstructed image using the additional coefficients are shown. 1 AC coefficient 2nd AC coefficient After sending

Expanded Lossy DCT-based Mode Spectral selection: After sending 3rd-5th AC coeff. 6th -9th AC coeff. 10th -15th AC coeff. 16th -20th AC coeff. 20th -30th AC coeff. 30th -40th AC coeff.

Expanded Lossy DCT-based Mode Step:Quantization and Encoding Successive approximation: In this way all the coefficients are sent in each run but single bits are differentiated according to their significance. The MSB bits are encoded first and then the LSB. For example, To encode the coefficient 13(1101), 8(1000) is encoded in the first run, 4(100) is encoded in the second run, 0(00) is encoded in the third run and 1(1) is encoded in the fourth run. So, the picture is refined gradually.

Expanded Lossy DCT-based Mode So, image display in this mode may be sequential or Progressive, bits per sample may be 8 or 12 and Entropy coding may be Huffman or arithmetic. This leads to 12 alternative processing modes. Image Display Bits Per Sample Entropy Encoding Sequential 8 Huffman Coding Arithmetic Coding 12 Progressive Successive Progressive Spectral

Lossless Mode

Lossless Mode Data units of single pixels are taken for image preparation. Any precision between 2 and 16 bits per pixel can be used Image processing and quantization use a predictive technique. For each pixel X, one of eight possible predictions is selected. The selection criterion is a prediction that is as good as possible. C B A X Principle of the prediction in the lossless mode

Lossless Mode The number of the chosen predictor, as well as the difference of the prediction to the actual value is passed to the entropy encoding step. Entropy encoding can use Huffman or arithmetic encoding techniques. Selection Value Prediction No Prediction 1 X=A 2 X=B 3 X=C 4 X=A+B-C 5 X=A+(B+C)/2 6 X=B+(A-C)/2 7 X=(A+B)/2 Predictors for lossless coding

Hierarchical Mode

Hierarchical Mode The hierarchical mode uses either the lossy DCT-based algorithms or the lossless compression. The main feature of this mode is the encoding of an image at different resolutions. The prepared image is initially sampled at a lower resolution (down sampled by 2n ). That smaller image is coded using another JPEG mode (Progressive, Sequential, or Lossless). Then the image is decoded and up-sampled. The difference between the up-sampled and the original image is encoded using Progressive, Sequential, or Lossless This process can be repeated multiple times. Good for viewing high resolution image on low resolution display.

Hierarchical Mode A three level JPEG encoder is shown below.

The End