Presenter: Tzu-Heng Henry Lee Research Advisor: Jian-Jiun Ding, Ph. D. Assistant professor Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University

 Introduction to Shape-Adaptive Image Compression  Morphological Segmentation Using Erosion  Shape-Adaptive Transform Algorithm  Quantization  Coding Technique of the Image Segment  Simulations  Conclusion and Future Work September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 2

 The idea is to exploit high correlation between the color values in the neighboring pixels within the same image segment.  Characteristics in an image segment usually share the similar color values(the color intensity variations are low).  The arbitrarily-shaped image segment can be completely represented by its shape and internal contents [1]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 3

 JPEG images normally display various kinds of undesired distortion artifacts such as  blocking,  blurring, and  Ringing.  Compressions with low bit-rates  Lossy quantization process is used to compress the DCT coefficients. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 4

 key features that distinguish the improved algorithm are built around two central components: Morphological segmentation, and Shape-adaptive DCT with orthogonal bases. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 5

Q:Why do we include this stage in our algorithm? A:  The color values at the edge of an segmented object usually vary significantly.  The contour region of a segment contains a great portion of the high frequency components Q:Why do we include this stage in our algorithm? A:  The color values at the edge of an segmented object usually vary significantly.  The contour region of a segment contains a great portion of the high frequency components September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 6

 This allows us to compress the contour sub- region and the interior sub-region of an arbitrarily image segment separately.  So we can minimize quantization noise and enhance overall quality of image compression. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 7 Contour sub-region Interior sub-region The overall internal region

September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 8

 Traditional Method:  fill zeros outside the contour of the arbitrarily image and treat the whole image block as a traditional image block [2].  Drawback:  This increases the high-order transform coefficients which are later truncated.  Leads to performance degradation. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 9

 Based on the concept of KLT(Karhunen-Loeve).  Generic transform that does not need to be computed for each image can be derived.  Lower compuational complexity.  Provides a good compromise between information packing ability and computational complexity [A1]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 10

 DCT produces less blocking artifacts compared to DFT.  1-D point of view.  The implicit n-point periodicity of the DFT  boundary discontinuities  High freq  Truncation  Gibb’s phenomenon  The DCT which has the implicit 2n-point periodicity does not produce such discontinuities [1]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 11

 DCT-based.  Since the height and width of an arbitrarily- shaped image segment are usually not the same,  we redefine the forward DCT as  for and September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 12

 The inverse DCT can also be re-written as  and the DCT basis is expressed as September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 13 The DCT bases are not yet customized for a particular arbitrarily-shaped image segment.

 Since we are using the traditional DCT bases, we can simply project these basis functions into subspace S B :  A linear combination of can be used to describe the arbitrarily segment vector P(x, y).  This operation removes the components of outside subspace S B [2]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 14

 An example of the projection operation:  shape matrix:formed by filling 1’s in the pixel position inside the contour of the arbitrary shape. Zeroes are filled in the region outside the contour. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University

 The 8  8 DCT bases with the shape of our example. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 16

 The number of orthogonal bases M is less than H  W  The same basis function could be repetitively chosen.  Generally the H  W shape-projected bases are not orthogonal because the number of transform coefficients may exceed the image segment size [2]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 17

 One of the methods to obtain orthogonal basis functions in the subspace S B is to use the Gram-Schmidt algorithm [2], [3], [4], [5].  We use the Gram-Schmidt process to reduce the bases to M orthogonal ones. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 18

 Before we use the Gram-Schmidt process to reduce the bases to M orthogonal ones, we reorder the H  W shape-projected bases by the zig-zag reordering matrix [6]: September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 19

 make the low frequency components to concentrate on the top-left corner  move the less important high frequency components to concentrate on the bottom-right corner of the matrix.  This is because the low frequency components contain a significant fraction of the total image energy [7]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 20

 Low-frequency AC coefficients are placed before high-frequency AC coefficients.  Makes upcoming entropy coding process much easier.  By keeping higher frequency coefficients (which are more likely to be zero after quantization) together, we can form long runs of zeros [8]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 21

 JPEG – fixed quantization matrix for 8X8 blocks  Our method – The length of the quantization array corresponding to the arbitrary-shape DCT coefficients is not fixed.  We define an extendable and segment shape- dependent quantization array Q(k) as a linearly increasing line: for k = 1, 2,…, M. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 22

 Need to encode the quantized arbitrary-shape DCT coefficients to bit stream.  We use the same coding technique that is used in JPEG.  The quantized coefficients are a series of integer values with large values at the beginning(DC terms) of the series followed by a large amount of zeros at the back(AC terms). September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 23

 The DC coefficient is treated separately from the AC coefficients.  Difference Encoding: It is encoded as the difference between the present DC term and the one from the previous block.  The AC-terms are encoded by zero-run length coding(ZRL) and the Huffman coding [6], [7]. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 24

 We directly combine all the encoded bit streams of all image segments.  In the ZRL coding process, we truncate the successive zeroes in the end of the coefficients, and replace them with an end-of-bit (EOB) symbol.  We can divide the bit stream to each image segment by the EOB symbol in the decoding process. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 25

September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 26

September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 27

September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 28 Shape-Adaptive Compression with Morphological Segmentation JPEG

September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 29 (Top) Original image (11080 bytes). (Left) JPEG compressed image (RMSE: and data size: 1128 bytes). (Right) Compressed image using our proposed algorithm (RMSE: and data size: 410 bytes) (Top)Original image (11080 bytes). (Left) JPEG compressed image (RMSE: and data size: 1428 bytes). (Right)Compressed image using our proposed algorithm (RMSE: and data size: 419 bytes)

 The complexity of Gram-Schmidt orthogonal process: O(n 2 )  n - the number of points of an image segment September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 30 What if an image segment is large? It would cost a lot of computational time

 Solutions: 1. Segment the image in more detail such that the number of points of an image segment is confined in an acceptable range. 2. A number of bases smaller than the dimension of the image segment can be chosen to avoid n being too large September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 31

 The JPEG method has a poorer because it cannot utilize the characteristics of the image.  Significant improvements on the distortion artifacts caused by the quantization process are made by using the shape-adaptive compression algorithm with morphological segmentation.  A higher compression rate with a comparable RMSE. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 32

 Improvements on Huffman Coding algorithm.  More efficient ways to segment the image.  Improvements on compression efficiency.  Elimination of the flaws on the erosion operation(small segment problems ) September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 33

 [1]R. C. Gonzalez and R. E. Woods, Digital Image Processing Second Edition, Prentice Hall, New Jersey,  [2]S. F. Chang and D. Messerschmitt, “Transform coding of arbitrarily shaped image segments,” Proc. 1st ACM Int. Conf. Multimedia Anaheim, CA, pp ,  [3]M. Gilge, T. Engelhardt, and R. Mehlan, “Coding of arbitrarily shaped image segments based on a generalized orthonormal transform,” Signal Process: Image Commun., vol. 1, pp. 153–180, Oct  [4]J. Apostolopoulos and J. Lim, “Coding arbitrarily-shaped regions,” Proc. SPIE Visual Commun. Image Process., pp , May  [5]R. Stasinski and J. Konrad, “A new class of fast shape-adaptive orthogonal transforms and their application to region-based image compression,” IEEE Trans. on Circuits and systems for Video Technology, vol. 9, pp. 16–34,  [6]W. B. Pennebaker and J. L. Mitchell, JPEG Still Image Data Compression Standard. New York: Van Nostrand Reinhold,  [7]C. K. Wallace. The JPEG still picture compression standard. Communications of the ACM, 34(4):31-44,  [8]T. Acharya amd A. K. Ray, Image Processing Principles and Applications, John Wiley & Sons, New Jersey. September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 34

September 18, 2015 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University 35 Duh?