EE5965 Advanced Image Processing Copyright Xin Li Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image smoothing based approach – Wavelet-thresholding based approach Deringing of wavelet coded images – Re-compression approach – PDE-based approach Post-processing by alternating projections: a unified approach

EE5965 Advanced Image Processing Copyright Xin Li Block Artifacts Smooth areas become blocky JPEG decoded image at 0.23bpp zoom

Why block artifacts occur? EE5965 Advanced Image Processing Copyright Xin Li x f(x) original x f(x) ^ B2B3B JPEG decoded at low bit rate Only DC component is preserved

EE5965 Advanced Image Processing Copyright Xin Li Deblocking as Denoising Standard image denoising algorithm JPEG compressed image postprocessed image th 1000 Manually tune the threshold parameter!

EE5965 Advanced Image Processing Copyright Xin Li Experiment Results before deblockingafter deblocking

EE5965 Advanced Image Processing Copyright Xin Li Wavelet-based Deblocking Behavior of block artifacts in wavelet domain

Fundamental Issues behind Deblocking Motivation - modeling uncertainty – Location of block artifacts is known (block boundaries) – How to distinguish significant coefficients generated by artifacts from those associated with true edges Strategy – Recall how JPEG2000 is free from block artifacts EE5965 Advanced Image Processing Copyright Xin Li

8 Deblocking via Wavelet Thresholding Wavelet Transform Block Wavelet Transform reorder X X Y1Y1 Y2Y2 X: JPEG decoded image

Apply both WT and block-based WT to X to get Y 1,Y 2 ; Locate the coefficients at block boundaries; If |Y 1 (i,j)|>T and |Y 2 (i,j)|<T, apply soft thresholding to Y 1 (i,j); Apply IWT to processed Y 1 to obtain deblocked image EE5965 Advanced Image Processing Copyright Xin Li Deblocking Algorithm

EE5965 Advanced Image Processing Copyright Xin Li Example Before deblocking (PSNR=27.39dB) After deblocking (PSNR=28.07dB)

EE5965 Advanced Image Processing Copyright Xin Li Ringing Artifacts Sharp edges become unnatural JPEG2000 decoded image at 0.125bpp zoom

Why ringing artifacts occur? EE5965 Advanced Image Processing Copyright Xin Li x(n) H1H1 Key observation: wavelet transform lacks translation invariance 2 x(n-1) H1H1 2 origin

EE5965 Advanced Image Processing Copyright Xin Li Deringing by Re-compression JPEG: JPEG2000 encoder JPEG -1 : JPEG2000 decoder

Example EE5965 Advanced Image Processing Copyright Xin Li before processing after processing

PDE-based Deringing The power of anisotropic diffusion – Nonlinear diffusion can handle a variety of noise – Which PDE is suitable for deringing? – Implication into wavelet coding EE5965 Advanced Image Processing Copyright Xin Li

EE5965 Advanced Image Processing Copyright Xin Li Perona-Malik Filtering PSNR=30.86dB PSNR=31.09dB

EE5965 Advanced Image Processing Copyright Xin Li Mean Curvature Filtering PSNR=31.09dB PSNR=30.27dB

EE5965 Advanced Image Processing Copyright Xin Li Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image smoothing based approach – Wavelet-thresholding based approach Deringing of wavelet coded images – Re-compression approach – PDE-based approach Post-processing by alternating projections: a unified approach

Recall: Alternating Projection EE5965 Advanced Image Processing Copyright Xin Li X0X0 X1X1 X2X2 X∞X∞ Projection-Onto-Convex-Set (POCS) Theorem: If C 1,…,C k are convex sets, then alternating projection P 1,…,P k will converge to the intersection of C 1,…,C k if it is not empty Alternating projection does not always converge in the case of non-convex set. Can you think of any counter-example? C1C1 C2C2

EE5965 Advanced Image Processing Copyright Xin Li Projection Operators ● Constraint set y y+T/2 y-T/2 ● Constraint set x B x B fP s (f)

EE5965 Advanced Image Processing Copyright Xin Li Projection-based Deblocking ● DCT quantization set DCT Quantization ● Smoothness constraint set C s ={f|f is smooth in the block boundaries} at block boundaries Linear edge detection operator Yongyi Yang; Galatsanos, N.P.; Katsaggelos, A.K.;, "Projection-based spatially adaptive reconstruction of block-transform compressed images,“ IEEE Trans. on Image Proc., vol.4, no.7, pp , Jul 1995

EE5965 Advanced Image Processing Copyright Xin Li Projection-based Deringing ● WT quantization set WT Quantization ● Smoothness constraint set Perona-Malik diffusion as a nonlinear projection operator Xin Li;, "Improved wavelet decoding via set theoretic estimation," IEEE Trans. on CSVT, vol.15, no.1, pp , Jan. 2005

Algorithm Flowchart EE5965 Advanced Image Processing Copyright Xin Li C 1 : observation constraint set C 2 : regularization constraint set

Summary Connection with models (PDE-based, wavelet- based, patch-based) – They serve as image prior/regularization constraint set – Jointly work with quantization (observation data) constraint set Convergence is NOT always guaranteed but can be terminated strategically. EE5965 Advanced Image Processing Copyright Xin Li