Iterated Denoising for Image Recovery Onur G. Guleryuz To see the animations and movies please use full-screen mode. Clicking on pictures to the left of PSNR curves should start the movies. There are also reminder notes for some slides. Presentation given at DCC 02.

Overview Problem definition. Main Algorithm. Rationale. Choice of transforms. Many simulation examples, movies, etc. Brought code. Can run for other images, for your images, etc. If interested, please find me during breaks or evenings. Errata for manuscript. Notices:

Problem Statement Image Lost Block Use surrounding spatial information to recover lost block via overcomplete denoising with hard-thresholding.* Generalizations: Irregularly shaped blocks, partial information,... Pretend “Image + Noise” Applications: Error concealment, damaged images,...

What is Overcomplete Denoising with Hard-thresholding? x y DCT (MxM) tilings Image Hard threshold coefficients with T Partially denoised result 1 Hard threshold coefficients with T Partially denoised result 2... Average partially denoised results for final denoised image. Utilized transform will be very important!

Examples (Figure 1 in the paper) dB dB dB dB

Main Algorithm I Denoising with hard-thresholding using overcomplete transforms Recover layer P by mainly using information from layers 0,…,P-1 (Figure 2 in the paper)

Main Algorithm II Assign initial values to layer pixels.  for i=1: number_of_layers recover layer i by overcomplete denoising with threshold T  end  T=T- dT T=T 0 while ( T > T ) F end

th k DCT block *Main Algorithm III x y DCT (MxM) tiling 1 Outer border of layer P Image Lost block o (k) y x Hard threshold block k coefficients if o (k) < M/2 y x OR (Figure 3 in the paper)

(Figure 4 in the paper) Example DCT Tilings and Selective Hard Thresholding

Rationale: Denoising and Recovery Main intuition: Keep coefficients of high SNR, zero out coefficients of low SNR. original transform coefficient error Assume that the transform yields a sparse image representation: Hard thresholding removes more noise than signal.

Rationale: Other Analogies Band limited reconstructions via POCS: Set of bandlimited (low pass) signals Set of possible signals given the available information.... Assumes low frequency Fourier coefficients are important and zeros out high frequencies coefficients. This work: Adaptively change sets at each iteration. Let data determine the important coefficients and which coefficients to zero out. Best subspaces to zero-out in a POCS setting. Optimal linear estimators. Sparse transforms.

Properties of Desired Transforms Periodic, approximately periodic regions: Transform should “see” the period Example: Minimum period 8 at least 8x8 DCT, ~ 3 level wavelet packets. Edge regions (sparsity may not be enough): Transform should “see” the slope of the edge.

Periodic Example (Figure 1 in the paper) DCT 9x dB

Periodic Example (period=8) (Figure 5 in the paper) DCT 8x8 Perf. Rec.

Periodic Example (Figure 6 in the paper) DCT 16x dB

Periodic Example DCT 24x dB

“Periodic” Example DCT 16x dB

“Periodic” Example DCT 24x dB

Edge Example DCT 8x dB

Edge Example (Figure 6 in the paper) Complex wavelets dB

Edge Example (Figure 6 in the paper) Complex wavelets dB

Edge Example (Figure 6 in the paper) DCT 24x dB

Edge Example (Figure 1 in the paper) Complex wavelets dB

Unsuccessful Recovery Example (Figure 7 in the paper) DCT 16x dB

Partially Successful Recovery Example (Figure 7 in the paper) DCT 16x dB

Edges and “Small Transforms” DCT 4x dB

Edges and “Small Transforms” dB DCT 4x4

Edge Example (Figure 6 in the paper) DCT 24x dB