A method for blind estimation of spatially correlated noise characteristics Nikolay N. Ponomarenkoa, Vladimir V. Lukina, Karen O. Egiazarianb, Jaakko T. Astolab a National Aerospace University, 61070, Kharkov, Ukraine; b Tampere University of Technology, FIN 33101, Tampere, Finland Nikolay Ponomarenko 19/1/2010

Images formed by Earth observation remote sensing (RS) systems Reasons of noise presence in formed images Propagation medium Apparatus imperfection Properties of registered data Distortions due to propagation medium Example: optical image registration Poisson distribution Light photons Nikolay Ponomarenko 19/1/2010

Design of a new method for digital image processing Processing method Processed image Image mathematical model Quality metric that characterizes designed method efficiency Distorted image Noise mathematical model Noise type and PDF: Gaussian Poisson Rayleigh Others Type of noise dependence on signal: Additive Multiplicative More complex dependence Spatial correlation: White (i.i.d.) noise Colored noise Nikolay Ponomarenko 19/1/2010

Spatially correlated (colored) noise in images (example) Process of image forming (processing) in digital cameras Image registration by camera matrix Mosaic image Interpolation Gamma-correction JPEG R G B White Poisson (or mixed Poisson + additive) noise Noise becomes not-Poisson and spatially correlated Noise becomes signal dependent with complex dependence Methods designed for white noise removal ARE NOT EFFICIENT for suppressing spatially correlated noise A filter should be adapted to properties of spatially correlated noise One has to know noise spect-rum It is necessary to estimate noise spectrum for an image to be processed Nikolay Ponomarenko 19/1/2010

Main problem – absence of absolutely homogeneous regions in real life images If there are quite many homogeneous regions in an image where only noise is present it is easy to estimate spatial spectrum of noise Image Detection and localization of image homogeneous regions Rejection of fragments with (potentially) abnormal distributions Averaged spectrum estimation Textural region (example) Quasi-homogeneous region Real life image fragment (example) Nikolay Ponomarenko 19/1/2010

Proposed approach to blind estimation of noise spatial spectrum The proposed approach combines four mutually complementing tools for blind estimation of noise spatial spectrum : DISCRETE COSINE TRANSFORM (DCT): being applied to blocks of natural images, it is able to decorrelate pixels values well (to separate data into information and noise components). IMAGE FRAGMENT SELF-SIMILARITY: information component for some image blocks can be similar whilst noise component is commonly not. ROBUST ESTIMATION: is used in order to diminish negative influence of image information component on estimation of noise variance and spatial spectrum. Method output –8x8 matrix of noise variance for spectral coefficients of DCT. Note that spatial spectrum estimation in DCT domain can be useful for DCT-based filtering and compression of noisy images. Image Image pixel decorrelation in blocks by means of DCT Pair-wise search of the most similar blocks Obtaining of robust estimates of variance and spectrum Nikolay Ponomarenko 19/1/2010

Details of the proposed method 1. The base block size is selected; below we use 8x8 pixel blocks since this size is convenient for accelerating search of similar blocks without losing the method efficiency and accuracy. 2. A random mask δ that has the values either 0 or 1 is generated; this mask separates DCT coefficients into two halves; the value 1 refers DCT coefficient to the first half and vice versa. Example of the generated δ 1 Nikolay Ponomarenko 19/1/2010

Details of the proposed method 3. For each image block A (position of the scanning window of size 8x8 pixels), its similarity with respect to all blocks B in the neighborhood of A is analyzed where distance R(A, B) is larger than r1 and smaller than r2. The distance R(A, B) is defined as max(AX-BX, AY-BY) where AX, AY and BX, BY are coordinates (indices) X and Y of left upper corners of blocks A and B. The recommended values r1 and r2 are r1=3 and r2=10. Similarity of the blocks A and B is calculated in the DCT domain according to (1) with taking into account the mask δ: MSED(A, B) = , where AD and BD are DCT matrices (coefficient sets) for blocks A and B. Nikolay Ponomarenko 19/1/2010

Details of the proposed method 4. The array D is used for storing K results of DCT(A-B) that correspond to minimal found values MSED(A, B). The recommended value K for images of size 512x512 pixels is K=1025. 5. As the result of carrying out the steps 1…4 for each pair of indices of DCT coefficients one obtains K values of these coefficients. The coefficients for which δ is equal to 1 are the results of searching the minimal values. Thus, it is impossible to use them for estimation of noise parameters. Other coefficients for which δ is equal to 0 in ideal case (for absolutely similar blocks) relate to noise component. In practice, considerable part of these coefficients can correspond to information content. This can lead to heavier tails of coefficient distribution. Thus, it is desirable to apply robust approaches to estimating σij2 for each coefficient with δij=0. One way to do this is to apply the estimate where 1.483 is the correcting factor. Nikolay Ponomarenko 19/1/2010

Details of the proposed method It is also possible to evaluate tail heaviness for the considered distributions that characterizes confidence of the obtained estimates. The estimate can be considered confident if The recommended value of Tr is Tr=6. 6. After performing the steps 1…5, one has the estimates for one half of DCT coefficients of the block. Then, the mask δ is inverted (δnew = 1 - δ) and the steps 1…5 are repeated one more time; as the result, the estimates are obtained for all other coefficients of the block. 7. The values of estimates that are not confident can be replaced by the minimal confident estimate of the nearest coefficients of the matrix. If the total number of non-confident estimates is too large, e.g., 50%, then the decision that the considered method is not worth applying for estimation of noise parameters for a given image can be undertaken. Nikolay Ponomarenko 19/1/2010

Quality criterion of an accuracy of noise spectrum estimation Estimation of parameters for spatially correlated additive Gaussian noise Quality criterion of an accuracy of noise spectrum estimation Recall that the main intention of such estimation is to accurately evaluate the matrix . Integrally, accuracy of estimation can be characterized by the parameter where L=MN-1 (M and N define the block size and are both equal to 8 for the considered case), is the estimated standard deviation of noise for DCT coefficient with indices i and j, σij is its true value that has been determined by simulations for the image Homog (large homogeneous region) with multiple realizations of spatially correlated noise. Nikolay Ponomarenko 19/1/2010

Estimation of parameters for spatially correlated additive Gaussian noise Compared methods Method Reference PGE Foi, A. Practical denoising of clipped or overexposed noisy images, Proceedings of the 16th European Signal Process. Conf., EUSIPCO 2008, 2008, pp. 1-5 SE0 Anisotropic Nonparametric Image Restoration DemoBox (available at http://www.cs.tut.fi/~lasip/2D/) AIQRF V.V. Lukin, S.K. Abramov, A.A. Zelensky, J. Astola, B. Vozel, B. Chehdi, Improved minimal inter-quantile distance method for blind estimation of noise variance in images, Proceedings of SPIE Conference Image and Signal Proc. for Remote Sensing, 2007, SPIE Vol. 6748, pp. 1-12 RDCT N.N. Ponomarenko, V.V. Lukin, S.K. Abramov, K.O. Egiazarian, J.T. Astola , Blind evaluation of additive noise variance in textured images by nonlinear processing of block DCT coefficients, Proc. of Int. Conf. Image Proc.: Algor. and Systems II, Santa Clara, USA, Jan. 2003, SPIE Vol. 5014. pp. 178-189 BECNS The proposed method Nikolay Ponomarenko 19/1/2010

MSE ξ of estimation of spatially correlated noise DCT spectrum Estimation of parameters for spatially correlated additive Gaussian noise MSE ξ of estimation of spatially correlated noise DCT spectrum Nikolay Ponomarenko 19/1/2010

Estimation of parameters for spatially correlated additive Gaussian noise True and estimated parameters of spatially correlated noise for σ2=100 for the highly textural image (Baboon) Nikolay Ponomarenko 19/1/2010

Filtering results (PSNR, ΔPSNR, dB) for spatially correlated noise Filtering efficiency based on estimated parameters of spatially correlated additive Gaussian noise (in terms of PSNR) Filtering results (PSNR, ΔPSNR, dB) for spatially correlated noise Nikolay Ponomarenko 19/1/2010

Filtering efficiency based on estimated parameters of spatially correlated additive Gaussian noise (visual quality) Filtering results (PSNR-HVS-M, ΔPSNR-HVS-M, dB) for spatially correlated noise Nikolay Ponomarenko 19/1/2010

Output of conventional Output of modified (adaptive) Efficiency of two-stage automatic procedure Noise parameters estimation – Image filtering Modified DCT based filter that takes into account spatial correlation of noise by setting frequency dependent thresholds allows increasing output PSNR by 2..4 dB in comparison to conventional DCT filter. Output of conventional DCT filter Output of modified (adaptive) DCT filter Noisy image fragment Nikolay Ponomarenko 19/1/2010

Real life RS image processing applications Formed image Blind estimation of noise spatial spectrum and variance Lossless compression (LOSSLESS) Lossy compression with introducing negligible (compared to noise) distortions Noise pre-filtering CR is about 1.5…3 Analysis and interpreting CR equals to 3…8 Near-lossy compression without visually noticeable distortions CR equals to 4…10 Nikolay Ponomarenko 19/1/2010

Analysis and compression of AVIRIS hyperspectral images Hyperspectral airborne remote sensing system. Flight altitude - 20 km, carrier speed – 730 km/h. 224 sub-bands with wavelengths from 400 to 2500 nm. Images are corrupted by spatially correlated quasi-Gaussian noise with complex dependence of its statistics on local mean. Spatial resolution – 17 m. AVIRIS DCT spectrum of noise Nikolay Ponomarenko 19/1/2010

Analysis and compression of images formed by TerraSAR-X (Germany) Satellite was launched in June, 2007. Altitude 512…530 km. Synthetic aperture radar (SAR). Images are corrupted by spatially correlated noise with Rayleigh PDF: Resolution 1 m (area 10 km x 5 km) Resolution 3 m (area 30 km х 50 km) Resolution 18 m (100 km х 150 km) TerraSAR-X DCT spatial spectrum Nikolay Ponomarenko 19/1/2010

Conclusions and practical problems The task of blind estimation of noise spatial spectrum in DCT domain is considered. It is shown that noise in many real life images is spatially correlated. The method for blind estimation able to perform even for highly textutral images is proposed and tested It is demonstrated that the use of the obtained estimate of noise spatial spectrum leads to considerable improvement of noise suppression efficiency (up to 2..4 dB in terms of PSNR). Noise spatial spectrum can be also exploited in DCT based coder adaptation resulting in increasing CR by 1.5…2 times. Practical problems There can be clipped (overexposed) areas that should be detected before applying the designed blind estimation method. If an image is subject to complex and/or unknown homomorphic transforms, noise statistics becomes rather complex and it has to be taken into account Nikolay Ponomarenko 19/1/2010