1. A method for blind estimation of spatially correlated noise characteristics
Nikolay N. Ponomarenkoa, Vladimir V. Lukina,
Karen O. Egiazarianb, Jaakko T. Astolab
a National Aerospace University, 61070, Kharkov, Ukraine;
b Tampere University of Technology, FIN 33101, Tampere, Finland
Nikolay Ponomarenko
19/1/2010

2. Images formed by Earth observation remote sensing (RS) systems
Reasons of noise presence in formed images
Propagation medium
Apparatus imperfection
Properties of registered data
Distortions due to propagation medium
Example: optical image registration
Poisson distribution
Light photons
Nikolay Ponomarenko
19/1/2010

3. Design of a new method for digital image processing
Processing method
Processed image
Image mathematical model
Quality metric that characterizes designed method efficiency
Distorted image
Noise mathematical model
Noise type and PDF:
Gaussian
Poisson
Rayleigh
Others
Type of noise dependence on signal:
Additive
Multiplicative
More complex dependence
Spatial
correlation:
White (i.i.d.) noise
Colored noise
Nikolay Ponomarenko
19/1/2010

4. Spatially correlated (colored) noise in images (example)
Process of image forming (processing) in digital cameras
Image registration by camera matrix
Mosaic
image
Interpolation
Gamma-correction
JPEG R G B
White Poisson (or mixed Poisson + additive) noise
Noise becomes not-Poisson and spatially correlated
Noise becomes signal dependent with complex dependence
Methods designed for white noise removal ARE NOT EFFICIENT for suppressing spatially correlated noise
A filter should be adapted to properties of spatially correlated noise
One has to know noise spect-rum
It is necessary to estimate noise spectrum for an image to be processed
Nikolay Ponomarenko
19/1/2010

5. Main problem – absence of absolutely homogeneous regions in real life images
If there are quite many homogeneous regions in an image where only noise is present it is easy to estimate spatial spectrum of noise
Image
Detection and localization of image homogeneous regions
Rejection of fragments with (potentially) abnormal distributions
Averaged spectrum estimation
Textural region
(example)
Quasi-homogeneous
region
Real life image fragment
(example)
Nikolay Ponomarenko
19/1/2010

6. Proposed approach to blind estimation of noise spatial spectrum
The proposed approach combines four mutually complementing tools for blind estimation of noise spatial spectrum :
DISCRETE COSINE TRANSFORM (DCT): being applied to blocks of natural images, it is able to decorrelate pixels values well (to separate data into information and noise components).
IMAGE FRAGMENT SELF-SIMILARITY: information component for some image blocks can be similar whilst noise component is commonly not.
ROBUST ESTIMATION: is used in order to diminish negative influence of image information component on estimation of noise variance and spatial spectrum.
Method output –8x8 matrix of noise variance for spectral coefficients of DCT.
Note that spatial spectrum estimation in DCT domain can be useful for DCT-based filtering and compression of noisy images.
Image
Image pixel decorrelation in blocks by means of DCT
Pair-wise search of the most similar blocks
Obtaining of robust estimates of variance and spectrum
Nikolay Ponomarenko
19/1/2010

7. Details of the proposed method
1. The base block size is selected; below we use 8x8 pixel blocks since this size is convenient for accelerating search of similar blocks without losing the method efficiency and accuracy.
2. A random mask δ that has the values either 0 or 1 is generated; this mask separates DCT coefficients into two halves; the value 1 refers DCT coefficient to the first half and vice versa.
Example of the generated δ 1
Nikolay Ponomarenko
19/1/2010

8. Details of the proposed method
3. For each image block A (position of the scanning window of size 8x8 pixels), its similarity with respect to all blocks B in the neighborhood of A is analyzed where distance R(A, B) is larger than r1 and smaller than r2. The distance R(A, B) is defined as max(AX-BX, AY-BY) where AX, AY and BX, BY are coordinates (indices) X and Y of left upper corners of blocks A and B. The recommended values r1 and r2 are r1=3 and r2=10. Similarity of the blocks A and B is calculated in the DCT domain according to (1) with taking into account the mask δ:
MSED(A, B) = ,
where AD and BD are DCT matrices (coefficient sets) for blocks A and B.
Nikolay Ponomarenko
19/1/2010

9. Details of the proposed method
4. The array D is used for storing K results of DCT(A-B) that correspond to minimal found values MSED(A, B). The recommended value K for images of size 512x512 pixels is K=1025.
5. As the result of carrying out the steps 1…4 for each pair of indices of DCT coefficients one obtains K values of these coefficients. The coefficients for which δ is equal to 1 are the results of searching the minimal values. Thus, it is impossible to use them for estimation of noise parameters. Other coefficients for which δ is equal to 0 in ideal case (for absolutely similar blocks) relate to noise component. In practice, considerable part of these coefficients can correspond to information content. This can lead to heavier tails of coefficient distribution. Thus, it is desirable to apply robust approaches to estimating σij2 for each coefficient with δij=0. One way to do this is to apply the estimate
where is the correcting factor.
Nikolay Ponomarenko
19/1/2010

10. Details of the proposed method
It is also possible to evaluate tail heaviness for the considered distributions that characterizes confidence of the obtained estimates. The estimate can be considered confident if
The recommended value of Tr is Tr=6.
6. After performing the steps 1…5, one has the estimates for one half of DCT coefficients of the block. Then, the mask δ is inverted (δnew = 1 - δ) and the steps 1…5 are repeated one more time; as the result, the estimates are obtained for all other coefficients of the block.
7. The values of estimates that are not confident can be replaced by the minimal confident estimate of the nearest coefficients of the matrix. If the total number of non-confident estimates is too large, e.g., 50%, then the decision that the considered method is not worth applying for estimation of noise parameters for a given image can be undertaken.
Nikolay Ponomarenko
19/1/2010

11. Quality criterion of an accuracy of noise spectrum estimation
Estimation of parameters for spatially correlated additive Gaussian noise
Quality criterion of an accuracy of noise spectrum estimation
Recall that the main intention of such estimation is to accurately evaluate the matrix Integrally, accuracy of estimation can be characterized by the parameter
where L=MN-1 (M and N define the block size and are both equal to 8 for the considered case), is the estimated standard deviation of noise for DCT coefficient with indices i and j, σij is its true value that has been determined by simulations for the image Homog (large homogeneous region) with multiple realizations of spatially correlated noise.
Nikolay Ponomarenko
19/1/2010

12. Estimation of parameters for spatially correlated additive Gaussian noise
Compared methods
Method
Reference
PGE
Foi, A. Practical denoising of clipped or overexposed noisy images, Proceedings of the 16th European Signal Process. Conf., EUSIPCO 2008, 2008, pp. 1-5
SE0
Anisotropic Nonparametric Image Restoration DemoBox
(available at
AIQRF
V.V. Lukin, S.K. Abramov, A.A. Zelensky, J. Astola, B. Vozel, B. Chehdi, Improved minimal inter-quantile distance method for blind estimation of noise variance in images, Proceedings of SPIE Conference Image and Signal Proc. for Remote Sensing, 2007, SPIE Vol. 6748, pp. 1-12
RDCT
N.N. Ponomarenko, V.V. Lukin, S.K. Abramov, K.O. Egiazarian, J.T. Astola , Blind evaluation of additive noise variance in textured images by nonlinear processing of block DCT coefficients, Proc. of Int. Conf. Image Proc.: Algor. and Systems II, Santa Clara, USA, Jan. 2003, SPIE Vol pp
BECNS
The proposed method
Nikolay Ponomarenko
19/1/2010

13. MSE ξ of estimation of spatially correlated noise DCT spectrum
Estimation of parameters for spatially correlated additive Gaussian noise
MSE ξ of estimation of spatially correlated noise DCT spectrum
Nikolay Ponomarenko
19/1/2010

14. Estimation of parameters for spatially correlated additive Gaussian noise
True and estimated parameters of spatially correlated noise for σ2=100 for the highly textural image (Baboon)
Nikolay Ponomarenko
19/1/2010

15. Filtering results (PSNR, ΔPSNR, dB) for spatially correlated noise
Filtering efficiency based on estimated parameters of spatially correlated additive Gaussian noise (in terms of PSNR)
Filtering results (PSNR, ΔPSNR, dB) for spatially correlated noise
Nikolay Ponomarenko
19/1/2010

16. Filtering efficiency based on estimated parameters of spatially correlated additive Gaussian noise (visual quality)
Filtering results (PSNR-HVS-M, ΔPSNR-HVS-M, dB) for spatially correlated noise
Nikolay Ponomarenko
19/1/2010

17. Output of conventional Output of modified (adaptive)
Efficiency of two-stage automatic procedure Noise parameters estimation – Image filtering
Modified DCT based filter that takes into account spatial correlation of noise by setting frequency dependent thresholds allows increasing output PSNR by 2..4 dB in comparison to conventional DCT filter.
Output of conventional
DCT filter
Output of modified (adaptive)
DCT filter
Noisy image fragment
Nikolay Ponomarenko
19/1/2010

18. Real life RS image processing applications
Formed image
Blind estimation of noise spatial spectrum and variance
Lossless compression
(LOSSLESS)
Lossy compression with introducing negligible (compared to noise) distortions
Noise pre-filtering
CR is about
1.5…3
Analysis and
interpreting
CR equals
to 3…8
Near-lossy compression without visually noticeable distortions
CR equals to
4…10
Nikolay Ponomarenko
19/1/2010

19. Analysis and compression of AVIRIS hyperspectral images
Hyperspectral airborne remote sensing system.
Flight altitude - 20 km, carrier speed – 730 km/h.
224 sub-bands with wavelengths from 400 to 2500 nm.
Images are corrupted by spatially
correlated quasi-Gaussian noise
with complex dependence of
its statistics on local mean.
Spatial resolution – 17 m.
AVIRIS
DCT spectrum of noise
Nikolay Ponomarenko
19/1/2010

20. Analysis and compression of images formed by TerraSAR-X (Germany)
Satellite was launched in June, Altitude 512…530 km.
Synthetic aperture radar (SAR).
Images are corrupted by spatially
correlated noise with Rayleigh PDF:
Resolution 1 m (area 10 km x 5 km)
Resolution 3 m (area 30 km х 50 km)
Resolution 18 m (100 km х 150 km)
TerraSAR-X
DCT spatial spectrum
Nikolay Ponomarenko
19/1/2010

21. Conclusions and practical problems
The task of blind estimation of noise spatial spectrum in DCT domain is considered.
It is shown that noise in many real life images is spatially correlated.
The method for blind estimation able to perform even for highly textutral images is proposed and tested
It is demonstrated that the use of the obtained estimate of noise spatial spectrum leads to considerable improvement of noise suppression efficiency (up to 2..4 dB in terms of PSNR).
Noise spatial spectrum can be also exploited in DCT based coder adaptation resulting in increasing CR by 1.5…2 times.
Practical problems
There can be clipped (overexposed) areas that should be detected before applying the designed blind estimation method.
If an image is subject to complex and/or unknown homomorphic transforms, noise statistics becomes rather complex and it has to be taken into account
Nikolay Ponomarenko
19/1/2010