1. Aleksey S. Rubel1, Vladimir V. Lukin1,
Ninth International Workshop on Video Processing and Quality Metrics for Consumer Electronics - VPQM 2015  1
Block Matching and 3D Collaborative Filtering Adapted to Additive Spatially Correlated Noise 
 Aleksey S. Rubel1, Vladimir V. Lukin1,
Karen O. Egiazarian2
1Department of Transmitters, Receivers and Signal Processing, National Aerospace University, Kharkov, Ukraine
2Department of Signal Processing, Tampere University of Technology, Finland

2. Motivations 2
Additive white Gaussian noise is a commonly used model in image processing. However, noise in images acquired by recent devices (sensors) is not white due to demosaicing procedure and has it low-frequency spectrum.
Additive spatially correlated noise appears itself as a more distinctive (annoying) distortion than AWGN.
Human visual system (HVS) is more sensitive to distortions of low frequency data than to distortions in high-frequency part of data spectrum.
Spatial correlation of noise pixels is usually essential only for neighboring pixels.
Recently, a wide variety of denoising techniques oriented only for AWGN removal has been proposed. Several of them are based on discrete cosine transform (DCT) due to its good energy compactness and sparseness properties. BM3D filter has very high efficiency of AWGN removal.
Modifications of image filtering are needed in considered cases and they should be more oriented on removing the distortions in low-frequency part of image spectrum.

3. Additive spatially correlated noise modelling3
The commonly used conventional AWGN model is:
where It is the true image, In is the image corrupted by additive noise, i and j are indices of pixels, NGaussian denotes zero mean additive white Gaussian noise that is modelled by 2D realization of Gaussian random process with standard deviation of the noise σ.
Two cases of additive spatially correlated noise are considered: with middle (AMSCN) and strong (ASSCN) spatially correlation. To obtain additive spatially correlated noise, 2D zero mean AWGN realization is needed. Then, 2D FIR filters with certain matrices of weights are applied.
Additive middle spatially correlated noise – AMSCN
Additive strong spatially correlated noise – ASSCN
1/16
1/8
1/4
1/9
After this, scaling of the obtained 2D data is employed to provide a desired variance. Spatially correlated noise is then added to an original image and produces a corrupted image with distinctive distortions like graininess.

4. Additive spatially correlated noise (SD=15) appearance4
AWGN
AMSCN
ASSCN #5
#15
#23

5. BM3D filter 11
BM3D uses 2D DCT spectrum domain for block matching and grouping.
Classical Euclidean distance is used for additive Gaussian noise suppression by BM3D filter.
Search robustness becomes a substantial issue for this technique.
Collaborative filtering is performed after 3D transform (2D DCT and 1D Haar, as standard combination). Denoising mechanism is similar to the DCT-based filter.
Without grouping (if there are no similar blocks in search region), BM3D turns into 2D conventional DCT filter.

6. Full BM3D adaptation to spatially correlated noise6
Possible ways and ideas
It has been shown earlier (for spatially correlated noise) that standard Bray-Curtis and Canberra distances applied to BM3D are able provide a higher efficiency of block matching than the standard Euclidean distance used in conversional version of BM3D.
(see A.S. Rubel, V.V. Lukin, K.O. Egiazarian, “Metric performance in similar blocks search and their use in collaborative 3D filtering of grayscale images”, Proceedings of SPIE Image Processing: Algorithms and Systems. Vol. 9019, USA, 2014, 12 p.)
The procedure (block matching) is essential because spatially correlated noise affects low-frequency DCT components that distort a reference block and other blocks related to it.
As a result, a similarity search might find blocks similar to each other not in the content but in realizations of spatially correlated noise.
Then, it is worth using some weighting operation in block similarity estimation to take into account 2D spectrum of the noise (it is assumed that this spectrum is known).
The second basic procedure (collaborative denoising) works with the groups of similar blocks. Thus, setting fixed (frequency independent) threshold can lead to an inefficient denoising.
Then, it is worth using some frequency-dependent thresholds in denoising procedure to take into account normalized 2D spectrum of the noise (it is assumed that this spectrum is priory known).

7. Block matching adaptation for spatially correlated noise7
Presenting image blocks transformed by DCT as vectors, the weighted Euclidian, Bray-Curtis and Canberra distances in DCT domain can be expressed as:
where x and y are blocks (vectors) of data in DCT domain, i denotes an index of vector element, n defines sample size (equals to 64), w denotes a weighting function that is calculated as
where Wi is a value of i-th component of the DCT spectrum of noise which is supposed to be a priori known or pre-estimated with an appropriate accuracy. 7

8. Collaborative denoising adaptation for spatially correlated noise8
The modified procedure with the frequency-dependent thresholds is the following:
where
σ is a standard deviation (SD) of the noise,
σi denotes a frequency-dependent SD for a spatially correlated noise,
Bin is an input 3D transformed group of blocks (applying the separable 2D DCT in spatial coordinates and the 1D Haar transform in the similarity direction) and expressed as a 2D array (because of vector presentation of each block),
Bout is the output filtered data,
β is an adjusting parameter set to 2.7,
Wn denotes the i-th value of the normalized spectrum,
no other modifications or changes have been done to the standard BM3D algorithm. 8

9. TID2013 test images 9

10. Evaluation of denoising performance10
Improvement of traditional PSNR, as performance criterion: or
Improvement of PSNR-HVS-M, as performance criterion:
V. Lukin, N. Ponomarenko, K. Egiazarian, “HVS-Metric-Based Performance Analysis Of Image Denoising Algorithms”, Proceedings of EUVIP, Paris, France, 2011, pp
We have considered seven versions of the BM3D filter that can be applied for removal of non-white noise. The first one is the standard version of the filter. Six others use frequency-dependent thresholds (FDT) and the following similarity metrics:
standard and weighted Euclidian distance (BM3D-FDT&E and BM3D-FDT&wE);
standard and weighted Bray-Curtis distance (BM3D-FDT&BC and BM3D-FDT&wBC);
standard and weighted Canberra distance (BM3D-FDT&C and BM3D-FDT&wC).

11. Denoising performance of TID2013 images by IPSNR for AMSCN11
σ2 = 5
σ2 = 15
Denoising efficiency depends upon image content and is the smallest for complex structure images such as the test images ## 1, 5, 13, 14, 18 irrespectively of the used modification of the BM3D filter.

12. Denoising performance of TID2013 images by IPSNR-HVS-M for AMSCN12
σ2 = 5
σ2 = 15
The standard BM3D designed for AWGN removal practically always produces the worst results.

13. Denoising performance of TID2013 images by IPSNR for ASSCN
σ2 = 5
σ2 = 15
The use of both versions of Canberra distance and the weighted Bray-Curtis distance results in the best performance; the use of other distances produce intermediate results.

14. Denoising performance of TID2013 images by IPSNR-HVS-M for ASSCN14
σ2 = 5
σ2 = 15
The benefit due to exploiting the best distances and frequency-dependent thresholds can reach 4 dB (compared to the standard BM3D filter) for simple structure images and intensive noise.

15. Examples of denoising 15 #5 #23
Zoomed 60*60 pixels fragments of test images ##5 and 23 from TID2013 are shown below: original fragments, noisy fragments distorted by ASSCN with noise SD=15, filtered fragments by different versions including BM3D).
Noise-free image
Noisy image
Standard BM3D
BM3D-FDT&C
BM3D-FDT&wBC #5
#23

16. Conclusions 16
BM3D adaptation for additive spatially correlated noise can be easily introduced into the stage of similar block search and DCT-based filtering.
Such adaptation takes into account 2D spectrum of spatially correlated noise assumed to be a priori known or pre-estimated.
Special kind of weighting is exploited in distance calculation and frequency dependent threshold setting. Both types of modifications contribute to improvement of the filter performance.
The largest improvement takes place for simple structure images whilst for complex-structure images improvement can be negligible, especially for low level of the noise.
The use of Canberra metric (even if weighting is not used) is expedient since the versions of BM3D filter with these distances produce the best or nearly the best results for both considered types of spatially correlated noise.
The benefit slightly depends upon noise properties (degree of spatial correlation).

17. Block Matching and 3D Collaborative Filtering Adapted to Additive Spatially Correlated Noise17
Thank you!
Karen O. Egiazarian
Vladimir V. Lukin
Aleksey S. Rubel