1. Adaptive Filtering of Raster Map Images Minjie Chen*, Mantao Xu and Pasi Fränti Speech and Image Processing Unit (SIPU) School of Computing University of Eastern Finland, FINLAND

2. Topographic or road maps Few colors Detailed spatial structures

3. Requirement No over-smooth, remain readable Number of colors does not increase Preserve spatial structures Noise Model Impulsive Noise Gaussian Noise Original: 4 colors Scanning Image: 11063 colors Quantized Image: 4 colors

4. Impulsive Noise Vector Median (VM, AVM) Peer Group Filtering (PGF,FPGF ) Morphological Filtering Context Tree Modelling Discrete Universal Denoiser (DUDE) Gaussian Noise Wavelet denoising using Gaussian scale mixtures (GSM) Non-local mean (NLM) Dictionary- based method (K-SVD) Blocking matching and 3D filtering (BM3D) Markov random fields/conditional random fields (FoE,ARF) Non-Local Sparse Models for Image Restoration (NLSM) Patch-based Locally Optimal Wiener Filtering for Image Denoising (PLOW) Sparsity-based Image Denoising via Dictionary Learning and Structural Clustering (CSR) Most algorithms are designed for continuous-tone(photographic) images Complicated spatial structures in the map will be destroyed

5. Covert multi-dimension image filtering problem into a series of binary image filtering problem using layer seperating After filtering on each layer, select suitable layer ordering to reconstruct the image Step 1: Divide color image into Multi-Layer binary images Step 2: Filter binary layers separately Step 3: Layer ordering decision Step 4: Merge the filtered layers

6. Layer separating Filter each layer Merging step with global color priority

7. Global criterion : select colors according to their frequency Lowest priority is used as background color PRIORITY Highest Priority 618416 230020 173358 22975 3458 349 Same color priority for whole image No difference for different regions

8. Segment the image into several regions with different background color, set different ordering criterions for these regions

9. Merging with local color priority

10. Process each color layer by soft morphological dilation and filling holes operations. Large blocks regions are possible background regions.

12. Region labeling labels connected pixels After region labeling, sort segments according to their size If the size of the segments is larger than a threshold, select as background region candidate The goal is to extract large connected segments in each layer for later background filling step

13. SegmentSize 1765241 2662515 3187677 483904 551170 621044 Six large segments are detected from different layer 2,5 from black layer, 3 from blue, 1,4 from white, 6 from brown

14. Evaluate if those large segments detected in Step 3.2 are real background regions. First create a blank background image Evaluate those segments one by one from large segment to small segment. If it is real background, add to background image, labeling the region with the layer color

15. Two features used for validating if it is background segment Feature 1: How many change after dilation and image filling operation Background segment’s size does not change much, have large ratio Region 1: Before 526337 After 765241 Ratio 0.6878(ADDED) Region 2: Before 193606 After 662515 Ratio 0.2922(NOT ADDED)

16. Feature 2: Does this segment overlap with background already filled? In common condition, it should have a small value, this feature can enhance the filling accuracy If feature 1 is close to 1, this feature can also be large, it will add a background inside one background segment Region 2: Overlap percentage 97% (NOT ADDED) Region 1: Overlap percentage 0% (ADDED) Already labeled region 1 Blank Background

17. Fill block 1 S1/S2PoADD(Y/N) 10.68780Y 20.29220.9712N 30.83730.0172Y 40.80240.0149Y 50.23020.9953N 60.15671N Fill block 3 Fill block 4 Blank background image We can set different threshold in filling step, causing different background images. Classification can be done to decide threshold for different type of images.

18. Large unfilled regions ---- set as small background region. Small unfilled region ---- merge it to the nearest background. Set as small background region Merge to the nearest background After processing

19. For all background region, calculate its corresponding color histogram We then get different color priority for different region

22. The current method does not work if color number is large Priority between small structures not considered Special local patterns combined with multiple color is missed

23. Colors with low conditional probability will be replaced by the dominated color Frequency XX 409 3 4 5 9 Frequency 2 193 3 6 Context Template

24. Context Dilution 6-color, 20-pixel template has 6 20 = 3,656,158,440,062,976 contexts Most contexts has rare appearance, cause inaccurate conditional probability estimation. Context Tree Modeling Only appeared contexts are allocated in memory Time complexity O(N), where N is the length of a data sequence. Tree spanning is terminated once the frequency of the context on a given node is less than a predefined value(N) ThresholdContexts N=256 23676 N=12835847 N=6454736 N=32 82260

25. Including noise pixels in the surrounding contexts makes good conditional probability estimation difficult. For contexts with rare appearance, a merging process is done to collect the statistics of all similar contexts. Time complexity is O(kNM 2 ), where M is the number of colors, k is the depth of the context tree, N is the number of contexts with rare appearance. Example of context merging Noisy pixel Most consistent sub-context

26. Discrete Universal Denoiser(DUDE) M is the number of colors, δ is the noise level, u 0 is the color with highest conditional probability. DUDE has a so-called “asymptotic optimality” property for M-ary symmetric noise. Estimation of δ Estimated by the minimum conditional probability occurred for contexts with “sufficient frequency”: This decision rule is designed for the count statistics collected on the noisy image. For a clean image, the decision rule is

27. Iterative algorithm to optimize both the estimation of the indexed image and its color palette. The distance between RGB color vector to its corresponding component in the color palette, and its conditional probability of local context are taken into account as an information fusion. Definitions: X: index image, Y: corresponding image, CP=(m 1,m 2,…m M ) color palette where m i = (m i (r), m i (g), m i (b)), y x = (r x,g x,b x ) the color intensity of x in RGB space. Input: Y X, CP ← Conduct color quantization based on Y Σ ← Estimate the quantization variance based on X, Y,CP according to (3) For T iterations DO: Given X, update P(x|c) Update X, according to (1) Update CP and σ according to (2) and (3) End-For Output: Y, X, CP. General Scheme for filtering Gaussian noise (1) (2) (3)

28. Noisy imageQuantized image Filtering after 1 iteration Filtering after 5 iterations

29. δ = 0.05 Error rate (%) AVMPGFCTDUDEACS Set#18.651.401.040.940.65 Set#24.191.260.99 0.78 Set#311.83.262.412.091.59 Set#46.551.211.421.340.82 Set#59.276.024.362.662.02 σ =25PSNR GSMNLMBM3DARFACS Set#124.424.926.224.247.1 Set#224.023.526.523.359.2 Set#323.722.423.723.027.0 Set#424.624.125.523.833.4 Set#525.124.326.125.127.7

31. Statistical filtering for raster map images Can process images distorted by impulsive noise, additive Gaussian noise and mixture Gaussian- impulsive noise Both color distribution in RGB space and the conditional probabilities of local context are considered

32. Pruning?Merging? Reorder the context pixel-wise? Our solution: An voting-based method to optimize the context selection

33. identify good or bad context by top-down tracing from the root of tree.

34. OriginalNoisy OriginalNoisy Noisy pixel with bad context Noisy pixel with bad context can not filter correctly. Can we use bad context for noise estimation?

35. Bad contexts will have higher score difference after the noisy pixel is removed. We can accumulated these score difference for bad context to find noisy pixel, which is the voting image R: Include noisy pixel may cause bad context.

36. F(z)- F(c)-0.392.295.41-0.38 00000 00000 00000 00000 00000 Voting image 00000 0002.290 005.4100 00000 00000 00000 0002.290 005.4100 00000 00000 +

37. F(z)- F(c)1.39-1.24-1.333.58 00000 0002.290 005.4100 00000 00000 Voting image 0001.390 003.582.290 005.4100 00000 00000 0001.390 003.5800 00000 00000 00000 +

38. F(z)- F(c)-0.625.691.96-0.40 0001.390 003.582.290 005.4100 00000 00000 Voting image 0001.390 003.582.290 0011.000 01.96000 00000 00000 00000 005.6900 01.96000 00000 +

39. F(z)- F(c)4.15-2.86-2.322.24 0001.390 003.582.290 0011.000 01.96000 00000 Voting image 0001.390 007.732.290 02.2411.000 01.96000 00000 00000 004.1500 02.24000 00000 00000 + -0.65

40. F(z)- F(c)-2.023.23-1.40-1.63 0001.390 007.732.290 02.2411.000 01.96000 00000 Voting image 0001.390 007.732.290 02.2416.500 01.963.2300 00000 00000 00000 005.5100 003.2300 00000 + 5.51

41. F(z)- F(c)1.791.38-1.43-1.16 0001.390 007.732.290 02.2416.500 01.963.2300 00000 Voting image 0001.390 007.732.290 02.2418.32.350 01.963.231.380 00000 00000 00000 001.792.350 0001.380 00000 + 2.35 Noisy pixel have a high voting value

42. Bad context include black noisy pixel southeast reduced context F(z)- F(c)4.15 5.69 Voting value 7.73 Bad context include white noisy pixel northwest south west south F(z)- F(c) reduced context 5.415.511.79 Voting value 18.30 3.58

43. voting image NoisyOriginal

44. voting imagenoisy image

45. If contexts are not good context, pixels with low voting value are selected to construct new adaptive context.

46. OriginalNoisy Iteration 1Iteration 2 DUDEProposed Noisy pixels with contaminated contexts are filtered correctly by optimal context selection

47. Adaptive context selection via a voting-based noise estimation scheme Can process raster map images distorted by impulsive noise, additive Gaussian noise or mixture Gaussian-impulsive noise Extension for optimizing the context selection for denoising gray-scale image, e.g. voting-based method to optimize the weighting coefficient in NLM, PLOW, K-SVD.

48. M. Chen, M. Xu and P. Fränti, "Multi-layer filtering approach for map images", IEEE Int. Conf. on Image Processing (ICIP'09), Cairo, Egypt, 3953-3956, 2009. M. Chen, M. Xu and P. Fränti, "Statistical filtering of raster map images", IEEE Int. Conf. on Multimedia & Expo (ICME'10), Singapore, 394-399, 2010. (oral) M. Chen, M. Xu, P. Fränti, "Adaptive Context-tree based Statistical Filtering of Raster Map Images Denoising", IEEE Trans. on Multimedia, 16(3), 1195-1207, 2011. M. Chen, M. Xu, P. Fränti, "Adaptive Filtering of Raster Map Images Using Optimal Context Selection", IEEE Int. Conf. on Image Processing (ICIP’11), 77-80, Brussels, Belgium, 2011.(oral)