Adaptive Filtering of Raster Map Images Minjie Chen*, Mantao Xu and Pasi Fränti Speech and Image Processing Unit (SIPU) School of Computing University of Eastern Finland, FINLAND

Topographic or road maps Few colors Detailed spatial structures

Requirement No over-smooth, remain readable Number of colors does not increase Preserve spatial structures Noise Model Impulsive Noise Gaussian Noise Original: 4 colors Scanning Image: colors Quantized Image: 4 colors

Impulsive Noise Vector Median (VM, AVM) Peer Group Filtering (PGF,FPGF ) Morphological Filtering Context Tree Modelling Discrete Universal Denoiser (DUDE) Gaussian Noise Wavelet denoising using Gaussian scale mixtures (GSM) Non-local mean (NLM) Dictionary- based method (K-SVD) Blocking matching and 3D filtering (BM3D) Markov random fields/conditional random fields (FoE,ARF) Non-Local Sparse Models for Image Restoration (NLSM) Patch-based Locally Optimal Wiener Filtering for Image Denoising (PLOW) Sparsity-based Image Denoising via Dictionary Learning and Structural Clustering (CSR) Most algorithms are designed for continuous-tone(photographic) images Complicated spatial structures in the map will be destroyed

Covert multi-dimension image filtering problem into a series of binary image filtering problem using layer seperating After filtering on each layer, select suitable layer ordering to reconstruct the image Step 1: Divide color image into Multi-Layer binary images Step 2: Filter binary layers separately Step 3: Layer ordering decision Step 4: Merge the filtered layers

Layer separating Filter each layer Merging step with global color priority

Global criterion : select colors according to their frequency Lowest priority is used as background color PRIORITY Highest Priority Same color priority for whole image No difference for different regions

Segment the image into several regions with different background color, set different ordering criterions for these regions

Merging with local color priority

Process each color layer by soft morphological dilation and filling holes operations. Large blocks regions are possible background regions.

Region labeling labels connected pixels After region labeling, sort segments according to their size If the size of the segments is larger than a threshold, select as background region candidate The goal is to extract large connected segments in each layer for later background filling step

SegmentSize Six large segments are detected from different layer 2,5 from black layer, 3 from blue, 1,4 from white, 6 from brown

Evaluate if those large segments detected in Step 3.2 are real background regions. First create a blank background image Evaluate those segments one by one from large segment to small segment. If it is real background, add to background image, labeling the region with the layer color

Two features used for validating if it is background segment Feature 1: How many change after dilation and image filling operation Background segment’s size does not change much, have large ratio Region 1: Before After Ratio (ADDED) Region 2: Before After Ratio (NOT ADDED)

Feature 2: Does this segment overlap with background already filled? In common condition, it should have a small value, this feature can enhance the filling accuracy If feature 1 is close to 1, this feature can also be large, it will add a background inside one background segment Region 2: Overlap percentage 97% (NOT ADDED) Region 1: Overlap percentage 0% (ADDED) Already labeled region 1 Blank Background

Fill block 1 S1/S2PoADD(Y/N) Y N Y Y N N Fill block 3 Fill block 4 Blank background image We can set different threshold in filling step, causing different background images. Classification can be done to decide threshold for different type of images.

Large unfilled regions ---- set as small background region. Small unfilled region ---- merge it to the nearest background. Set as small background region Merge to the nearest background After processing

For all background region, calculate its corresponding color histogram We then get different color priority for different region

The current method does not work if color number is large Priority between small structures not considered Special local patterns combined with multiple color is missed

Colors with low conditional probability will be replaced by the dominated color Frequency XX Frequency Context Template

Context Dilution 6-color, 20-pixel template has 6 20 = 3,656,158,440,062,976 contexts Most contexts has rare appearance, cause inaccurate conditional probability estimation. Context Tree Modeling Only appeared contexts are allocated in memory Time complexity O(N), where N is the length of a data sequence. Tree spanning is terminated once the frequency of the context on a given node is less than a predefined value(N) ThresholdContexts N= N= N= N=

Including noise pixels in the surrounding contexts makes good conditional probability estimation difficult. For contexts with rare appearance, a merging process is done to collect the statistics of all similar contexts. Time complexity is O(kNM 2 ), where M is the number of colors, k is the depth of the context tree, N is the number of contexts with rare appearance. Example of context merging Noisy pixel Most consistent sub-context

Discrete Universal Denoiser(DUDE) M is the number of colors, δ is the noise level, u 0 is the color with highest conditional probability. DUDE has a so-called “asymptotic optimality” property for M-ary symmetric noise. Estimation of δ Estimated by the minimum conditional probability occurred for contexts with “sufficient frequency”: This decision rule is designed for the count statistics collected on the noisy image. For a clean image, the decision rule is

Iterative algorithm to optimize both the estimation of the indexed image and its color palette. The distance between RGB color vector to its corresponding component in the color palette, and its conditional probability of local context are taken into account as an information fusion. Definitions: X: index image, Y: corresponding image, CP=(m 1,m 2,…m M ) color palette where m i = (m i (r), m i (g), m i (b)), y x = (r x,g x,b x ) the color intensity of x in RGB space. Input: Y X, CP ← Conduct color quantization based on Y Σ ← Estimate the quantization variance based on X, Y,CP according to (3) For T iterations DO: Given X, update P(x|c) Update X, according to (1) Update CP and σ according to (2) and (3) End-For Output: Y, X, CP. General Scheme for filtering Gaussian noise (1) (2) (3)

Noisy imageQuantized image Filtering after 1 iteration Filtering after 5 iterations

δ = 0.05 Error rate (%) AVMPGFCTDUDEACS Set# Set# Set# Set# Set# σ =25PSNR GSMNLMBM3DARFACS Set# Set# Set# Set# Set#

Statistical filtering for raster map images Can process images distorted by impulsive noise, additive Gaussian noise and mixture Gaussian- impulsive noise Both color distribution in RGB space and the conditional probabilities of local context are considered

Pruning?Merging? Reorder the context pixel-wise? Our solution: An voting-based method to optimize the context selection

identify good or bad context by top-down tracing from the root of tree.

OriginalNoisy OriginalNoisy Noisy pixel with bad context Noisy pixel with bad context can not filter correctly. Can we use bad context for noise estimation?

Bad contexts will have higher score difference after the noisy pixel is removed. We can accumulated these score difference for bad context to find noisy pixel, which is the voting image R: Include noisy pixel may cause bad context.

F(z)- F(c) Voting image

F(z)- F(c) Voting image

F(z)- F(c) Voting image

F(z)- F(c) Voting image

F(z)- F(c) Voting image

F(z)- F(c) Voting image Noisy pixel have a high voting value

Bad context include black noisy pixel southeast reduced context F(z)- F(c) Voting value 7.73 Bad context include white noisy pixel northwest south west south F(z)- F(c) reduced context Voting value

voting image NoisyOriginal

voting imagenoisy image

If contexts are not good context, pixels with low voting value are selected to construct new adaptive context.

OriginalNoisy Iteration 1Iteration 2 DUDEProposed Noisy pixels with contaminated contexts are filtered correctly by optimal context selection

Adaptive context selection via a voting-based noise estimation scheme Can process raster map images distorted by impulsive noise, additive Gaussian noise or mixture Gaussian-impulsive noise Extension for optimizing the context selection for denoising gray-scale image, e.g. voting-based method to optimize the weighting coefficient in NLM, PLOW, K-SVD.

M. Chen, M. Xu and P. Fränti, "Multi-layer filtering approach for map images", IEEE Int. Conf. on Image Processing (ICIP'09), Cairo, Egypt, , M. Chen, M. Xu and P. Fränti, "Statistical filtering of raster map images", IEEE Int. Conf. on Multimedia & Expo (ICME'10), Singapore, , (oral) M. Chen, M. Xu, P. Fränti, "Adaptive Context-tree based Statistical Filtering of Raster Map Images Denoising", IEEE Trans. on Multimedia, 16(3), , M. Chen, M. Xu, P. Fränti, "Adaptive Filtering of Raster Map Images Using Optimal Context Selection", IEEE Int. Conf. on Image Processing (ICIP’11), 77-80, Brussels, Belgium, 2011.(oral)