Download presentation
Presentation is loading. Please wait.
Published byTorbjΓΆrn BergstrΓΆm Modified over 5 years ago
1
Lecture 7 Patch based methods: nonlocal means, BM3D, K- SVD, data-driven (tight) frame
2
Outline Nonlocal methods Dictionary learning methods Low rank models
Nonlocal means BM3D Dictionary learning methods K-SVD Data-driven tight frame Low rank models
3
Nonlocal Methods Local means and BM3D
4
Denoising: Gaussian Smoothing Revisited
* = *h h π₯ π = 1 πΆ π π π¦(π) π β πβπ π 2
5
Perona-Malik: Anisotropic Filtering
Edges β smooth only along edges βSmoothβ regions β smooth isotropically gradient
6
Ideal: Spatially Adaptive Smoothing
Non uniform smoothing Depending on image content: Smooth where possible Preserve fine details How? *h3 *h2 *h1
7
Bilateral Filtering (Smith and Brady, 1997)
Formula
8
* * * Gaussian Smoothing Same Gaussian kernel everywhere
Slides taken from Sylvain Paris, Siggraph 2007 Gaussian Smoothing * input output * * Same Gaussian kernel everywhere Averages across edges β blur
9
* * * Bilateral Filtering Kernel shape depends on image content
Slides taken from Sylvain Paris, Siggraph 2007 Bilateral Filtering * input output * * Kernel shape depends on image content Avoids averaging across edges
10
Nonlocal Means (NLM): Motivation
Assume a static scene Consider multiple images y(π‘) at different times The signal π₯(π‘) remains constant π(π‘) varies over time with zero mean
11
Nonlocal Means (NLM): Motivation
Average multiple images over time
12
Nonlocal Means (NLM): Motivation
Average multiple images over time
13
Nonlocal Means (NLM): Motivation
Average multiple images over time
14
Redundancy in natural images
Glasner et al. (2009)
15
Single image βtime-likeβ denoising
Unfortunately, patches are not exactly the same β simple averaging just wonβt work
16
NLM Buades, Cole and Morel (2005)
π€(π,π) π€(π,π) π€(π,π ) Use a weighted average based on similarity π₯ π = 1 πΆ π π π¦(π) π β πΊπππ· y π π βy π π 2 π 2 π€ π,π
17
From Bilateral Filters to NLM
π₯ π ππΏπ 1π₯1 = 1 πΆ π π π¦(π) π β π¦ π βπ¦(π) π 2 π₯ π ππΏπ = 1 πΆ π π π¦(π) π β πΊπππ· y π π βy π π 2 π 2 Patch similarity
18
Performance Evaluation
Gaussian Smoothing Anisotropic Filtering Bilateral Filtering NLM Windowed Weiner Hard WT Soft WT Buades et al. (2005)
19
Variational Formulation of NLM
Gilboa and Osher (2008) Nonlocal (partial) derivative Nonlocal gradientοΌ π» π€ π’:πΊβπΊΓπΊ Inner product: π£ 1 , π£ 2 β Ξ©ΓΞ© π£ 1 π₯,π¦ π£ 2 π₯,π¦ ππ₯ππ¦ With such inner product, we can define divergence ππ π£ π€ π£ π₯ : Ξ©ΓΞ©βΞ© And Laplacian
20
Variational Formulation of NLM
Gilboa and Osher (2008) Nonlocal TV Nonlocal ROF Nonlocal TV- πΏ 1
21
Variational Formulation of NLM
Gilboa and Osher (2008)
22
Application in Surface Denoising
Dong et al. (2008)
23
Application in Surface Denoising
Dong et al. (2008)
24
Application in 4D CT Reconstruction
Tian et al. (2011)
25
Whatβs Next? The idea of grouping sounds good β reduces mixing
Denoise = βextract the common (the signal)β NLM: common = weighted average Can a sparser representation do better?
26
BM3D: Dabov et al. (2007) Block Matching 3D collaborative filtering
Group patches with similar local structure (BM) Jointly denoise each group (3D) Smart Fusion of multiple estimates
27
BM3D: Dabov et al. (2007) R R Single BM3D Estimate Filter / R
Block matching Inverse 3D transform Filter / thresholding R R R 3D grouping Denoised 3D group 3D transform
28
BM3D: Dabov et al. (2007) For every noisy reference block:
Calculate SSD between noisy blocks If SSD<thr β add to group
29
BM3D: Dabov et al. (2007) 3D transform 3D transform Image Patch Domain
Sparse Domain
30
BM3D: Dabov et al. (2007) Collaborative filtering
Use hard thresholding or Wiener filter Each patch in the group gets a denoised estimate Unlike NLM β where only central pixel in reference patch got an estimate Filter / Thresholding R R Noisy patches Denoised Patches
31
BM3D: Dabov et al. (2007) Multiple BM3D Estimate R R R R R R R R
Collaborative filtering R R thr R R Multiple BM3D Estimate R Collaborative filtering R thr R R
32
BM3D: Dabov et al. (2007) R t ? Fusion
33
BM3D: Dabov et al. (2007) Each pixel gets multiple estimates from different groups Naive approach Average all estimates of each pixel β¦. not all estimates are as good Suggestion Give higher weight to more reliable estimates
34
π€β 1 #πππ ππππ πΆππππππππππ‘π
BM3D: Dabov et al. (2007) Give each estimate a weight according to denoising quality of its group Quality = Sparsity induced by the denoising Hard thresholding π€β 1 #πππ ππππ πΆππππππππππ‘π Weiner filtering π€β 1 πΉπππ‘ππ 2
35
BM3D: Dabov et al. (2007) Noise may result in poor matching β Degrades de-noising performance Improvements: Match using a smoothed version of the image Perform BM3D in 2 phases: Basic BM3D estimate β improved 3D groups Final BM3D Variational formulation: BM3D frame Danielyan, Katkovnik and Egiazarian. BM3D frames and variational image deblurring. IEEE TIP. 21(4):
36
BM3D: Dabov et al. (2007) Results
37
Dictionary Learning K-SVD, Data-Driven Tight Frame
38
Sparse Coding and Dictionary Learning
39
Dictionary Selection Which D to use? A fixed set of basis:
Steerable wavelet Contourlet DCT Basis β¦β¦ Data adaptive dictionary β learn from data K-SVD ( β 0 -norm) Data-Driven Tight Frame (DDTF)
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.