EE565 Advanced Image Processing Copyright Xin Li 20081 Why do we Need Image Model in the first place? Any image processing algorithm has to work on a collection.

Slides:



Advertisements
Similar presentations
CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising Jinxiang Chai.
Advertisements

CSCE643: Computer Vision Bayesian Tracking & Particle Filtering Jinxiang Chai Some slides from Stephen Roth.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: The Linear Prediction Model The Autocorrelation Method Levinson and Durbin.
Data-driven methods: Texture (Sz 10.5) Cs129 Computational Photography James Hays, Brown, Spring 2011 Many slides from Alexei Efros.
Pattern Recognition and Machine Learning: Kernel Methods.
Accelerating Spatially Varying Gaussian Filters Jongmin Baek and David E. Jacobs Stanford University.
Multi-Task Compressive Sensing with Dirichlet Process Priors Yuting Qi 1, Dehong Liu 1, David Dunson 2, and Lawrence Carin 1 1 Department of Electrical.
Uncertainty Representation. Gaussian Distribution variance Standard deviation.
ECE 472/572 - Digital Image Processing Lecture 8 - Image Restoration – Linear, Position-Invariant Degradations 10/10/11.
Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes (CVPR’10) Shengcai Liao, Guoying Zhao, Vili Kellokumpu,
Pattern Recognition and Machine Learning
Transform-based Models
Resampling techniques Why resampling? Jacknife Cross-validation Bootstrap Examples of application of bootstrap.
Texture Reading: Chapter 9 (skip 9.4) Key issue: How do we represent texture? Topics: –Texture segmentation –Texture-based matching –Texture synthesis.
Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.
EE565 Advanced Image Processing Copyright Xin Li Statistical Modeling of Natural Images in the Wavelet Space Parametric models of wavelet coefficients.
EE565 Advanced Image Processing Copyright Xin Li Different Frameworks for Image Processing Statistical/Stochastic Models: Wiener’s MMSE estimation.
Wavelet-based Texture Synthesis
Simple Linear Regression and Correlation
(1) A probability model respecting those covariance observations: Gaussian Maximum entropy probability distribution for a given covariance observation.
Despeckle Filtering in Medical Ultrasound Imaging
Linear Algebra and Image Processing
ENG4BF3 Medical Image Processing
Image Representation Gaussian pyramids Laplacian Pyramids
Motivation from Real-World Applications EE565 Advanced Image Processing Copyright Xin Li Noisy Photos Noisy ultrasound data.
1 Patch Complexity, Finite Pixel Correlations and Optimal Denoising Anat Levin, Boaz Nadler, Fredo Durand and Bill Freeman Weizmann Institute, MIT CSAIL.
With many thanks for slides & images to: FIL Methods group, Virginia Flanagin and Klaas Enno Stephan Dr. Frederike Petzschner Translational Neuromodeling.
Why do we Need Image Model in the first place?
CAP5415: Computer Vision Lecture 4: Image Pyramids, Image Statistics, Denoising Fall 2006.
Wavelets and Denoising Jun Ge and Gagan Mirchandani Electrical and Computer Engineering Department The University of Vermont October 10, 2003 Research.
1 Part 5 Response of Linear Systems 6.Linear Filtering of a Random Signals 7.Power Spectrum Analysis 8.Linear Estimation and Prediction Filters 9.Mean-Square.
INDEPENDENT COMPONENT ANALYSIS OF TEXTURES based on the article R.Manduchi, J. Portilla, ICA of Textures, The Proc. of the 7 th IEEE Int. Conf. On Comp.
EE4328, Section 005 Introduction to Digital Image Processing Linear Image Restoration Zhou Wang Dept. of Electrical Engineering The Univ. of Texas.
2 2  Background  Vision in Human Brain  Efficient Coding Theory  Motivation  Natural Pictures  Methodology  Statistical Characteristics  Models.
Chapter 21 R(x) Algorithm a) Anomaly Detection b) Matched Filter.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Definitions Random Signal Analysis (Review) Discrete Random Signals Random.
School of Electrical & Computer Engineering Image Denoising Using Steerable Pyramids Alex Cunningham Ben Clarke Dy narath Eang ECE November 2008.
EE565 Advanced Image Processing Copyright Xin Li Motivating Applications HDTV Internet video Artistic reproduction Widescreen movie.
Why do we Need Statistical Model in the first place? Any image processing algorithm has to work on a collection (class) of images instead of a single one.
Fields of Experts: A Framework for Learning Image Priors (Mon) Young Ki Baik, Computer Vision Lab.
Image Denoising Using Wavelets
Statistical Modeling of Images and its Application into Denoising What is statistics and why? a mathematical science pertaining to the collection, analysis,
EE565 Advanced Image Processing Copyright Xin Li Image Denoising Theory of linear estimation Spatial domain denoising techniques Conventional Wiener.
EE565 Advanced Image Processing Copyright Xin Li Why do we Need Image Model in the first place? Any image processing algorithm has to work on a collection.
EE565 Advanced Image Processing Copyright Xin Li Image Denoising: a Statistical Approach Linear estimation theory summary Spatial domain denoising.
Patch-based Image Interpolation: Algorithms and Applications
Patch-based Nonlocal Denoising for MRI and Ultrasound Images Xin Li Lane Dept. of CSEE West Virginia University.
EE565 Advanced Image Processing Copyright Xin Li Statistical Modeling of Natural Images in the Wavelet Space Why do we need transform? A 30-min.
Mixture Kalman Filters by Rong Chen & Jun Liu Presented by Yusong Miao Dec. 10, 2003.
Image Priors and the Sparse-Land Model
SuperResolution (SR): “Classical” SR (model-based) Linear interpolation (with post-processing) Edge-directed interpolation (simple idea) Example-based.
The Story of Wavelets Theory and Engineering Applications
EE565 Advanced Image Processing Copyright Xin Li Further Improvements Gaussian scalar mixture (GSM) based denoising* (Portilla et al.’ 2003) Instead.
Wavelet Thresholding for Multiple Noisy Image Copies S. Grace Chang, Bin Yu, and Martin Vetterli IEEE TRANSACTIONS
Single Image Interpolation via Adaptive Non-Local Sparsity-Based Modeling The research leading to these results has received funding from the European.
EE565 Advanced Image Processing Copyright Xin Li Application of Wavelets (I): Denoising Problem formulation Frequency-domain solution: linear Wiener.
EE5965 Advanced Image Processing Copyright Xin Li Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image.
WAVELET NOISE REMOVAL FROM BASEBAND DIGITAL SIGNALS IN BANDLIMITED CHANNELS Dr. Robert Barsanti SSST March 2010, University of Texas At Tyler.
1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.
Bayesian fMRI analysis with Spatial Basis Function Priors
Biointelligence Laboratory, Seoul National University
Questions from lectures
Directional Multiscale Modeling of Images
The Story of Wavelets Theory and Engineering Applications
The Chinese University of Hong Kong
Image Analysis Image Restoration.
Image Denoising in the Wavelet Domain Using Wiener Filtering
A Tutorial on Bayesian Speech Feature Enhancement
The Story of Wavelets Theory and Engineering Applications
Lecture 7 Patch based methods: nonlocal means, BM3D, K- SVD, data-driven (tight) frame.
Presentation transcript:

EE565 Advanced Image Processing Copyright Xin Li Why do we Need Image Model in the first place? Any image processing algorithm has to work on a collection (class) of images instead of a single one Mathematical model gives us the abstraction of common properties of the images within the same class Model is our hypothesis and images are our observation data In physics, can F=ma explain the relationship between force and acceleration?  In image processing, can this model fit this class of images?

Content EE565 Advanced Image Processing Copyright Xin Li regression models transform models patch models PDE models synthesisdenoisingcompression… theory LS method PCA/KLT NN/kNN Diff. Geo. 2D Kalman filtering, Gaussian MRF, Edge-directed prediction Pyramid-based synthesis, wavelet thresholding, EZW/SPIHT Image quilting, Nonlocal-mean/BM3D, ??? Reaction-diffusion, MCD/PMD, ???

EE565 Advanced Image Processing Copyright Xin Li Theory Thread: Statistical vs. Deterministic They are different languages invented by mathematicians to facilitate the communication of scientific results (just like English vs. Chinese spoken by people in different countries) None is better than other – pick up the one you feel most comfortable with We adopt a statistical language most of the time in this class

EE565 Advanced Image Processing Copyright Xin Li The Curse of Dimensionality Even for a small-size image such as 64- by-64, we need to model it by a random process in 4096-dimensional space (R 4096 ) whose covariance matrix is sized by 4096-by-4096 More importantly, we ask ourselves: do we need to consider all pixels simultaneously?

EE565 Advanced Image Processing Copyright Xin Li A Simple Idea: Locality The conditional pdf is determined by a local neighborhood N past samples In PDE-based framework, locality refers to the modeling based on local gradients (approximation of low-order derivatives)

EE565 Advanced Image Processing Copyright Xin Li Parametric vs. Nonparametric non-parametric sampling Input image XkXk Parametric model

EE565 Advanced Image Processing Copyright Xin Li Spatial vs. Wavelet

EE565 Advanced Image Processing Copyright Xin Li Complete vs. Overcomplete

EE565 Advanced Image Processing Copyright Xin Li Marginal PDF of wavelet coefficients where Laplacian Gaussian P: shape parameter : variance parameter

EE565 Advanced Image Processing Copyright Xin Li Joint PDF of Wavelet Coefficients Neighborhood I(Q): {Left,Up,cousin and aunt} X= Y= Joint pdf of two correlated random variables X and Y Can you use this model to interpret why EZW works?

EE565 Advanced Image Processing Copyright Xin Li Good Bad Spatially Fixed vs. Adaptive Models

EE565 Advanced Image Processing Copyright Xin Li Locality Revisited: “relativity theory” for image processing Input image N past samples The definition of local neighborhood has to be relative

EE565 Advanced Image Processing Copyright Xin Li Application I: Image Denoising Spatial domain denoising techniques Conventional Wiener filtering Spatially adaptive Wiener filtering Wavelet domain denoising Wavelet thresholding: hard vs. soft Wavelet-domain adaptive Wiener filtering From local to nonlocal denoising

EE565 Advanced Image Processing Copyright Xin Li Linear Frequency Weighting FT Power spectrum |X| 2

EE565 Advanced Image Processing Copyright Xin Li Spatially Adaptive Wiener Filtering of Wavelet Coefficients Basic assumption: image source is modeled by a nonstationary Gaussian process Signal variance is locally estimated from the windowed noisy observation data T T N=T 2 Recall

EE565 Advanced Image Processing Copyright Xin Li Wavelet Thresholding DWT IWTThresholding YX ~ Hard thresholding Soft thresholding Noisy signal denoised signal

EE565 Advanced Image Processing Copyright Xin Li Spatially Adaptive Wiener Filtering in Wavelet Domain Wavelet high-band coefficients are modeled by a Gaussian random variable with zero mean and spatially varying variance Apply Wiener filtering to wavelet coefficients, i.e., estimated in the same way as spatial-domain (Slide 15)

EE565 Advanced Image Processing Copyright Xin Li Translation Invariant Denoising Noisy image T ce T ce -1 ThresholdingWD = shift(m K,n K ) WD shift(-m K,-n K ) shift(m 1,n 1 ) WD shift(-m 1,-n 1 ) Avg denoised image  (m k,n k ): a pair of integers, k=1-K (K: redundancy ratio)

EE565 Advanced Image Processing Copyright Xin Li Further Improvements Gaussian scalar mixture (GSM) based denoising (Portilla et al.’ 2003) Instead of estimating the variance, it explicitly addresses the issue of uncertainty with variance estimation Hidden Markov Model (HMM) based denoising (Romberg et al.’ 2001) Build a HMM for wavelet high-band coefficients (refer to the posted paper)

EE565 Advanced Image Processing Copyright Xin Li Nonlocal Patch-based Denoising WD T T -1 ThresholdingWD = Noisy patches Denoised patches

EE565 Advanced Image Processing Copyright Xin Li Application II: Texture Syntehsis Spatial-domain models Parametric autoregressive model Nonparametric resampling based Wavelet-domain models Histogram matching based Parametric models based joint-statistics

EE565 Advanced Image Processing Copyright Xin Li Spatial-Domain Parametric Texture Synthesis

EE565 Advanced Image Processing Copyright Xin Li Nonparametric Texture Synthesis

EE565 Advanced Image Processing Copyright Xin Li Wavelet-domain Histogram Matching

EE565 Advanced Image Processing Copyright Xin Li Wavelet-Domain Parametric Texture Models original synthesized

EE565 Advanced Image Processing Copyright Xin Li Other Applications Interpolation Spatial-domain covariance-based models PDE-based (nonlinear diffusion) models Compression Statistical modeling of wavelet coefficients Dual to wavelet-based image denoising Object recognition From NN/kNN to sparsity-based method

EE565 Advanced Image Processing Copyright Xin Li Summary on Theory Image models are at the foundation of any image processing algorithm Statistical models help us deal with the uncertainty in observation data Appropriate image representation (e.g., prediction/transform) facilitates the modeling task Spatial adaptation is important – to have a good model for a wide class of images Localized models are popular and powerful but nonlocal models might prevail later

EE565 Advanced Image Processing Copyright Xin Li Summary on Practice MATLAB provides a user friendly platform for testing your ideas You can see what you have done Experimental efficiency is important Avoid loops and test small-size images C/C++ programming skills are a plus Efficient implementation could make a difference