Recent Trends in Signal Representations and Their Role in Image Processing Michael Elad The CS Department The Technion – Israel Institute of technology.

Slides:



Advertisements
Similar presentations
Compressive Sensing IT530, Lecture Notes.
Advertisements

MMSE Estimation for Sparse Representation Modeling
Joint work with Irad Yavneh
Fast Bayesian Matching Pursuit Presenter: Changchun Zhang ECE / CMR Tennessee Technological University November 12, 2010 Reading Group (Authors: Philip.
Submodular Dictionary Selection for Sparse Representation Volkan Cevher Laboratory for Information and Inference Systems - LIONS.
Sparse Representations and the Basis Pursuit Algorithm* Michael Elad The Computer Science Department – Scientific Computing & Computational mathematics.
K-SVD Dictionary-Learning for Analysis Sparse Models
Wangmeng Zuo, Deyu Meng, Lei Zhang, Xiangchu Feng, David Zhang
Learning sparse representations to restore, classify, and sense images and videos Guillermo Sapiro University of Minnesota Supported by NSF, NGA, NIH,
Extensions of wavelets
* * Joint work with Michal Aharon Freddy Bruckstein Michael Elad
1 Micha Feigin, Danny Feldman, Nir Sochen
Ilias Theodorakopoulos PhD Candidate
An Introduction to Sparse Coding, Sparse Sensing, and Optimization Speaker: Wei-Lun Chao Date: Nov. 23, 2011 DISP Lab, Graduate Institute of Communication.
Sparse & Redundant Signal Representation, and its Role in Image Processing Michael Elad The CS Department The Technion – Israel Institute of technology.
Dictionary-Learning for the Analysis Sparse Model Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa 32000,
Image Super-Resolution Using Sparse Representation By: Michael Elad Single Image Super-Resolution Using Sparse Representation Michael Elad The Computer.
Sparse and Overcomplete Data Representation
SRINKAGE FOR REDUNDANT REPRESENTATIONS ? Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa 32000, Israel.
Mathematics and Image Analysis, MIA'06
Image Denoising via Learned Dictionaries and Sparse Representations
Over-Complete & Sparse Representations for Image Decomposition and Inpainting Michael Elad The Computer Science Department The Technion – Israel Institute.
Image Denoising and Beyond via Learned Dictionaries and Sparse Representations Michael Elad The Computer Science Department The Technion – Israel Institute.
An Introduction to Sparse Representation and the K-SVD Algorithm
ITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa.
New Results in Image Processing based on Sparse and Redundant Representations Michael Elad The Computer Science Department The Technion – Israel Institute.
Optimized Projection Directions for Compressed Sensing Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa.
Image Denoising with K-SVD Priyam Chatterjee EE 264 – Image Processing & Reconstruction Instructor : Prof. Peyman Milanfar Spring 2007.
Random Convolution in Compressive Sampling Michael Fleyer.
* Joint work with Michal Aharon Guillermo Sapiro
Introduction to Compressive Sensing
Analysis of the Basis Pustuit Algorithm and Applications*
Image Decomposition and Inpainting By Sparse & Redundant Representations Michael Elad The Computer Science Department The Technion – Israel Institute of.
Sparse Representations of Signals: Theory and Applications *
SOS Boosting of Image Denoising Algorithms
Over-Complete & Sparse Representations for Image Decomposition*
A Weighted Average of Sparse Several Representations is Better than the Sparsest One Alone Michael Elad The Computer Science Department The Technion –
A Sparse Solution of is Necessarily Unique !! Alfred M. Bruckstein, Michael Elad & Michael Zibulevsky The Computer Science Department The Technion – Israel.
Sparse and Redundant Representation Modeling for Image Processing Michael Elad The Computer Science Department The Technion – Israel Institute of technology.
Topics in MMSE Estimation for Sparse Approximation Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa 32000,
Alfredo Nava-Tudela John J. Benedetto, advisor
The Quest for a Dictionary. We Need a Dictionary  The Sparse-land model assumes that our signal x can be described as emerging from the PDF:  Clearly,
Compressed Sensing Compressive Sampling
AMSC 6631 Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Midyear Report Alfredo Nava-Tudela John J. Benedetto,
Cs: compressed sensing
Sparse Representations of Signals: Theory and Applications * Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000,
“A fast method for Underdetermined Sparse Component Analysis (SCA) based on Iterative Detection- Estimation (IDE)” Arash Ali-Amini 1 Massoud BABAIE-ZADEH.
 Karthik Gurumoorthy  Ajit Rajwade  Arunava Banerjee  Anand Rangarajan Department of CISE University of Florida 1.
Fast and incoherent dictionary learning algorithms with application to fMRI Authors: Vahid Abolghasemi Saideh Ferdowsi Saeid Sanei. Journal of Signal Processing.
Learning to Sense Sparse Signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization Julio Martin Duarte-Carvajalino, and Guillermo Sapiro.
Sparse & Redundant Representation Modeling of Images Problem Solving Session 1: Greedy Pursuit Algorithms By: Matan Protter Sparse & Redundant Representation.
Image Decomposition, Inpainting, and Impulse Noise Removal by Sparse & Redundant Representations Michael Elad The Computer Science Department The Technion.
A Weighted Average of Sparse Representations is Better than the Sparsest One Alone Michael Elad and Irad Yavneh SIAM Conference on Imaging Science ’08.
Image Priors and the Sparse-Land Model
Single Image Interpolation via Adaptive Non-Local Sparsity-Based Modeling The research leading to these results has received funding from the European.
Jianchao Yang, John Wright, Thomas Huang, Yi Ma CVPR 2008 Image Super-Resolution as Sparse Representation of Raw Image Patches.
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Alfred M. Bruckstein (Technion), David L. Donoho (Stanford), Michael.
My Research in a Nut-Shell Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa 32000, Israel Meeting with.
Sparsity Based Poisson Denoising and Inpainting
Compressive Coded Aperture Video Reconstruction
Jeremy Watt and Aggelos Katsaggelos Northwestern University
Wavelets : Introduction and Examples
A Motivating Application: Sensor Array Signal Processing
Optimal sparse representations in general overcomplete bases
Sparse and Redundant Representations and Their Applications in
Sparse and Redundant Representations and Their Applications in
Improving K-SVD Denoising by Post-Processing its Method-Noise
* * Joint work with Michal Aharon Freddy Bruckstein Michael Elad
Outline Sparse Reconstruction RIP Condition
Presentation transcript:

Recent Trends in Signal Representations and Their Role in Image Processing Michael Elad The CS Department The Technion – Israel Institute of technology Haifa 32000, Israel Research Day on Image Analysis and Processing Industrial Affiliate Program (IAP) The Computer Science Department – The Technion January 27 th, 2005

Recent Advances in Signal Representations and Applications to Image Processing 2 Today’s Talk is About Sparsity and Overcompleteness We will show today that Sparsity & Overcompleteness can & should be used to design new powerful signal/image processing tools (e.g., transforms, priors, models, …), The obtained machinery works very well – we will show these ideas deployed to applications.

Recent Advances in Signal Representations and Applications to Image Processing 3 Agenda 1.A Visit to Sparseland Motivating Sparsity & Overcompleteness 2.Problem 1: Transforms & Regularizations How & why should this work? 3.Problem 2: What About D? The quest for the origin of signals 4.Problem 3: Applications Image filling in, denoising, separation, compression, … Welcome to Sparseland

Recent Advances in Signal Representations and Applications to Image Processing 4 Generating Signals in Sparseland MM K N A fixed Dictionary Every column in D (dictionary) is a prototype signal (Atom). The vector  is generated randomly with few non-zeros in random locations and random values. A sparse & random vector N

Recent Advances in Signal Representations and Applications to Image Processing 5 Simple: Every generated signal is built as a linear combination of few atoms from our dictionary D Rich: A general model: the obtained signals are a special type mixture- of-Gaussians (or Laplacians). Sparseland Signals Are Special Multiply by D MM

Recent Advances in Signal Representations and Applications to Image Processing 6 Transforms in Sparseland ? MM MM Assume that x is known to emerge from.. We desire simplicity, independence, and expressiveness. MM How about “Given x, find the α that generated it in ” ? TT

Recent Advances in Signal Representations and Applications to Image Processing 7 Known So, In Order to Transform … We need to solve an under-determined linear system of equations: Sparsity is measured using the L 0 norm: Among all (infinitely many) possible solutions we want the sparsest !!

Recent Advances in Signal Representations and Applications to Image Processing 8 Are there practical ways to get ? Signal’s Transform in Sparseland Multiply by D A sparse & random vector 4 Major Questions Is ? Under which conditions? How effective are those ways? How would we get D?

Recent Advances in Signal Representations and Applications to Image Processing 9 Inverse Problems in Sparseland ? Assume that x is known to emerge from. M MM Suppose we observe, a “blurred” and noisy version of x with. How will we recover x? Noise How about “find the α that generated the x …” again? Q

Recent Advances in Signal Representations and Applications to Image Processing 10 Inverse Problems in Sparseland ? Multiply by D A sparse & random vector “blur” by H 4 Major Questions (again!) How would we get D? Are there practical ways to get ? How effective are those ways? Is ? How far can it go?

Recent Advances in Signal Representations and Applications to Image Processing 11 Back Home … Any Lessons? Several recent trends worth looking at:  JPEG to JPEG From (L 2 -norm) KLT to wavelet and non- linear approximation  From Wiener to robust restoration – From L 2 -norm (Fourier) to L 1. (e.g., TV, Beltrami, wavelet shrinkage …)  From unitary to richer representations – Frames, shift-invariance, bilateral, steerable, curvelet  Approximation theory – Non-linear approximation  ICA and related models Sparsity. Overcompleteness. Sparsity. Sparsity & Overcompleteness. Independence and Sparsity. Sparseland is HERE

Recent Advances in Signal Representations and Applications to Image Processing 12 To Summarize so far … We do! this model is relevant to us just as well. Who cares? So, what are the implications? Sparsity? looks complicated (a) Practical solvers? (b) How will we get D? (c) Applications? The Sparseland model for signals is interesting since it is rich and yet every signal has a simple description MM We need to solve (or approximate the solution of) a linear system with more unknowns than equations, while finding the sparsest possible solution

Recent Advances in Signal Representations and Applications to Image Processing 13 Agenda 1.A Visit to Sparseland Motivating Sparsity & Overcompleteness 2.Problem 1: Transforms & Regularizations How & why should this work? 3.Problem 2: What About D? The quest for the origin of signals 4.Problem 3: Applications Image filling in, denoising, separation, compression, … T Q

Recent Advances in Signal Representations and Applications to Image Processing 14 Lets Start with the Transform … known Put formally, Our dream for Now: Find the sparsest solution of Known

Recent Advances in Signal Representations and Applications to Image Processing 15 Questions to Address Is ? Under which conditions? Are there practical ways to get ? How effective are those ways? How would we get D? 4 Major Questions

Recent Advances in Signal Representations and Applications to Image Processing 16 Multiply by D MM Question 1 – Uniqueness? Suppose we can solve this exactly Why should we necessarily get ? It might happen that eventually.

Recent Advances in Signal Representations and Applications to Image Processing 17 Matrix “Spark” Rank = 4 Spark = 3 Example: Donoho & Elad (‘02) Definition: Given a matrix D,  =Spark{D} is the smallest and and number of columns that are linearly dependent.

Recent Advances in Signal Representations and Applications to Image Processing 18 Uniqueness Rule Suppose this problem has been solved somehow This result implies that if generates signals using “sparse enough” , the solution of the above will find it exactly. MM If we found a representation that satisfy Then necessarily it is unique (the sparsest). Uniqueness Donoho & Elad (‘02)

Recent Advances in Signal Representations and Applications to Image Processing 19 Are there reasonable ways to find ? MM Question 2 – Practical P 0 Solver? Multiply by D

Recent Advances in Signal Representations and Applications to Image Processing 20 Matching Pursuit (MP) Mallat & Zhang (1993) Next steps: given the previously found atoms, find the next one to best fit … The Orthogonal MP (OMP) is an improved version that re-evaluates the coefficients after each round. The MP is a greedy algorithm that finds one atom at a time. Step 1: find the one atom that best matches the signal.

Recent Advances in Signal Representations and Applications to Image Processing 21 Basis Pursuit (BP) Instead of solving Solve Instead Chen, Donoho, & Saunders (1995) The newly defined problem is convex. It has a Linear Programming structure. Very efficient solvers can be deployed:  Interior point methods [Chen, Donoho, & Saunders (`95)],  Sequential shrinkage for union of ortho-bases [Bruce et.al. (`98)],  If computing Dx and D T  are fast, based on shrinkage [Elad (`05)].

Recent Advances in Signal Representations and Applications to Image Processing 22 Multiply by D MM Question 3 – Approx. Quality? How effective are the MP/BP in finding ?

Recent Advances in Signal Representations and Applications to Image Processing 23 Evaluating the “Spark” The Mutual Incoherence is a property of the dictionary (just like the “Spark”). The smaller it is, the better the dictionary. The Mutual Incoherence M is the largest entry in absolute value outside the main diagonal of D T D. DTDT = D DTDDTD Compute Assume normalized columns

Recent Advances in Signal Representations and Applications to Image Processing 24 BP and MP Equivalence Given a signal x with a representation, Assuming that, BP and MP are Guaranteed to find the sparsest solution. Donoho & Elad (‘02) Gribonval & Nielsen (‘03) Tropp (‘03) Temlyakov (‘03) Equivalence  MP is typically inferior to BP!  The above result corresponds to the worst-case.  Average performance results are available too, showing much better bounds [Donoho (`04), Candes et.al. (`04), Elad and Zibulevsky (`04)].

Recent Advances in Signal Representations and Applications to Image Processing 25 What About Inverse Problems? “blur” by H We had similar questions regarding uniqueness, practical solvers, and their efficiency. It turns out that similar answers are applicable here due to several recent works [Donoho, Elad, and Temlyakov (`04), Tropp (`04), Fuchs (`04)]. Multiply by D A sparse & random vector

Recent Advances in Signal Representations and Applications to Image Processing 26 To Summarize so far … The Sparseland model for signals is relevant to us. We can design transforms and priors based on it Use pursuit Algorithms Find the sparsest solution? A sequence of works during the past 3-4 years gives theoretic justifications for these tools behavior Why works so well? What next? (a)How shall we find D? (b)Will this work for applications? Which?

Recent Advances in Signal Representations and Applications to Image Processing 27 Agenda 1.A Visit to Sparseland Motivating Sparsity & Overcompleteness 2.Problem 1: Transforms & Regularizations How & why should this work? 3.Problem 2: What About D ? The quest for the origin of signals 4.Problem 3: Applications Image filling in, denoising, separation, compression, …

Recent Advances in Signal Representations and Applications to Image Processing 28 Problem Setting Multiply by D MM Given these P examples and a fixed size [N  K] dictionary D: 1. Is D unique? 2. How would we find D?

Recent Advances in Signal Representations and Applications to Image Processing 29 Uniqueness? If is rich enough* and if then D is unique. Uniqueness Aharon, Elad, & Bruckstein (`05) Comments: MM “Rich Enough”: The signals from could be clustered to groups that share the same support. At least L+1 examples per each are needed. This result is proved constructively, but the number of examples needed to pull this off is huge – we will show a far better method next. A parallel result that takes into account noise can be constructed similarly.

Recent Advances in Signal Representations and Applications to Image Processing 30 Each example has a sparse representation with no more than L atoms Each example is a linear combination of atoms from D Practical Approach – Objective D  X A

Recent Advances in Signal Representations and Applications to Image Processing 31 The K–SVD Algorithm – General D Initialize D Sparse Coding Use MP or BP Dictionary Update Column-by-Column by SVD computation Aharon, Elad, & Bruckstein (`04) X

Recent Advances in Signal Representations and Applications to Image Processing 32 K–SVD Sparse Coding Stage D X For the j th example we solve Pursuit Problem !!!

Recent Advances in Signal Representations and Applications to Image Processing 33 K–SVD Dictionary Update Stage D G k : The examples in and that use the column d k. The content of d k influences only the examples in G k. Let us fix all A apart from the k th column and seek both d k and the k th column to better fit the residual!

Recent Advances in Signal Representations and Applications to Image Processing 34 d k is obtained by SVD on the examples’ residual in G k. K–SVD Dictionary Update Stage D Residual E We should solve:

Recent Advances in Signal Representations and Applications to Image Processing 35 To Summarize so far … The Sparseland model for signals is relevant to us. In order to use it effectively we need to know D Use the K-SVD algorithm How D can be found? We have established a uniqueness result and shown how to practically train D using the K-SVD Will it work well? What next? (a)Deploy to applications (b)Generalize in various ways: multiscale, non-negative factorization, speed-up, … (c)Performance guarantees?

Recent Advances in Signal Representations and Applications to Image Processing 36 Agenda 1.A Visit to Sparseland Motivating Sparsity & Overcompleteness 2.Problem 1: Transforms & Regularizations How & why should this work? 3.Problem 2: What About D? The quest for the origin of signals 4.Problem 3: Applications Image filling in, denoising, separation, compression, …

Recent Advances in Signal Representations and Applications to Image Processing 37 Inpainting (1) [Elad, Starck, & Donoho (`04)] Source Outcome predetermined dictionary: Curvelet+DCT

Recent Advances in Signal Representations and Applications to Image Processing 38 Inpainting (2) Source Outcome predetermined dictionary: Curvelet+DCT

Recent Advances in Signal Representations and Applications to Image Processing 39 10,000 sample 8-by-8 images. 441 dictionary elements. Approximation method: MP. K-SVD: Overcomplete Haar K-SVD on Images [Aharon, Elad, & Bruckstein (`04)]

Recent Advances in Signal Representations and Applications to Image Processing 40 Filling-In Missing Pixels 90% missing pixels Haar Results Average # coefficients 4.48 RMSE: K-SVD Results Average # coefficients 4.27 RMSE: 25.32

Recent Advances in Signal Representations and Applications to Image Processing 41 Filling-In Missing Pixels

Recent Advances in Signal Representations and Applications to Image Processing 42 K-SVD dictionary Haar dictionary DCT dictionary OMP with error bound Compression

Recent Advances in Signal Representations and Applications to Image Processing 43 DCT Results BPP = 1.01 RMSE = 8.14 K-SVD Dictionary BPP = RMSE = 7.67 Haar Dictionary BPP = RMSE = 8.41 Compression

Recent Advances in Signal Representations and Applications to Image Processing 44 Compression

Recent Advances in Signal Representations and Applications to Image Processing 45 Today We Have Discussed 1.A Visit to Sparseland Motivating Sparsity & Overcompleteness 2.Problem 1: Transforms & Regularizations How & why should this work? 3.Problem 2: What About D ? The quest for the origin of signals 4.Problem 3: Applications Image filling in, denoising, separation, compression, …

Recent Advances in Signal Representations and Applications to Image Processing 46 Summary Sparsity and Over- completeness are important ideas that can be used in designing better tools in signal/image processing We are working on resolving those difficulties: Performance of pursuit alg. Speedup of those methods, Training the dictionary, Demonstrating applications, … There are difficulties in using them! Future transforms and regularizations will be data- driven, non-linear, overcomplete, and promoting sparsity. The dream?