1. Sparse and Redundant Representations and Their Applications in
Signal and Image Processing (236862)
Section 4: MAP vs. MMSE Estimation
Winter Semester, 2018/2019
Michael (Miki) Elad

2. Meeting Plan Quick review of the material covered
Answering questions from the students and getting their feedback
Discussing a new material – MMSE Estimation via the Stochastic Resonance Concept – Given by Dror Simon
Administrative issues

3. Overview of the Material
Sparseland: An Estimation Point of View
A Strange Experiment
A Crash-Course on Estimation Theory
Sparseland: Approximate Estimation
MMSE: Back to Reality

4. Your Questions and Feedback
Answer: MAP Estimation and the Relation to OMP:
In the slides we derive the MAP estimation and get a strange looking expression, and yet we claim that OMP approximates it. We did see that for |s|=1 the two align, but is there a clearer view of the connection between the estimation approach and the OMP or the pursuit we saw so many times in this course?
Answer: Let’s Try …
Here is an attempt to clarify this connection. It will be obtained by deriving the MAP little bit differently, and by relying on different assumptions

5. Your Questions and Feedback
Lets define our estimation goal:
In this step we add the support as an unknown, so that we search for the most probable representation and its support TOGETHER
Using Bayes rule we obtain:

6. Your Questions and Feedback
Returning to our MAP goal:

7. Your Questions and Feedback
So, this is what we just got:
Suppose that this is our probability
Elastic Net
This leads to:
which is the P0 we kept advocating, with a twist

8. Your Questions and Feedback

9. New Material? MMSE Estimation: Another Point of View
We are hooked to the idea that noise is … noise, i.e. an annoying disturbance to be avoided
For example, consider the previous chapter about denoising …
Could noise be helpful in engineering processes? Are you aware of cases in which noise (or randomness) is beneficial?
Observe that the RandOMP described in this chapter used randomness to achieve its MMSE estimation
How could we offer a parallel to the RandOMP that is BP-based?
This brings us to Dror Simon’s presentation on Stochastic Resonance

10. Administrative Issues
Lets talk about the following:
Your feedback is very much needed in the Technion’s course (MISHAAL HAMARTZE) & (+ free text comments)
Are there questions about the final project?