IPIM, IST, José Bioucas, Shrinkage/Thresholding Iterative Methods Nonquadratic regularizers Total Variation lp- norm Wavelet orthogonal/redundant representations sparse regression Majorization Minimization revisietd IST- Iterative Shrinkage Thresolding Methods TwIST-Two step IST

IPIM, IST, José Bioucas, Linear Inverse Problems -LIPs

IPIM, IST, José Bioucas, References J. Bioucas-Dias and M. Figueiredo, "A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration“ Submitted to IEEE Transactions on Image processing, M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization-Minimization Algorithms for Wavelet-Based Image Deconvolution'', Submitted to IEEE Transactions on Image processing, 2006.

IPIM, IST, José Bioucas, More References M. Figueiredo and R. Nowak, “An EM algorithm for wavelet-based image restoration,” IEEE Trans. on Image Processing, vol. 12, no. 8, pp. 906–916, J. Bioucas-Dias, “Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors,” IEEE Trans. on Image Processing, vol. 15, pp. 937–951, A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision, vol. 20, pp , P. Combettes and V. Wajs, “Signal recovery by proximal forwardbackward splitting,” SIAM Journal on Multiscale Modeling & Simulation vol. 4, pp. 1168–1200, 2005 I. Daubechies, M. Defriese, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint”, Communications on Pure and Applied Mathematics, vol. LVII, pp , 2004

IPIM, IST, José Bioucas, Majorization Minorization (MM) Framework Let EM is an algorithm of this type. Majorization Minorization algorithm:....with equality if and only if Easy to prove monotonicity: Notes: should be easy to maximize

IPIM, IST, José Bioucas, MM Algorithms for LIPs IST Class: Majorize IRS Class: Majorize IST/IRS: Majorize and

IPIM, IST, José Bioucas, MM Algorithms: IST class Assume that

IPIM, IST, José Bioucas, MM Algorithms: IST class Majorizer: Let: IST Algorithm

IPIM, IST, José Bioucas, MM Algorithms: IST class Overrelaxed IST Algorithm Convergence: [Combettes and V. Wajs, 2004] is convex the set of minimizers, G, of is non-empty  2 ]0,1] Then converges to a point in G

IPIM, IST, José Bioucas, Denoising with convex regularizers Denoising function also known as the Moreou proximal mapping Classes of convex regularizers: 1- homogeneous (TV, lp-norm (p>1)) 2- p power of an lp norm

IPIM, IST, José Bioucas, Homogeneous regularizers Then where is a closed convex set and denotes the orthogonal projection on the convex set

IPIM, IST, José Bioucas, Total variation regularization Total variation [S. Osher, L. Rudin, and E. Fatemi, 1992]  is convex (although not strightly) and 1-homogeneous Total variation is a discontinuity-preserving regularizer have the same TV

IPIM, IST, José Bioucas, Then Total variation regularization [Chambolle, 2004]

IPIM, IST, José Bioucas, Total variation denoising

IPIM, IST, José Bioucas, Total variation deconvolution 2000 IST iterations !!!

IPIM, IST, José Bioucas, Weighted lp-norms  is convex (although not strightly) and 1-homogeneous There is no closed form expression for excepts for some particular cases Thus

IPIM, IST, José Bioucas, Soft thresholding: p=1 Thus

IPIM, IST, José Bioucas, Soft thresholding: p=1

IPIM, IST, José Bioucas, Soft thresholding: p=1

IPIM, IST, José Bioucas, Another way to look at it: Since L is convex: the point is a global minimum of L iif where is the subdifferential of L at f’

IPIM, IST, José Bioucas, Example: Wavelet-based restoration Wavelet basis Wavelet coefficients Detail coefficients (h – high pass filter) Approximation coefficients (g-low pass filter) g,h – quadrature mirror filters DWT, Harr, J=2

IPIM, IST, José Bioucas, Example: Wavelet-based restoration Histogram of coefficients - h Histogram of coefficients – log h