1. Using Neumann Series to Solve Inverse Problems in Imaging Christopher Kumar Anand

2. Anand-Neumann Series-AdvOl-Feb2007 2 Inverse Problem given measurements ( ) model ( ) Solve for.

3. Anand-Neumann Series-AdvOl-Feb2007 3 Seismic Imaging 1. Bang 2. Listen 3. Solve Acoustic Equations http://en.wikipedia.org/wiki/Seismology

4. Anand-Neumann Series-AdvOl-Feb2007 4 Magnetic Resonance Imaging 0. Tissue Density 1. Phase Modulation 2. Sample Fourier Transform 3. Invert Linear System

5. Anand-Neumann Series-AdvOl-Feb2007 5 Challenging when... model is big 1 000 000 000 variables model is nonlinear data is inexact (usually know error probabilistically)

6. Anand-Neumann Series-AdvOl-Feb2007 6 Imaging discretize continuous model regular volume/area elements sparse structure

7. Anand-Neumann Series-AdvOl-Feb2007 7 Solutions: Noise filter noisy solution 1. convolution filter 2. bilateral filter 3. Anisotropic Diffusion (uses pde) regularize via penalty 4. energy 5. Total Variation 6. something new

8. Anand-Neumann Series-AdvOl-Feb2007 8 Solutions: Problem Size I. use a fast method (i.e. based on FFT) II. use (parallelizable) iterative method a. Conjugate Gradient b. Neumann series III. use sparsity c. choose penalties with sparse Hessians IV. use fast hardware d. 1000-way parallelizable e. single precision

9. Anand-Neumann Series-AdvOl-Feb2007 9 Cell BE 25 GFlops DP 200 GFlops SP need 384-way ||ism 4-way SIMD 8-way cores 6-times unrolling double buffering

10. Anand-Neumann Series-AdvOl-Feb2007 10 Solutions: Nonlinearity use iterative method A. sequential projection onto convex sets B. trust region C. sequential quadratic approximations

11. Anand-Neumann Series-AdvOl-Feb2007 11 Plan of Talk A. example/benchmark B. optimization 1. fit to data 2. regularization i. new penalty (with optimized gradient) ii. nonlinear penalties C. solution 3. operator decomposition 4. Neumann series D. proof of convergence E. numerical example 5. noise reduction 6. linear convergence

12. Anand-Neumann Series-AdvOl-Feb2007 12 Example/Benchmark complex image complex data model

13. Anand-Neumann Series-AdvOl-Feb2007 13 Example/Benchmark diagonal blocks easily invertible

14. Anand-Neumann Series-AdvOl-Feb2007 14 Example/Benchmark direct inverse

15. Anand-Neumann Series-AdvOl-Feb2007 15 Optimizationfit-to-data quadratic penalties nonlinear penalties

16. Anand-Neumann Series-AdvOl-Feb2007 16 Fit to Data typically dense transformation use fast matrix-vector products e.g. FFT-based forward/adjoint problem linear forward problem gives quadratic objective

17. Anand-Neumann Series-AdvOl-Feb2007 17 Bilateral Filter spatial kernel range kernel Bilateral Regularizer

18. Anand-Neumann Series-AdvOl-Feb2007 18 Quadtratic Case optimize direction of gradient LP problem like FIR filter design heuristic choice of “stop band”

19. Anand-Neumann Series-AdvOl-Feb2007 19 Optimal Spatial Kernel discrete kernel not rotational symmetric exact in quadratic case quadratic case use in all cases all cases

20. Anand-Neumann Series-AdvOl-Feb2007 20 TV-like similar properties to Total Variation won’t smooth edges not differentiable

21. Anand-Neumann Series-AdvOl-Feb2007 21 Sequential Quadratic TV-like sequential quadratic approximation tangent to TV-like

22. Anand-Neumann Series-AdvOl-Feb2007 22 Mask penalize pixel values outside object

23. Anand-Neumann Series-AdvOl-Feb2007 23 Segmentation probability of observing pixels assumes discrete pixel values equal likelihood equal normal error

24. Anand-Neumann Series-AdvOl-Feb2007 24 Solve: Operator Decomposition“small” block diagonal with sparse banded blocks linear-time Cholesky decomposition leads to formal expression image rows blocks =

25. Anand-Neumann Series-AdvOl-Feb2007 25 plus (poly-order)(fast algorithm) linear in problem size Calculate Using Fast Matrix-Vector Ops

26. Anand-Neumann Series-AdvOl-Feb2007 26 Row by Row each block corresponds to a row each block can be calculated in parallel (number-rows)-way parallelism

27. Anand-Neumann Series-AdvOl-Feb2007 27 Even Better: Use Minimax Polynomial calculate

28. Anand-Neumann Series-AdvOl-Feb2007 28 minimax polynomial error step shortening Convergence next error current error

29. Anand-Neumann Series-AdvOl-Feb2007 29 Safe in Single- Precision recalculate gradient at each outer iteration numerical error only builds up during polynomial evaluation coefficients well-behaved (and in our control)

30. Anand-Neumann Series-AdvOl-Feb2007 30 Numerical Tests start with quadratic penalties add nonlinear penalties and change weights with and without time fixed budget for computation

31. Anand-Neumann Series-AdvOl-Feb2007 31 10 iterations I. 10 iterations with bi2 regularization

32. Anand-Neumann Series-AdvOl-Feb2007 32 100 iterations I. 10 iterations with bi2 regularization II. introduce other penalties 1. masking 2. magnet 3. segmentation

33. Anand-Neumann Series-AdvOl-Feb2007 33 15 iterations optimized c simple c

34. Anand-Neumann Series-AdvOl-Feb2007 34 Absolute Error iteration error

35. Anand-Neumann Series-AdvOl-Feb2007 35 Relative (to limit) Error iteration error

36. Anand-Neumann Series-AdvOl-Feb2007 36 Conclusion highly-parallel safe in single precision robust with respect to noise accommodates nonlinear penalties

37. Anand-Neumann Series-AdvOl-Feb2007 37 Thanks to: of students and colleagues in the