1. Martin Burger Institut für Numerische und Angewandte Mathematik European Institute for Molecular Imaging CeNoS Quantitative Molecular Imaging – A Mathematical Challenge (?)

2. Martin Burger Molecular Imaging 2 Stuttgart, 27.5.2008 Mathematical Imaging@WWU Christoph Brune Alex Sawatzky Frank Wübbeling Thomas Kösters Martin Benning Marzena Franek Christina Stöcker Mary Wolfram (Linz) Thomas Grosser Jahn Müller

3. Martin Burger Molecular Imaging 3 Stuttgart, 27.5.2008 Major Cooperation Partners: SFB 656 /EIMI Otmar Schober (Nuclear Medicine) Klaus Schäfers (Medical Physics, EIMI) Florian Büther (EIMI) Funding: Regularization with Singular Energies (DFG), SFB 656 (DFG), European Institute for Molecular Imaging (WWU + SIEMENS Medical Solutions)

4. Martin Burger Molecular Imaging 4 Stuttgart, 27.5.2008 Major Cooperation Partners: Nanoscopy Andreas Schönle, Stefan Hell (MPI Göttingen) Thorsten Hohage, Axel Munk (Univ Göttingen) Nico Bissantz (Bochum) Funding: „Verbundprojekt INVERS“ (BMBF )

5. Martin Burger Molecular Imaging 5 Stuttgart, 27.5.2008 21st Century Imaging Imaging nowadays mainly separates into two steps - Image Reconstruction: computation of an image from (indirectly) measured data – solution of inverse problems - Image Processing: improvement of given images / image sequences – filtering, variational problems Mathematical issues and approaches (as well as communities) are very separated Images are passed on from step 1 to step 2 Is this an optimal approach ?

6. Martin Burger Molecular Imaging 6 Stuttgart, 27.5.2008 Image reconstruction and inverse problems Inverse Problems consist in reconstruction of the cause of an observed effect (via a mathematical model relating them) Diagnosis in medicine is a prototypical example (non- invase approaches always need indirect measurements) Crime is another one … "The grand thing is to be able to reason backwards." Arthur Conan Doyle (A study in scarlet)

7. Martin Burger Molecular Imaging 7 Stuttgart, 27.5.2008 Medical Imaging: CT Classical image reconstruction example: computerized tomography (CT) Mathematical Problem: Reconstruction of a density function from its line integrals Inversion of the Radon transform cf. Natterer 86, Natterer-Wübbeling 02

8. Martin Burger Molecular Imaging 8 Stuttgart, 27.5.2008 Medical Imaging: CT + Low noise level + High spatial resolution + Exact reconstruction + Reasonable Costs - Restricted to few seconds (radiation exposure, 20 mSiewert) - No functional information - Few mathematical challenges left Soret, Bacharach, Buvat 07 Schäfers et al 07

9. Martin Burger Molecular Imaging 9 Stuttgart, 27.5.2008 Medical Imaging: MR + Low noise level + High spatial resolution + Reconstruction by Fourier inversion + No radiation exposure + Good contrast in soft matter - Low tracer sensitivity - Limited functional information - Expensive - Few mathematical challenges left Courtesy Carsten Wolters, University Hospital Münster

10. Martin Burger Molecular Imaging 10 Stuttgart, 27.5.2008 Molecular Imaging: PET (Human / Small animal) Positron-Emission-Tomography Data: detecting decay events of an radioactive tracer Decay events are random, but their rate is proportional to the tracer uptake (Radon transform with random directions) Imaging of molecular properties

11. Martin Burger Molecular Imaging 11 Stuttgart, 27.5.2008 Medical Imaging: PET + High sensitivity + Long time (mins ~ 1 hour, radiation exposure 8-12 mSiewert) + Functional information + Many open mathematical questions - Few anatomical information - High noise level and disturbing effects (damping, scattering, … ) - Low spatial resolution Soret, Bacharach, Buvat 07 Schäfers et al 07

12. Martin Burger Molecular Imaging 12 Stuttgart, 27.5.2008 Image reconstruction in PET Stochastic models needed: typically measurements drawn from Poisson model „Image“ u equals density function (uptake) of tracer Linear Operator K equals Radon-transform Possibly additional (Gaussian) measurement noise b

13. Martin Burger Molecular Imaging 13 Stuttgart, 27.5.2008 Data model Image Data Otmar Schober Klaus Schäfers

14. Martin Burger Molecular Imaging 14 Stuttgart, 27.5.2008 Image reconstruction in PET Same model with different K can be used for imaging with photons (microscopy, CCD cameras,..) Typically the Poisson statistic is good (many photon counts), measurement noise dominates In PET (and modern nanoscopy) the opposite is true !

15. Martin Burger Molecular Imaging 15 Stuttgart, 27.5.2008 Maximum Likelihood / Bayes Reconstruct maximum-likelihood estimate Model of posterior probability (Bayes)

16. Martin Burger Molecular Imaging 16 Stuttgart, 27.5.2008 EM-Algorithm: A fixed point iteration Continuum limit (relative entropy) Optimality condition leads to fixed point equation

17. Martin Burger Molecular Imaging 17 Stuttgart, 27.5.2008 EM-Algorithm: A fixed point iteration Fixed point iteration Convergence analysis for exact data: descent in objective functional in Kullback-Leibler divergence (relative entropy) between to consecutive iterations (images) Regularizing properties for ill-posed problems not completely clear, partial results Resmerita-Iusem-Engl 07

18. Martin Burger Molecular Imaging 18 Stuttgart, 27.5.2008 PET Reconstruction: Small Animal PET Reconstruction with optimized EM-Algorithm, Good statistics

19. Martin Burger Molecular Imaging 19 Stuttgart, 27.5.2008 EM-Algorithm: A fixed point iteration Fixed point iteration Convergence analysis for exact data: descent in objective functional in Kullback-Leibler divergence (relative entropy) between to consecutive iterations (images) Regularizing properties for ill-posed problems not completely clear, partial results Resmerita-Iusem-Engl 07 u k + 1 = u k K ¤ 1 K ¤ ( f K u k )

20. Martin Burger Molecular Imaging 20 Stuttgart, 27.5.2008 EM-Algorithm at the limit Bad statistics arising due to lower radioactive activity or isotopes decaying fast (e.g. H 2 O 15 ) Desireable for patients Desireable for certain quantitative investigations (H 2 O 15 is useful tracer for blood flow) ~10.000 Events ~600 Events

21. Martin Burger Molecular Imaging 21 Stuttgart, 27.5.2008 PET at the resolution limit How can we get reasonable answers in the case of bad data ? Need additional (a-priori) information about: - known structures in the image - desired structures to be investigated - dynamics (4D imaging)

22. Martin Burger Molecular Imaging 22 Stuttgart, 27.5.2008 Back to Bayes EM algorithm uses uniform prior probability distribution, any image explains data is considered of equal relevance Prior probability can be related to regularization functional (such as energy in statistical mechanics) Same analysis yields regularized log-likelihood P ( u ) » e ¡ R ( u ) Z [ K u ¡ fl og ( K u )] + ® R ( u )

23. Martin Burger Molecular Imaging 23 Stuttgart, 27.5.2008 Minimization of penalized log-likelihood Minimization of subject to nonnegativity is a difficult task Combines nonlocal part (including K ) with local regularization functional (typically dependent on u and 5u ) Ideally ingredients of EM-step should be used (Implementations of K and K* including several corrections) Z [ K u ¡ fl og ( K u )] + ® R ( u )

24. Martin Burger Molecular Imaging 24 Stuttgart, 27.5.2008 Minimization of penalized log-likelihood Assume K is convex, but not necessarily differentiable Optimality condition for a positive solution For simplicity assume K*1 = 1 in the following (standard operator scaling) K ¤ 1 ¡ K ¤ ( f K u ) + ®p = 0 ; p 2 @ R ( u )

25. Martin Burger Molecular Imaging 25 Stuttgart, 27.5.2008 Minimization of penalized log-likelihood Simplest idea: gradient-type method Not robust if J nonsmooth, possibly extreme damping needed for gradient-dependent J Better: evaluate nonlocal term at last iterate and subgradient at new iteration No preservation of positivity (even with damping) 1 ¡ K ¤ ( f K u k ) + ®p k + 1 = 0 ; p k + 1 2 @ R ( u k + 1 )

26. Martin Burger Molecular Imaging 26 Stuttgart, 27.5.2008 Minimization of penalized log-likelihood Improved: approximate also first term Realized via two-step method u k + 1 u k ¡ K ¤ ( f K u k ) + ®p k + 1 = 0 ; p k + 1 2 @ R ( u k + 1 ) u k + 1 = 2 = u k K ¤ ( f K u k ) ¡ u k + 1 u k + 1 = 2 u k + ®p k + 1 = 0 ; p k + 1 2 @ R ( u k + 1 )

27. Martin Burger Molecular Imaging 27 Stuttgart, 27.5.2008 Minimization of penalized log-likelihood Assume K is convex, but not necessarily differentiable Optimality condition for a positive solution subject to nonnegativity of u K ¤ 1 ¡ K ¤ ( f K u ) + ®p = 0 ; p 2 @ R ( u ) u k + 1 = 2 = u k K ¤ 1 K ¤ ( f K u k )

28. Martin Burger Molecular Imaging 28 Stuttgart, 27.5.2008 Hybrid Imaging: PET-CT (PET-MR) Hybrid imaging becomes increasingly popular. Combine - Anatomical information (CT or MR) - Functional information (PET) Anatomical information yields a-priori knowledge about structures, e.g. exact tumor location and size Soret, Bacharach, Buvat 07

29. Martin Burger Molecular Imaging 29 Stuttgart, 27.5.2008 Regularization and Constraints Anatomical priors (CT images) can be incorporated into the reconstruction process as constraints or as regularization: - constraints: uptake equals zero in certain tissues - regularization: penalization of (high) uptake in certain tissues Both cases can be unified into a penalization functional of the form with P possibly infinite in the constrained case R ( u ) = Z ­ P ( x ; u ( x )) d x

30. Martin Burger Molecular Imaging 30 Stuttgart, 27.5.2008 TV-Methods Penalization of total Variation Formal Exact ROF-Model for denoising g : minimize total variation subject to Rudin-Osher-Fatemi 89,92

31. Martin Burger Molecular Imaging 31 Stuttgart, 27.5.2008 Why TV-Methods ? Cartooning Linear Filter TV-Method

32. Martin Burger Molecular Imaging 32 Stuttgart, 27.5.2008 Why TV-Methods ? Cartooning ROF Model with increasing allowed variance

33. Martin Burger Molecular Imaging 33 Stuttgart, 27.5.2008 TV-Methods There exists Lagrange Parameter, such that ROF is equivalent to Optimality condition Compare with ¡ u k + 1 u k + 1 = 2 u k + ®p k + 1 = 0 ; p k + 1 2 @ J ( u k + 1 ) ¡ ug + ®p = 0 ; p 2 @ J ( u ) ; ® = 1 ¸

34. Martin Burger Molecular Imaging 34 Stuttgart, 27.5.2008 EM-TV Methods EM-step TV-correction step by minimizing in order to obtain next iterate ¡ u k + 1 = 2 = u k K ¤ ( f K u k ) 1 2 Z ­ ( uu k + 1 = 2 ) 2 u k + ® J ( u ) ! m i n u 2 BV

35. Martin Burger Molecular Imaging 35 Stuttgart, 27.5.2008 Damped EM-TV Methods EM-step Damped TV-correction step by minimizing in order to obtain next iterate ¡ u k + 1 = 2 = u k K ¤ ( f K u k ) ¿ 2 Z ­ ( uu k ) 2 u k + 1 2 Z ­ ( uu k + 1 = 2 ) 2 u k + ® J ( u ) ! m i n u 2 BV ¡

36. Martin Burger Molecular Imaging 36 Stuttgart, 27.5.2008 EM-TV: Analysis - Iterates well-defined in BV (existence, uniqueness) - Preservation of positivity (as usual for EM-step, maximum principle for TV minimization) - Descent of the objective functional with damping (yields uniform bound in BV and hence stability) Remaining issue: - Second derivative of logarithmic likelihood term is not uniformly bounded in general (related to lower bound for density)

37. Martin Burger Molecular Imaging 37 Stuttgart, 27.5.2008 Computational Issues in TV-minimization - Regularization term not differentiable, not strictly convex - Degenerate differential operator - No strong convergence in BV - Discontinuous solutions expected - Large data sets (3D / 4D Imaging, future 4 D / 5D with different regularization in dimensions > 3 ) Solution approaches: - Dual or primal-dual schemes - Parallel implementations based on dual domain decomposition

38. Martin Burger Molecular Imaging 38 Stuttgart, 27.5.2008 Primal dual discretization Use characterization of subgradients as elements of the convex set Optimality condition for ROF can be reformulated as a primal- dual (or dual) variational inequality

39. Martin Burger Molecular Imaging 39 Stuttgart, 27.5.2008 Primal dual discretization Discretize variational inequality by finite elements, usually on square / cubical elements - piecewise constant for u (discontinuous anyway) - Raviart-Thomas for p (stability) Or higher-order alternatives

40. Martin Burger Molecular Imaging 40 Stuttgart, 27.5.2008 Error estimation Error estimates need appropriate distance measure, generalized Bregman-distance mb-Osher 04 mb 08 DFG funding, „Regularisierung mit Singulären Energien“, 2008-2011

41. Martin Burger Molecular Imaging 41 Stuttgart, 27.5.2008 Parallel Implementations Diploma thesis Jahn Müller, jointly supervised with Sergej Gorlatch (Computer Science, WWU)

42. Martin Burger Molecular Imaging 42 Stuttgart, 27.5.2008 ~600 Events EM EM-TV

43. Martin Burger Molecular Imaging 43 Stuttgart, 27.5.2008 EM-TV reconstruction from synthetic data Bild Daten EM EM-TV

44. Martin Burger Molecular Imaging 44 Stuttgart, 27.5.2008 H 2 O 15 Data – Right Ventricular EM EM-Gauss EM-TV

45. Martin Burger Molecular Imaging 45 Stuttgart, 27.5.2008 H 2 O 15 Data – Left Ventricular EM EM-Gauss EM-TV

46. Martin Burger Molecular Imaging 46 Stuttgart, 27.5.2008 Quantification Results can be used as input for quantification Standard approach: Rough region partition in PET images Computation of mean physiological parameters (e.g. perfusion) in each region (parameter fit in ordinary differential equations Needs 4D PET reconstructions DFG-Funding SFB 656, Subproject PM6 (mb/Klaus Schäfers)

47. Martin Burger Molecular Imaging 47 Stuttgart, 27.5.2008 Quantification Remaining problem: systematic error for TV-Methods Variation reduced too strongly, quantitative Values can differ in particular in small structures Problems in quantitative evaluations

48. Martin Burger Molecular Imaging 48 Stuttgart, 27.5.2008 Quantitative PET Contrast correction via iterative Regularization Prior probability centered at zero Adaptation: maximum likelihood estimater of Poisson-TV model. Second step with shifted prior probability Iterative algorithm, EM-TV can be used for each substep Osher-mb-Goldfarb-Xu-Yin 05, mb-Gilboa-Osher-Xu 06 p ( u ) » exp ( ¡ J ( u )) ^ u 1 p ( u ) » exp ( ¡ [ J ( u ) + J ( ^ u 1 ) + h ^ p 1 ; u ¡ ^ u 1 i])

49. Martin Burger Molecular Imaging 49 Stuttgart, 27.5.2008 Quantitative PET Contrast correction via iterative Regularization Significant improvement of quantitative densities Often not visible in images Directly visible for small structures, e.g. for analogous problem in nanoscopy (4-Pi, STED) Operator K is a convolution

50. Martin Burger Molecular Imaging 50 Stuttgart, 27.5.2008 Nanoscopy – STED & 4Pi Stimulated Emission Depletion (Stefan Hell, MPI Göttingen) BMBF funded, „INVERS“, Göttingen(MPI+Univ)-Münster-Bochum, Leica

51. Martin Burger Molecular Imaging 51 Stuttgart, 27.5.2008 Nanoscopy at the limit Syntaxin PC 12, 53 nm EM EM-TV Iterated EM-TV Christoph Brune

52. Martin Burger Molecular Imaging 52 Stuttgart, 27.5.2008 3D Cell Structure Christoph Brune EM-TVIterated EM-TV

53. Martin Burger Molecular Imaging 53 Stuttgart, 27.5.2008 Quantification of Blood Flow Current quantification with radioactive water has limited resolution, due to poor quality of reconstructions at each time step

54. Martin Burger Molecular Imaging 54 Stuttgart, 27.5.2008 Outlook: Imaging of Physiological Quantities Instead of reconstruction images with bad statistics, use direct model based inversion from 4D data Schematic data model: Physiological Activation Images PET data quantities,3D+1D curves, 4D 4D 4D F, r, C A C T u f Nonlinear physiological models, lead to nonlinear inverse problems !!!

55. Martin Burger Molecular Imaging 55 Stuttgart, 27.5.2008 Myocardial Blood Flow Two-compartment model: computation of flow into tissue C T from arterial blood flow C A aus Nonlinearity in the ODE, exponential dependence of C T on F @ C T ( x ; t ) @ t = F ( x )( C A ( t ) ¡ C T ( x ; t ) V D )

56. Martin Burger Molecular Imaging 56 Stuttgart, 27.5.2008 Myocardial Blood Flow Left ventricular image computed from the ODE solution via Indicator functions obtained from segmentation (EM-TV). Corrections by spillover terms s 1, s 2

57. Martin Burger Molecular Imaging 57 Stuttgart, 27.5.2008 Quantification of Myocardial Blood Flow Solve nonlinear inverse problem again by two-step procedure, i.e. EM alternated with parameter identification in coupled ODEs A-priori knowledge on parameters in regularization and constraints

58. Martin Burger Molecular Imaging 58 Stuttgart, 27.5.2008 Quantification of Myocardial Blood Flow Sequence of reconstructed images by EM method (3 D reconstruction in each time frame)

59. Martin Burger Molecular Imaging 59 Stuttgart, 27.5.2008 Quantification of Myocardial Blood Flow Sequence of reconstructed images based on blood flow model (4 D reconstruction)

60. Martin Burger Molecular Imaging 60 Stuttgart, 27.5.2008 Quantification of Myocardial Blood Flow Sequence of reconstructed images based on blood flow model (4 D reconstruction)