Martin Burger Institut für Numerische und Angewandte Mathematik European Institute for Molecular Imaging CeNoS Quantitative Molecular Imaging – A Mathematical Challenge (?)

Martin Burger Molecular Imaging 2 Stuttgart, Mathematical Christoph Brune Alex Sawatzky Frank Wübbeling Thomas Kösters Martin Benning Marzena Franek Christina Stöcker Mary Wolfram (Linz) Thomas Grosser Jahn Müller

Martin Burger Molecular Imaging 3 Stuttgart, Major Cooperation Partners: SFB 656 /EIMI Otmar Schober (Nuclear Medicine) Klaus Schäfers (Medical Physics, EIMI) Florian Büther (EIMI) Funding: Regularization with Singular Energies (DFG), SFB 656 (DFG), European Institute for Molecular Imaging (WWU + SIEMENS Medical Solutions)

Martin Burger Molecular Imaging 4 Stuttgart, Major Cooperation Partners: Nanoscopy Andreas Schönle, Stefan Hell (MPI Göttingen) Thorsten Hohage, Axel Munk (Univ Göttingen) Nico Bissantz (Bochum) Funding: „Verbundprojekt INVERS“ (BMBF )

Martin Burger Molecular Imaging 5 Stuttgart, st Century Imaging Imaging nowadays mainly separates into two steps - Image Reconstruction: computation of an image from (indirectly) measured data – solution of inverse problems - Image Processing: improvement of given images / image sequences – filtering, variational problems Mathematical issues and approaches (as well as communities) are very separated Images are passed on from step 1 to step 2 Is this an optimal approach ?

Martin Burger Molecular Imaging 6 Stuttgart, Image reconstruction and inverse problems Inverse Problems consist in reconstruction of the cause of an observed effect (via a mathematical model relating them) Diagnosis in medicine is a prototypical example (non- invase approaches always need indirect measurements) Crime is another one … "The grand thing is to be able to reason backwards." Arthur Conan Doyle (A study in scarlet)

Martin Burger Molecular Imaging 7 Stuttgart, Medical Imaging: CT Classical image reconstruction example: computerized tomography (CT) Mathematical Problem: Reconstruction of a density function from its line integrals Inversion of the Radon transform cf. Natterer 86, Natterer-Wübbeling 02

Martin Burger Molecular Imaging 8 Stuttgart, Medical Imaging: CT + Low noise level + High spatial resolution + Exact reconstruction + Reasonable Costs - Restricted to few seconds (radiation exposure, 20 mSiewert) - No functional information - Few mathematical challenges left Soret, Bacharach, Buvat 07 Schäfers et al 07

Martin Burger Molecular Imaging 9 Stuttgart, Medical Imaging: MR + Low noise level + High spatial resolution + Reconstruction by Fourier inversion + No radiation exposure + Good contrast in soft matter - Low tracer sensitivity - Limited functional information - Expensive - Few mathematical challenges left Courtesy Carsten Wolters, University Hospital Münster

Martin Burger Molecular Imaging 10 Stuttgart, Molecular Imaging: PET (Human / Small animal) Positron-Emission-Tomography Data: detecting decay events of an radioactive tracer Decay events are random, but their rate is proportional to the tracer uptake (Radon transform with random directions) Imaging of molecular properties

Martin Burger Molecular Imaging 11 Stuttgart, Medical Imaging: PET + High sensitivity + Long time (mins ~ 1 hour, radiation exposure 8-12 mSiewert) + Functional information + Many open mathematical questions - Few anatomical information - High noise level and disturbing effects (damping, scattering, … ) - Low spatial resolution Soret, Bacharach, Buvat 07 Schäfers et al 07

Martin Burger Molecular Imaging 12 Stuttgart, Image reconstruction in PET Stochastic models needed: typically measurements drawn from Poisson model „Image“ u equals density function (uptake) of tracer Linear Operator K equals Radon-transform Possibly additional (Gaussian) measurement noise b

Martin Burger Molecular Imaging 13 Stuttgart, Data model Image Data Otmar Schober Klaus Schäfers

Martin Burger Molecular Imaging 14 Stuttgart, Image reconstruction in PET Same model with different K can be used for imaging with photons (microscopy, CCD cameras,..) Typically the Poisson statistic is good (many photon counts), measurement noise dominates In PET (and modern nanoscopy) the opposite is true !

Martin Burger Molecular Imaging 15 Stuttgart, Maximum Likelihood / Bayes Reconstruct maximum-likelihood estimate Model of posterior probability (Bayes)

Martin Burger Molecular Imaging 16 Stuttgart, EM-Algorithm: A fixed point iteration Continuum limit (relative entropy) Optimality condition leads to fixed point equation

Martin Burger Molecular Imaging 17 Stuttgart, EM-Algorithm: A fixed point iteration Fixed point iteration Convergence analysis for exact data: descent in objective functional in Kullback-Leibler divergence (relative entropy) between to consecutive iterations (images) Regularizing properties for ill-posed problems not completely clear, partial results Resmerita-Iusem-Engl 07

Martin Burger Molecular Imaging 18 Stuttgart, PET Reconstruction: Small Animal PET Reconstruction with optimized EM-Algorithm, Good statistics

Martin Burger Molecular Imaging 19 Stuttgart, EM-Algorithm: A fixed point iteration Fixed point iteration Convergence analysis for exact data: descent in objective functional in Kullback-Leibler divergence (relative entropy) between to consecutive iterations (images) Regularizing properties for ill-posed problems not completely clear, partial results Resmerita-Iusem-Engl 07 u k + 1 = u k K ¤ 1 K ¤ ( f K u k )

Martin Burger Molecular Imaging 20 Stuttgart, EM-Algorithm at the limit Bad statistics arising due to lower radioactive activity or isotopes decaying fast (e.g. H 2 O 15 ) Desireable for patients Desireable for certain quantitative investigations (H 2 O 15 is useful tracer for blood flow) ~ Events ~600 Events

Martin Burger Molecular Imaging 21 Stuttgart, PET at the resolution limit How can we get reasonable answers in the case of bad data ? Need additional (a-priori) information about: - known structures in the image - desired structures to be investigated - dynamics (4D imaging)

Martin Burger Molecular Imaging 22 Stuttgart, Back to Bayes EM algorithm uses uniform prior probability distribution, any image explains data is considered of equal relevance Prior probability can be related to regularization functional (such as energy in statistical mechanics) Same analysis yields regularized log-likelihood P ( u ) » e ¡ R ( u ) Z [ K u ¡ fl og ( K u )] + ® R ( u )

Martin Burger Molecular Imaging 23 Stuttgart, Minimization of penalized log-likelihood Minimization of subject to nonnegativity is a difficult task Combines nonlocal part (including K ) with local regularization functional (typically dependent on u and 5u ) Ideally ingredients of EM-step should be used (Implementations of K and K* including several corrections) Z [ K u ¡ fl og ( K u )] + ® R ( u )

Martin Burger Molecular Imaging 24 Stuttgart, Minimization of penalized log-likelihood Assume K is convex, but not necessarily differentiable Optimality condition for a positive solution For simplicity assume K*1 = 1 in the following (standard operator scaling) K ¤ 1 ¡ K ¤ ( f K u ) + ®p = 0 ; p R ( u )

Martin Burger Molecular Imaging 25 Stuttgart, Minimization of penalized log-likelihood Simplest idea: gradient-type method Not robust if J nonsmooth, possibly extreme damping needed for gradient-dependent J Better: evaluate nonlocal term at last iterate and subgradient at new iteration No preservation of positivity (even with damping) 1 ¡ K ¤ ( f K u k ) + ®p k + 1 = 0 ; p k + 1 R ( u k + 1 )

Martin Burger Molecular Imaging 26 Stuttgart, Minimization of penalized log-likelihood Improved: approximate also first term Realized via two-step method u k + 1 u k ¡ K ¤ ( f K u k ) + ®p k + 1 = 0 ; p k + 1 R ( u k + 1 ) u k + 1 = 2 = u k K ¤ ( f K u k ) ¡ u k + 1 u k + 1 = 2 u k + ®p k + 1 = 0 ; p k + 1 R ( u k + 1 )

Martin Burger Molecular Imaging 27 Stuttgart, Minimization of penalized log-likelihood Assume K is convex, but not necessarily differentiable Optimality condition for a positive solution subject to nonnegativity of u K ¤ 1 ¡ K ¤ ( f K u ) + ®p = 0 ; p R ( u ) u k + 1 = 2 = u k K ¤ 1 K ¤ ( f K u k )

Martin Burger Molecular Imaging 28 Stuttgart, Hybrid Imaging: PET-CT (PET-MR) Hybrid imaging becomes increasingly popular. Combine - Anatomical information (CT or MR) - Functional information (PET) Anatomical information yields a-priori knowledge about structures, e.g. exact tumor location and size Soret, Bacharach, Buvat 07

Martin Burger Molecular Imaging 29 Stuttgart, Regularization and Constraints Anatomical priors (CT images) can be incorporated into the reconstruction process as constraints or as regularization: - constraints: uptake equals zero in certain tissues - regularization: penalization of (high) uptake in certain tissues Both cases can be unified into a penalization functional of the form with P possibly infinite in the constrained case R ( u ) = Z ­ P ( x ; u ( x )) d x

Martin Burger Molecular Imaging 30 Stuttgart, TV-Methods Penalization of total Variation Formal Exact ROF-Model for denoising g : minimize total variation subject to Rudin-Osher-Fatemi 89,92

Martin Burger Molecular Imaging 31 Stuttgart, Why TV-Methods ? Cartooning Linear Filter TV-Method

Martin Burger Molecular Imaging 32 Stuttgart, Why TV-Methods ? Cartooning ROF Model with increasing allowed variance

Martin Burger Molecular Imaging 33 Stuttgart, TV-Methods There exists Lagrange Parameter, such that ROF is equivalent to Optimality condition Compare with ¡ u k + 1 u k + 1 = 2 u k + ®p k + 1 = 0 ; p k + 1 J ( u k + 1 ) ¡ ug + ®p = 0 ; p J ( u ) ; ® = 1 ¸

Martin Burger Molecular Imaging 34 Stuttgart, EM-TV Methods EM-step TV-correction step by minimizing in order to obtain next iterate ¡ u k + 1 = 2 = u k K ¤ ( f K u k ) 1 2 Z ­ ( uu k + 1 = 2 ) 2 u k + ® J ( u ) ! m i n u 2 BV

Martin Burger Molecular Imaging 35 Stuttgart, Damped EM-TV Methods EM-step Damped TV-correction step by minimizing in order to obtain next iterate ¡ u k + 1 = 2 = u k K ¤ ( f K u k ) ¿ 2 Z ­ ( uu k ) 2 u k Z ­ ( uu k + 1 = 2 ) 2 u k + ® J ( u ) ! m i n u 2 BV ¡

Martin Burger Molecular Imaging 36 Stuttgart, EM-TV: Analysis - Iterates well-defined in BV (existence, uniqueness) - Preservation of positivity (as usual for EM-step, maximum principle for TV minimization) - Descent of the objective functional with damping (yields uniform bound in BV and hence stability) Remaining issue: - Second derivative of logarithmic likelihood term is not uniformly bounded in general (related to lower bound for density)

Martin Burger Molecular Imaging 37 Stuttgart, Computational Issues in TV-minimization - Regularization term not differentiable, not strictly convex - Degenerate differential operator - No strong convergence in BV - Discontinuous solutions expected - Large data sets (3D / 4D Imaging, future 4 D / 5D with different regularization in dimensions > 3 ) Solution approaches: - Dual or primal-dual schemes - Parallel implementations based on dual domain decomposition

Martin Burger Molecular Imaging 38 Stuttgart, Primal dual discretization Use characterization of subgradients as elements of the convex set Optimality condition for ROF can be reformulated as a primal- dual (or dual) variational inequality

Martin Burger Molecular Imaging 39 Stuttgart, Primal dual discretization Discretize variational inequality by finite elements, usually on square / cubical elements - piecewise constant for u (discontinuous anyway) - Raviart-Thomas for p (stability) Or higher-order alternatives

Martin Burger Molecular Imaging 40 Stuttgart, Error estimation Error estimates need appropriate distance measure, generalized Bregman-distance mb-Osher 04 mb 08 DFG funding, „Regularisierung mit Singulären Energien“,

Martin Burger Molecular Imaging 41 Stuttgart, Parallel Implementations Diploma thesis Jahn Müller, jointly supervised with Sergej Gorlatch (Computer Science, WWU)

Martin Burger Molecular Imaging 42 Stuttgart, ~600 Events EM EM-TV

Martin Burger Molecular Imaging 43 Stuttgart, EM-TV reconstruction from synthetic data Bild Daten EM EM-TV

Martin Burger Molecular Imaging 44 Stuttgart, H 2 O 15 Data – Right Ventricular EM EM-Gauss EM-TV

Martin Burger Molecular Imaging 45 Stuttgart, H 2 O 15 Data – Left Ventricular EM EM-Gauss EM-TV

Martin Burger Molecular Imaging 46 Stuttgart, Quantification Results can be used as input for quantification Standard approach: Rough region partition in PET images Computation of mean physiological parameters (e.g. perfusion) in each region (parameter fit in ordinary differential equations Needs 4D PET reconstructions DFG-Funding SFB 656, Subproject PM6 (mb/Klaus Schäfers)

Martin Burger Molecular Imaging 47 Stuttgart, Quantification Remaining problem: systematic error for TV-Methods Variation reduced too strongly, quantitative Values can differ in particular in small structures Problems in quantitative evaluations

Martin Burger Molecular Imaging 48 Stuttgart, Quantitative PET Contrast correction via iterative Regularization Prior probability centered at zero Adaptation: maximum likelihood estimater of Poisson-TV model. Second step with shifted prior probability Iterative algorithm, EM-TV can be used for each substep Osher-mb-Goldfarb-Xu-Yin 05, mb-Gilboa-Osher-Xu 06 p ( u ) » exp ( ¡ J ( u )) ^ u 1 p ( u ) » exp ( ¡ [ J ( u ) + J ( ^ u 1 ) + h ^ p 1 ; u ¡ ^ u 1 i])

Martin Burger Molecular Imaging 49 Stuttgart, Quantitative PET Contrast correction via iterative Regularization Significant improvement of quantitative densities Often not visible in images Directly visible for small structures, e.g. for analogous problem in nanoscopy (4-Pi, STED) Operator K is a convolution

Martin Burger Molecular Imaging 50 Stuttgart, Nanoscopy – STED & 4Pi Stimulated Emission Depletion (Stefan Hell, MPI Göttingen) BMBF funded, „INVERS“, Göttingen(MPI+Univ)-Münster-Bochum, Leica

Martin Burger Molecular Imaging 51 Stuttgart, Nanoscopy at the limit Syntaxin PC 12, 53 nm EM EM-TV Iterated EM-TV Christoph Brune

Martin Burger Molecular Imaging 52 Stuttgart, D Cell Structure Christoph Brune EM-TVIterated EM-TV

Martin Burger Molecular Imaging 53 Stuttgart, Quantification of Blood Flow Current quantification with radioactive water has limited resolution, due to poor quality of reconstructions at each time step

Martin Burger Molecular Imaging 54 Stuttgart, Outlook: Imaging of Physiological Quantities Instead of reconstruction images with bad statistics, use direct model based inversion from 4D data Schematic data model: Physiological Activation Images PET data quantities,3D+1D curves, 4D 4D 4D F, r, C A C T u f Nonlinear physiological models, lead to nonlinear inverse problems !!!

Martin Burger Molecular Imaging 55 Stuttgart, Myocardial Blood Flow Two-compartment model: computation of flow into tissue C T from arterial blood flow C A aus Nonlinearity in the ODE, exponential dependence of C T on C T ( x ; t t = F ( x )( C A ( t ) ¡ C T ( x ; t ) V D )

Martin Burger Molecular Imaging 56 Stuttgart, Myocardial Blood Flow Left ventricular image computed from the ODE solution via Indicator functions obtained from segmentation (EM-TV). Corrections by spillover terms s 1, s 2

Martin Burger Molecular Imaging 57 Stuttgart, Quantification of Myocardial Blood Flow Solve nonlinear inverse problem again by two-step procedure, i.e. EM alternated with parameter identification in coupled ODEs A-priori knowledge on parameters in regularization and constraints

Martin Burger Molecular Imaging 58 Stuttgart, Quantification of Myocardial Blood Flow Sequence of reconstructed images by EM method (3 D reconstruction in each time frame)

Martin Burger Molecular Imaging 59 Stuttgart, Quantification of Myocardial Blood Flow Sequence of reconstructed images based on blood flow model (4 D reconstruction)

Martin Burger Molecular Imaging 60 Stuttgart, Quantification of Myocardial Blood Flow Sequence of reconstructed images based on blood flow model (4 D reconstruction)