1 Optimal Transport, Conformal Mappings, and Stochastic Methods for Registration and Surface Warping Allen Tannenbaum Georgia Institute of Technology Emory University

2 This lecture is dedicated to Gregory Randall, citizen of the world. Muchas gracias, Gregory (El Goyo).

3 Collaborators: Conformal Mappings: Angenent, Haker, Sapiro, Kikinis, Nain, Zhu Optimal Transport: Haker, Angenent, Kikinis, Zhu Stochastic Algorithms: Ben-Arous, Zeitouni, Unal, Nain

4 Surface Deformations and Flattening  Conformal and Area-Preserving Maps Optical Flow  Gives Parametrization of Surface Registration  Shows Details Hidden in Surface Folds  Path Planning Fly-Throughs  Medical Research Brain, Colon, Bronchial Pathologies Functional MR and Neural Activity  Computer Graphics and Visualization Texture Mapping

5 Mathematical Theory of Surface Mapping  Conformal Mapping: One-one Angle Preserving Fundamental Form  Examples of Conformal Mappings: One-one Holomorphic Functions Spherical Projection  Uniformization Theorem: Existence of Conformal Mappings Uniqueness of Mapping

6 Deriving the Mapping Equation

7 Deriving the Equation-Continued

8 The Mapping Equation

9 Finite Elements-I

10 Finite Elements-II

11 Finite Elements-III

12 Finite Elements-IV

13 Finite Elements-V

14 Finite Elements-VI

15 Summary of Flattening

16 Cortical Surface Flattening-Normal Brain

17 White Matter Segmentation and Flattening

18 Conformal Mapping of Neonate Cortex

19 Coordinate System on Cortical Surface

20 Principal Lines of Curvature on Brain Surface-I

21 Principal Lines of Curvatures on the Brain-II

22 Flattening Other Structures

23 Bladder Flattening

24 3D Ultrasound Cardiac Heart Map

25 High Intelligence=Bad Digestion Low Intelligence=Good Digestion Basic Principle

26 Flattening a Tube

27 Flattening a Tube-Continued

28 Flattening Without Distortion-I

29 Flattening Without Distortion-II

30 Flattening Without Distortion-III

31 Introduction: Colon Cancer  US: 3rd most common diagnosed cancer  US: 3rd most frequent cause of death  US: deaths every year  Most of the colorectal cancers arise from preexistent adenomatous polyps  Landis S, Murray T, Bolden S, Wingo Ph.Cancer Statitics Ca Cancer J Clin. 1999; 49:8-31.

32 Problems of CT Colonography  Proper preparation of bowel  How to ensure complete inspection  Nondistorting colon flattening program

33 Nondistorting colon flattening  Simulating pathologist’ approach  No Navigation is needed  Entire surface is visualized

34 Nondistorting Colon Flattening  Using CT colonography data  Standard protocol for CT colonography  Twenty-Six patients (17 m, 9 f)  Mean age 70.2 years (from 50 to 82)

35 Flattened Colon

36 Polyps Rendering

37 Finding Polyps on Original Images

38 Polyp Highlighted

39 Path-Planning Deluxe

40 Coronary Vessels-Rendering

41 Coronary Vessels: Fly-Through

42 Area-Preserving Flows-I

43 Area-Preserving Flows-II  The basic idea of the proof of the theorem is the contruction of an orientation-preserving automorphism homotopic to the identity.  As a corollary, we get that given M and N any two diffeomorphic surfaces with the same total area, there exists are area-preserving diffeomorphism. This can be constructed explicitly via a PDE.

44 Area-Preserving Flows for the Sphere-I

45 Area-Preserving Flows for the Sphere-II

46 Area-Preserving Flows of Minimal Distortion

47 Registration and Mass Transport Image registration is the process of establishing a common geometric frame of reference from two or more data sets from the same or different imaging modalities taken at different times. Multimodal registration proceeds in several steps. First, each image or data set to be matched should be individually calibrated, corrected from imaging distortions, cleaned from noise and imaging artifacts. Next, a measure of dissimilarity between the data sets must be established, so we can quantify how close an image is from another after transformations are applied to them. Similarity measures include the proximity of redefined landmarks, the distance between contours, the difference between pixel intensity values. One can extract individual features that to be matched in each data set. Once features have been extracted from each image, they must be paired to each other. Then, a the similarity measure between the paired features is formulated can be formulated as an optimization problem. We can use Monge-Kantorovich for the similarity measure in this procedure.

48 Mass Transportation Problems  Original transport problem was proposed by Gaspar Monge in 1781, and asks to move a pile of soil or rubble to an excavation with the least amount of work.  Modern measure-theoretic formulation given by Kantorovich in Problem is therefore known as Monge-Kantorovich Problem (MKP).  Many problems in various fields can be formulated in term of MKP: statistical physics, functional analysis, astrophysics, reliability theory, quality control, meteorology, transportation, econometrics, expert systems, queuing theory, hybrid systems, and nonlinear control.

49 Monge-Kantorovich Mass Transfer Problem-I

50 MK Mass Transfer Problem-II

51 Algorithm for Optimal Transport-I Subdomains with smooth boundaries and positive densities: We consider diffeomorphisms which map one density to the other: We call this the mass preservation (MP) property. We let u be a initial MP mapping.

52 Algorithm for Optimal Transport-II We consider a one-parameter family of MP maps derived as follows: Notice that from the MP property of the mapping s, and definition of the family,

53 Algorithm for Optimal Transport-III We consider a functional of the following form which we infimize with respect to the maps : Taking the first variation :

54 Algorithm for Optimal Transport-IV First Choice: This leads to following system of equations:

55 Algorithm for Optimal Transport-V This equation can be written in the non-local form: At optimality, it is known that where is a function. Notice therefore for an optimal solution, we have that the non-local equation becomes

56 Solution of L2 M-K and Polar Factorization For the L2 Monge-Kantorovich problem, we take This leads to the following “non-local” gradient descent equation: Notice some of the motivation for this approach. We take: The idea is to push the fixed initial u around (considered as a vector field) using the 1-parameter family of MP maps s(x,t), in such a manner as to remove the divergence free part. Thus we get that at optimality

57 Example of Mass Transfer-I We want to map the Lena image to the Tiffany one.

58 Example of Mass Transfer-II The first image is the initial guess at a mapping. The second is the Monge-Kantorovich improved mapping.

59 Morphing the Densities-I

60 Morphing the Densities-II (Brain Sag)

61 Deformation Map Brain deformation sequence. Two 3D MR data sets were used. First is pre-operative, and second during surgery, after craniotomy and opening of the dura. First image shows planar slice while subsequent images show 2D projections of 3D surfaces which constitute path from original slice. Here time t=0, 0.33, 0.67,and 1.0. Arrows indicate areas of greatest deformation.

62 Morphing-II

63 Morphing-III

64 Surface Warping-I M-K allows one to find area-correcting flattening. After conformally flattening surface, define density mu_0 to be determinant of Jacobian of inverse of flattening map, and mu_1 to be constant. MK optimal map is then area-correcting.

65 Surface Warping-II

66

67

68

69

70

71 fMRI and DTI for IGS

72 Data Fusion

73 More Data Fusion

74 Scale in Biological Systems

75 Multiscale / Complex System Modeling (from Kevrekidis) “Textbook” engineering modeling: macroscopic behavior through macroscopic models (e.g. conservation equations augmented by closures) Alternative (and increasingly frequent) modeling situation:  Models at a FINE / ATOMISTIC / STOCHASTIC level  Desired Behavior At a COARSER, Macroscopic Level E.g. Conservation equations, flow, reaction- diffusion, elasticity  Seek a bridge Between Microscopic/Stochastic Simulation And “Traditional, Continuum” Numerical Analysis When closed macroscopic equations are not available in closed form

76 Micro/Macro Models-Scale I

77 Micro/Macro Models-Scale II

78 How to Move Curves and Surfaces  Parameterized Objects: methods dominate control and visual tracking; ideal for filtering and state space techniques.  Level Sets: implicitly defined curves and surfaces. Several compromises; narrow banding, fast marching.  Minimize Directly Energy Functional: conjugate gradient on triangulated surface (Ken Brakke); dominates medical imaging.

79 Diffusions  Explains a wide range of physical phenomena Heat flow Diffusive transport: flow of fluids (i.e., water, air)  Modeling diffusion is important At macroscopic scale by a partial differential equation (PDE) At microscopic scale, as a collection of particles undergoing random walks  We are interested in replacing PDE by the associated microscopic system

80

81 Interacting Particle Systems-I  Spitzer (1970): “New types of random walk models with certain interactions between particles”  Defn: Continuous-time Markov processes on certain spaces of particle configurations  Inspired by systems of independent simple random walks on Z d or Brownian motions on R d  Stochastic hydrodynamics: the study of density profile evolutions for IPS

82 Interacting Particle Systems-II  Exclusion process: a simple interaction, precludes multiple occupancy --a model for diffusion of lattice gas  Voter model: spatial competition --The individual at a site changes opinion at a rate proportional to the number of neighbors who disagree  Contact process: a model for contagion --Infected sites recover at a rate while healthy sites are infected at another rate  Our goal: finding underlying processes of curvature flows

83 Motivations Do not use PDEs IPS already constructed on a discrete lattice (no discretization) Increased robustness towards noise and ability to include noise processes in the given system

84 Construction of IPS-I  S : a set of sites, e.g. S= Z d  W: a phase space for each site, W={0,1}  The state space: X=W S  Process  X  Local dynamics of the system: transition measures c

85 Construction of IPS-II  Connection between the process  and the rate function c:  Connection to the evolution of a profile function:

86 Curvature Driven Flows

87 Euclidean and Affine Flows

88 Euclidean and Affine Flows

89 Gauss-Minkowki Map

90 Parametrization of Convex Curves

91 Evolution of Densities

92 Curve Shortening Flows

93 Main Convergence Result

94 Birth/Death Zero Range Processes-I  S: discrete torus T N, W=N  Particle configuration space: N T N  Markov generator:

95 Birth/Death Zero Range Processes-II  Markov generator:

96 Birth/Death Zero Range Process-III  Markov generator:

97 The Tangential Component is Important

98 Curve Shortening as Semilinear Diffusion-I

99 Curve Shortening as Semilinear Diffusion-II

100 Curve Shortening as Semilinear Diffusion-III

101 Nonconvex Curves

102 Stochastic Interpretation-I

103 Stochastic Interpretation-II

104 Stochastic Interpretation-III

105 Stochastic Curve Shortening

106 Example of Stochastic Segmentation

107 Stochastic Tracking

108 Conclusions  Stochastic Methods are attractive alternative to level sets.  No increase in dimensionality.  Intrinsically discrete.  Robustness to noise.  Combination with other methods, e.g. Bayesian.