1. MIDAG@UNC Statistics of Anatomic Geometry Stephen Pizer, Kenan Professor Medical Image Display & Analysis Group University of North Carolina This tutorial and other relevant papers can be found at website: midag.cs.unc.edu Faculty: me, Ian Dryden, P. Thomas Fletcher, Xavier Pennec, Sarang Joshi, Carole Twining Stephen Pizer, Kenan Professor Medical Image Display & Analysis Group University of North Carolina This tutorial and other relevant papers can be found at website: midag.cs.unc.edu Faculty: me, Ian Dryden, P. Thomas Fletcher, Xavier Pennec, Sarang Joshi, Carole Twining

2. MIDAG@UNC Geometry of Objects in Populations via representations z ä ä Uses for probability density p(z) ä ä Sampling p(z) to communicate anatomic variability in atlases ä ä Issue: geometric propriety of samples? ä ä Log prior in posterior optimizing deformable model segmentation = registration ä ä Optimize z p(z|I), so log p(z) + log p(I|z)   Or E (z|I) ä ä Uses for probability density p(z) ä ä Sampling p(z) to communicate anatomic variability in atlases ä ä Issue: geometric propriety of samples? ä ä Log prior in posterior optimizing deformable model segmentation = registration ä ä Optimize z p(z|I), so log p(z) + log p(I|z)   Or E (z|I)

3. MIDAG@UNC Geometry of Objects in Populations via representations z ä ä Uses for probability density p(z) ä ä Compare two populations ä ä Medical science ä ä Hypothesis testing with null hypothesis p(z|healthy) = p(z|diseased) ä ä If null hypothesis is not accepted, find localities where probability densities differ and characterization of shape difference ä ä Diagnostic: Is particular patient’s geometry diseased? p(z|healthy, I) vs. p(z|diseased, I) ä ä Uses for probability density p(z) ä ä Compare two populations ä ä Medical science ä ä Hypothesis testing with null hypothesis p(z|healthy) = p(z|diseased) ä ä If null hypothesis is not accepted, find localities where probability densities differ and characterization of shape difference ä ä Diagnostic: Is particular patient’s geometry diseased? p(z|healthy, I) vs. p(z|diseased, I)

4. MIDAG@UNC Needs of Geometric Representation z & Probability Representation p(z) ä p(z) limited samples, i.e., b ä Accurate p(z) estimation with limited samples, i.e., beat High Dimension Low Sample Size (HDLSS: many features, few training cases) ä ä Measure of predictive strength of representation and statistics [Muller]: ä ä where “^” indicates projection onto training data principal space ä ä Primitives’ positional correspondence; cases alignment ä ä Easy fit of z to each training segmentation or image ä p(z) limited samples, i.e., b ä Accurate p(z) estimation with limited samples, i.e., beat High Dimension Low Sample Size (HDLSS: many features, few training cases) ä ä Measure of predictive strength of representation and statistics [Muller]: ä ä where “^” indicates projection onto training data principal space ä ä Primitives’ positional correspondence; cases alignment ä ä Easy fit of z to each training segmentation or image

5. MIDAG@UNC Needs of Geometric Representation z & Probability Representation p(z) ä ä Make significant geometric effects intuitive ä ä Null probabilities for geometrically illegal objects ä ä Localization ä ä Handle multiple objects and interstitial regions ä ä Speed and space ä ä Make significant geometric effects intuitive ä ä Null probabilities for geometrically illegal objects ä ä Localization ä ä Handle multiple objects and interstitial regions ä ä Speed and space

6. MIDAG@UNC Schedule of Tutorial ä ä Object representations (Pizer) ä ä PCA, ICA, hypothesis testing, landmark statistics, object- relative intensity statistics (Dryden) ä ä Statistics on Riemannian manfolds, of m-reps & diffusion tensors, maintaining geometric propriety (Fletcher) ä ä Statistics on Riemannian manfolds: extensions and applications (Pennec) ä ä Statistics on diffeomorphisms, groupwise registration, hypothesis testing on Riemannian manifolds (Joshi) ä ä Information theoretic measures on anatomy, correspondence, ASM, AAM (Twining) ä ä Multi-object statistics & segmentation (Pizer) ä ä Object representations (Pizer) ä ä PCA, ICA, hypothesis testing, landmark statistics, object- relative intensity statistics (Dryden) ä ä Statistics on Riemannian manfolds, of m-reps & diffusion tensors, maintaining geometric propriety (Fletcher) ä ä Statistics on Riemannian manfolds: extensions and applications (Pennec) ä ä Statistics on diffeomorphisms, groupwise registration, hypothesis testing on Riemannian manifolds (Joshi) ä ä Information theoretic measures on anatomy, correspondence, ASM, AAM (Twining) ä ä Multi-object statistics & segmentation (Pizer)

7. MIDAG@UNC Representations z of Deformation ä ä Landmarks ä ä Boundary of objects (b-reps) ä ä Points spaced along boundary ä ä or Coefficients of expansion in basis functions ä ä or Function in 3D with level set as object boundary ä ä Deformation velocity seq. per voxel ä ä Medial representation of objects’ interiors (m-reps) ä ä Landmarks ä ä Boundary of objects (b-reps) ä ä Points spaced along boundary ä ä or Coefficients of expansion in basis functions ä ä or Function in 3D with level set as object boundary ä ä Deformation velocity seq. per voxel ä ä Medial representation of objects’ interiors (m-reps)

8. MIDAG@UNC = (p 1, p 2, …,p N ) Landmarks as Representation z z = (p 1, p 2, …,p N ) ä ä First historically ä ä Kendall, Bookstein, Dryden & Mardia, Joshi ä ä Landmarks defined by special properties ä ä Won’t find many accurately in 3D ä ä Global   Alignment via minimization of inter-case  points distances 2 ä ä First historically ä ä Kendall, Bookstein, Dryden & Mardia, Joshi ä ä Landmarks defined by special properties ä ä Won’t find many accurately in 3D ä ä Global   Alignment via minimization of inter-case  points distances 2

9. MIDAG@UNC B-reps as Representation z ä ä Point samples: z = (p 1, p 2, …,p N ) ä ä Like landmarks; popular ä ä Characterization of local translations of shell ä ä Fit to training objects pretty easy ä ä Handles multi-object complexes ä ä Global ä ä Positional correspondence of primitives ä ä Slow reparametrization optimizing p(z) tightness ä ä Problems with geometrically improper fits ä ä Mesh by adding sample neighbors list ä ä Point, normal samples: z = ([p 1,n 1 ],…,[p N,n N ]) ä ä Easier to avoid geometrically improper fits ä ä Point samples: z = (p 1, p 2, …,p N ) ä ä Like landmarks; popular ä ä Characterization of local translations of shell ä ä Fit to training objects pretty easy ä ä Handles multi-object complexes ä ä Global ä ä Positional correspondence of primitives ä ä Slow reparametrization optimizing p(z) tightness ä ä Problems with geometrically improper fits ä ä Mesh by adding sample neighbors list ä ä Point, normal samples: z = ([p 1,n 1 ],…,[p N,n N ]) ä ä Easier to avoid geometrically improper fits

10. MIDAG@UNC B-reps as Representation z   Basis function coefficients z = (a 1, a 2, …,a M ) with p(u) =  k=1 M a k  k (u) ä ä Achieves geometric propriety ä ä Fitting to data well worked out and programmed ä ä Implicit, questionable positional correspondence ä ä Global, ä ä Unintuitive ä ä Alignment via first ellipsoid   Basis function coefficients z = (a 1, a 2, …,a M ) with p(u) =  k=1 M a k  k (u) ä ä Achieves geometric propriety ä ä Fitting to data well worked out and programmed ä ä Implicit, questionable positional correspondence ä ä Global, ä ä Unintuitive ä ä Alignment via first ellipsoid 7 12 1 Representations via spherical harmonics

11. MIDAG@UNC z = F, an image B-rep via F(x)’s level set: z = F, an image ä ä Allows topological variability ä ä Global ä ä Unintuitive, costly in space ä ä Fit to training cases easy: F = signed distance to boundary ä ä Modification by geometry limited diffusion ä ä Requires nonlinear statistics: not yet well developed ä ä Serious problems of geometric propriety if stats on F; needs stats on PDE for nonlinear diffusion ä ä Correspondence? ä ä Localization: via spatially varying PDE parameters?? ä ä Allows topological variability ä ä Global ä ä Unintuitive, costly in space ä ä Fit to training cases easy: F = signed distance to boundary ä ä Modification by geometry limited diffusion ä ä Requires nonlinear statistics: not yet well developed ä ä Serious problems of geometric propriety if stats on F; needs stats on PDE for nonlinear diffusion ä ä Correspondence? ä ä Localization: via spatially varying PDE parameters?? Topology change

12. MIDAG@UNC as representation z Deformation velocity sequence for each voxel as representation z ä ä z = ([v 1 (i.j), v 2 (i.j),…,v T (i.j)], (i.j)  pixels) ä ä Miller, Christensen, Joshi ä ä Labels in reference move with deformation ä ä Series of local interactions ä ä Deformation energy minimization ä ä Fluid flow; pretty slow ä ä Costly in space ä ä Slow and unsure to fit to training cases if change from atlas is large ä ä z = ([v 1 (i.j), v 2 (i.j),…,v T (i.j)], (i.j)  pixels) ä ä Miller, Christensen, Joshi ä ä Labels in reference move with deformation ä ä Series of local interactions ä ä Deformation energy minimization ä ä Fluid flow; pretty slow ä ä Costly in space ä ä Slow and unsure to fit to training cases if change from atlas is large

13. MIDAG@UNC M-reps as Representation z Represent the Egg, not the Eggshell ä ä The eggshell: object boundary primitives ä ä The egg: m-reps: object interior primitives ä ä Poor for object that is tube, slab mix ä ä Handles multifigure objects and multi- object complexes ä ä Interstitial space?? ä ä The eggshell: object boundary primitives ä ä The egg: m-reps: object interior primitives ä ä Poor for object that is tube, slab mix ä ä Handles multifigure objects and multi- object complexes ä ä Interstitial space??

14. MIDAG@UNC A deformable model of the object interior: the m-rep ä Object interior primitives: medial atoms ä Local displacement, bending/twisting, swelling: intuitive ä Neighbor geometry ä Objects, figures, atoms, voxels ä Object-relative coordinates ä Geometric impropriety: math check ä Object interior primitives: medial atoms ä Local displacement, bending/twisting, swelling: intuitive ä Neighbor geometry ä Objects, figures, atoms, voxels ä Object-relative coordinates ä Geometric impropriety: math check

15. MIDAG@UNC Medial atom as a nonlinear geometric transformation ä Medial atoms carry position, width, 2 orientations ä Local deformation T   3 ×  + × S 2 × S 2 ( ×  + for edge atoms) ä From reference atom ä Hub translation × Spoke magnification in common × Spoke 1 rotation × Spoke 2 rotation (× crest sharpness) ä M-rep is n-tuple of medial atoms ä T n, n local T’s, a curved, symmetric space ä Geodesic distance between atoms ä Nonlinear statistics are required ä Medial atoms carry position, width, 2 orientations ä Local deformation T   3 ×  + × S 2 × S 2 ( ×  + for edge atoms) ä From reference atom ä Hub translation × Spoke magnification in common × Spoke 1 rotation × Spoke 2 rotation (× crest sharpness) ä M-rep is n-tuple of medial atoms ä T n, n local T’s, a curved, symmetric space ä Geodesic distance between atoms ä Nonlinear statistics are required medial atom edge medial atom

16. MIDAG@UNC Fitting m-reps into training binaries ä ä Optimization penalties ä ä Distance between m-rep and binary image boundaries ä ä Irregularity penalty: deviation of each atom from geodesic average of its neighbors ä ä Yields correspondence(?) ä ä Avoids geometric impropriety(?) ä ä Interpenetration avoidance   Alignment via minimization of inter-case  atoms geodesic distances 2 ä ä Optimization penalties ä ä Distance between m-rep and binary image boundaries ä ä Irregularity penalty: deviation of each atom from geodesic average of its neighbors ä ä Yields correspondence(?) ä ä Avoids geometric impropriety(?) ä ä Interpenetration avoidance   Alignment via minimization of inter-case  atoms geodesic distances 2

17. MIDAG@UNC Schedule of Tutorial ä ä Object representations (Pizer) ä ä PCA, ICA, hypothesis testing, landmark statistics, object- relative intensity statistics (Dryden) ä ä Statistics on Riemannian manfolds, of m-reps & diffusion tensors, maintaining geometric propriety (Fletcher) ä ä Statistics on Riemannian manfolds: extensions and applications (Pennec) ä ä Statistics on diffeomorphisms, groupwise registration, hypothesis testing on Riemannian manifolds (Joshi) ä ä Information theoretic measures on anatomy, correspondence, ASM, AAM (Twining) ä ä Multi-object statistics & segmentation (Pizer) ä ä Object representations (Pizer) ä ä PCA, ICA, hypothesis testing, landmark statistics, object- relative intensity statistics (Dryden) ä ä Statistics on Riemannian manfolds, of m-reps & diffusion tensors, maintaining geometric propriety (Fletcher) ä ä Statistics on Riemannian manfolds: extensions and applications (Pennec) ä ä Statistics on diffeomorphisms, groupwise registration, hypothesis testing on Riemannian manifolds (Joshi) ä ä Information theoretic measures on anatomy, correspondence, ASM, AAM (Twining) ä ä Multi-object statistics & segmentation (Pizer)

18. MIDAG@UNC Multi-Object Statistics ä Need both ä Object statistics ä Inter-object relation statistics ä We choose m-reps because of effectiveness in expressing inter- object geometry ä Medial atoms as transformations of each other ä Relative positions of boundary ä Spokes as normals ä Object-relative coordinates

19. MIDAG@UNC Statistics at Any Scale Level ä Global: z ä By object z 1 k ä Object neighbors N(z 1 k ) ä By figure (atom mesh) z 2 k ä Figure neighbors N(z 2 k ) ä By atom (interior section) z 3 k ä Atom neighbors N(z 3 k ) ä By voxel or boundary vertex ä Voxel neighbors N(z 4 k ) ä Designed for HDLSS quad-mesh neighbor relations atom level voxel level

20. MIDAG@UNC Multiscale models of spatial parcelations ä Finer parcellation z j as j increases (scale decreases) ä Fuzzy edged apertures z j k, with fuzz (tolerance) decreasing as j increases ä Geometric representation z j k ä We use m-reps to represent objects at moderate scale and diffeomorphisms to modify that representation at small scale ä Level sets of pseudo-distance functions can represent the variable topology interstitial regions ä ä Provides localization ä Finer parcellation z j as j increases (scale decreases) ä Fuzzy edged apertures z j k, with fuzz (tolerance) decreasing as j increases ä Geometric representation z j k ä We use m-reps to represent objects at moderate scale and diffeomorphisms to modify that representation at small scale ä Level sets of pseudo-distance functions can represent the variable topology interstitial regions ä ä Provides localization

21. MIDAG@UNC Statistics of each entity in relation to its neighbors at its scale level  Focus on estimating p(z j k, {z j n : n  k}), via probabilities that reflect both inter- object (region) geometric relationship and object themselves (also for figures)  Markov random field  Conditional probabilities p(z j k | {z j n : n  k}) = p(z j k | {z j n :  N(z j k )})  Iterative Conditional Modes – convergence to joint mode of p(z j k, {z j n : n  k} | Image)  Focus on estimating p(z j k, {z j n : n  k}), via probabilities that reflect both inter- object (region) geometric relationship and object themselves (also for figures)  Markov random field  Conditional probabilities p(z j k | {z j n : n  k}) = p(z j k | {z j n :  N(z j k )})  Iterative Conditional Modes – convergence to joint mode of p(z j k, {z j n : n  k} | Image)

22. MIDAG@UNC Representation of multiple objects via residues from neighbor prediction  Inter-entity and inter-scale relation by removal of conditional mean of entity on prediction of its neighbors, then probability density on residue  p(z j k | {z j n :  N(z j k )}) = p(z j k  interpoland z j k : from N(z j k )}) z j k ä Restriction of z j k to its shape space ä Early coarse-to-fine posterior optimization segmentation results success- ful, but still under study ä Alternative to be explored ä Canonical correlation  Inter-entity and inter-scale relation by removal of conditional mean of entity on prediction of its neighbors, then probability density on residue  p(z j k | {z j n :  N(z j k )}) = p(z j k  interpoland z j k : from N(z j k )}) z j k ä Restriction of z j k to its shape space ä Early coarse-to-fine posterior optimization segmentation results success- ful, but still under study ä Alternative to be explored ä Canonical correlation

23. MIDAG@UNC Want more info? ä ä This tutorial, many papers on b-reps, m-reps, diffeomorphism-reps and their statistics and applications can be found at website http://midag.cs.unc.edu 12

24. MIDAG@UNC