1 Stephen Pizer, Sarang Joshi, Guido Gerig Medical Image Display & Analysis Group (MIDAG) University of North Carolina, USA with credit to P. T. Fletcher, C. Lu, M. Styner, A. Thall, P. Yushkevich And others in MIDAG This set of slides can be found at the website midag.cs.unc.edu/pubs/presentations/SPIE_tut.htm Tutorial:Anatomic Object Ensemble Representations for Segmentation & Statistical Characterization Tutorial: Anatomic Object Ensemble Representations for Segmentation & Statistical Characterization 17 February 2003

2 Segmentation ä Objective: Extract the most probable target object geometric conformation z given the image data I ä Requires prior on object geometry p(z) ä Requires a measure of match p(I|z) of the image to a particular object conformation, so the image must be represented in reference to the object geometric conformation

3 Statistical Geometric Characterization ä Requires priors p(class) and likelihoods p(z|class) ä Uses ä Medical science: determine geometric ways in which pathological and normal classes differ ä Diagnostic: determine if a particular patient’s geometry is in the pathological or the healthy class ä Educational: communicate anatomic variability in atlases ä Priors p( z ) for segmentation ä Monte Carlo generation of images

4 Part I Multiscale Geometric Primitives, Especially M-reps Multiscale Deformable Model Segmentation Stephen Pizer Tutorial: Anatomic Object Ensemble Representations for Segmentation & Statistical Characterization 17 February 2003

5 ä Relation of this object instance to other instances ä Representing the real world ä Basic entities: object ensembles & single objects ä Deformation while staying in statistical entity class ä Discrimination by shape class and by locality ä Mechanical deformation within a patient: interior primitive ä Relation to Euclidean space/projective Euclidean space ä Matching image data ä Multiple object-oriented scale levels ä Yields efficiency in segmentation: coarse to fine ä Yields efficiency in number of training samples for probabilities Object Representation Objectives

6 Object Ensembles & Single Objects ä Object descriptions ä Intuitive, related to anatomic understanding ä Mathematically correct ä Object interrelation descriptions ä Abutment and non-interpenetration Large scale Smaller scale

7 Multiple Object-oriented Scale Levels -- For Efficiency ä Scale based parents and neighbors ä Intuitive scale levels Ensemble Object Main figure Subfigure Slab through-sectionBoundary vertex

8 Multiple Object-oriented Scale Levels -- For Efficiency ä Scale based parents and neighbors ä Statistics via Markov random fields [Lu] ä Residue from parent: z k i = i th residue at scale level k ä Difference from neighbors’ prediction ä p(z k i relative to P(z k i ), z k i relative to N(z k i )) ä Efficiency of training from low dimension per probability ä Features with position and level of locality (scale) ä Feature selection [Yushkevich ]

9 Discussion of Scale ä Spatial aspects of a geometric feature ä Position ä Scale: 3 different types ä Spatial extent ä Region summarized ä Level of detail captured ä Residues from larger scales ä Distances to neighbors with which it has a statistical relationship ä Markov random field ä Consider point distribution model, landmarks, spherical harmonics, dense Euclidean positions, m-reps Large scale Smaller scale

10 Scale Situations in Various Statistical Geometric Analysis Approaches Coarse Fine Location Level of Detail Location Global coef for Multidetail featureDetail residues each level of detail Examples: boundary spherical boundary points,m-rep object harmonics, global dense positionhierarchy, principal components displacementswavelets

11  Atlas voxels with a displacement at each voxel:  x(x), label(x) ä Set of distinguished points {x i } with a displacement at each ä Landmarks ä Boundary points in a mesh ä With normal b = (x,n)  Loci of medial atoms: m = (x,F,r,  ) or end atom (x,F,r,  ) (show on Pablo) Object Representations: Atoms   u v t Hub Spoke Spoke 

12 Multiscale Object Representation via Interiors: M-reps ä Interiors (medial) at all but smallest scale levels ä Boundary displacement at smallest scale level ä Allows fixed structure in medial part ä Residues from previous scale level ä At each level recognizes invariances associated with shape ä Provide correspondence ä Across population & Across comparable structures ä Provides prediction by neighbors ä Translation, rotation, magnification ä Structure trained from population [Styner] ä Basis for deformable model segmentation          Continuous vs. sampled rep’ns boundary trad’l medial medial atom

13 M-rep Gives Multiscale Intrinsic Coord’s for Nonspherical & Nontubular Objects ä Here single-figure ä On medial locus ä (u,v) in r-proportional metric ä v along medial curve of medial sheet ä u across medial sheet ä t around crest ä Across narrow object dimension   along medial spokes ä Proportion of medial width r u v   u v t

14 Discrete M-rep Multifigure Objects and Multiobject Ensembles ä Meshes of medial atoms ä Objects connected as host, subfigures ä Hinge atoms of subfigure on boundary of parent figure ä Blend in hinge regions ä Special coordinate system (u,w,t) for blend region ä Multiple such objects, inter- related via neighbor’s figural coords w

15 M-rep Intrinsic Coordinates ä Within figure ä One medial atom provides a coordinate system for its neighbor atoms ä Position, Orientation, Metric ä Between subfigure and figure ä Host atoms’ coordinate systems provides coordinate system for protrusion or indentation hinge ä Between figures or between objects ä One object provides coordinate system for neighbor object boundary u v t

16 Interpolating Boundaries in a Figure ä Interpolate x, r via B-splines [ Yushkevich] ä Trimming curve via r<0 at outside control points ä Avoids corner problems of quadmesh ä Yields continuous boundary ä Via modified subdivision surface [Thall]  Approximate orthogonality at spoke ends ä Interpolated atoms via boundary and distance  At ends elongation  needs also to be interpolated ä Need to use synthetic medial geometry [Damon] Medial sheet Implied boundary

17 Sampled medial shape representation: M-rep tube figures ä Same atoms as for slabs ä r is radius of tube ä spokes are rotated about b ä Chain rather than mesh  b  x n x+ rR b,n (  )b x+rR b,n (-  )b

18 Segmentation by Deformable M-reps w ä For each scale level k, coarse to fine ä For all residues i at scale level k: z k i ä Maximize [log p(z k i relative to P(z k i ), z k i relative to N(z k i )) + log p(Image|{z j i, j>=k, all i})] i.e., maximize geometric typicality + geometry to image match (show on Pablo)

19 Intensity Profiles Template Used in Geometry to Image Match Mean profile image along red meridian line, from training or as analytic function of  /r Inside Outside Left Hippocampus Template to target image correspondence via figural coordinates

20 3-Scale Deformation of M-reps [Pizer, Joshi, Chaney, et al.] Segmentation of Kidney from CT Optimal movement Optimal warp Refined boundary Hand- placed

21 Three Stage - Single Figure Segmentation of Kidney from CT Axial, sagittal, and coronal target image slices Grey curve: before step. White curve: after step Optimal movement Optimal warp Refined boundary

22 Segmentation by Deformable M-reps Controlled Validations w ä Kidneys ä Human segmented ä Robust over all 12 kidney pairs ä Avg distance to human segn’s boundary: <1.7mm ä Clinically acceptable agreement with humans ä Monte Carlo produced ä Robust against initialization ä Other anecdotal validations ä Liver, male pelvis ensemble, caudate, hippocampus

23 For a copy of the slides in this talk see website: midag.cs.unc.edu/pubs/presentations/SPIE_tut.htm For background to this talk see tutorial at website: midag.cs.unc.edu/projects/object-shape/tutorial/index.htm or papers at midag.cs.unc.edu

24 References: Non-M-reps ä Voxel displacements and labels: Grenander, U and M Miller (1998). Computational anatomy: an emerging discipline. Quarterly of Applied Mathematics, 56: Christensen, G, S Joshi, and M Miller (1997). Volumetric transformation of brain anatomy. IEEE Transactions on Medical Imaging, 16(6): ä Landmarks: Dryden, I & K Mardia, (1998). Statistical Shape Analysis. John Wiley and Sons (Chichester). ä Point distribution models: T Cootes, A Hill, CJ Taylor (1994). Use of active shape models for locating structures in medical images. Image & Vision Computing 12: ä Spherical harmonic models: Kelemen, A, G Székely, G Gerig (1999). Elastic model-based segmentation of 3D neuroradiological data sets. IEEE Transactions of Medical Imaging, 18:

25 References: M-reps ä Overview: Pizer, S, G Gerig, S Joshi, S Aylward (2002). Multiscale medial shape-based analysis of image objects. Proc. IEEE, to appear. ä Deformable m-reps segmentation: Pizer, S, et al. (2002). Deformable m-reps for 3D medical image segmentation. Subm. for IJCV special UNC-MIDAG issue. mreps.pdf mreps.pdf  Figural coordinates: Pizer S, et al. (2002). Object models in multiscale intrinsic coordinates via m-reps. Image & Vision Computing special issue on generative model-based vision, to appear. ä Forming m-rep models: Styner, M et al., Statistical shape analysis of neuroanatomical structures based on medial models. Medical Image Analysis, to appear spring

26 References: M-reps  Continuous m-reps: Yushkevich, P et al. (2002). Continuous Medial Representations for Geometric Object Modeling in 2D and 3D. Image & Vision Computing special issue on generative model-based vision, to appear.  Implied boundaries via subdivision surfaces: Thall, A (2002). Fast C 2 interpolating subdivision surfaces using iterative inversion of stationary subdivision rules. UNC Comp. Sci. Tech. Rep. TR  Markov random fields: Lu, C, S Pizer, S Joshi (2003). A Markov Random Field approach to multi-scale shape analysis. Subm. to Scale Space. ä Math of m-reps --> boundaries: Damon, J (2002), Determining the geometry of boundaries of objects from medial data. UNC Math. Dept.