Download presentation
Presentation is loading. Please wait.
1
MIDAG@UNC The Uses of Object Shape from Images in Medicine Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina Credits: Many on MIDAG, especially Daniel Fritsch, Guido Gerig, Edward Chaney, Elizabeth Bullitt, Stephen Aylward, George Stetten, Gregg Tracton, Tom Fletcher, Andrew Thall, Paul Yushkevich, Nikki Levine, Greg Clary, David Chen Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina Credits: Many on MIDAG, especially Daniel Fritsch, Guido Gerig, Edward Chaney, Elizabeth Bullitt, Stephen Aylward, George Stetten, Gregg Tracton, Tom Fletcher, Andrew Thall, Paul Yushkevich, Nikki Levine, Greg Clary, David Chen
2
MIDAG@UNC Object Representation in Medical Image Analysis ä Extract an object from image(s) [segmentation] ä Radiotherapy ä Tumor; plan to hit it ä Radiosensitive normal anatomy; plan to miss it plan to miss it ä Surgery ä Plan to remove it ä Plan to miss it ä During surgery, view where it is & effect of treatment & effect of treatment ä Radiology ä View it to judge its pathology ä Extract an object from image(s) [segmentation] ä Radiotherapy ä Tumor; plan to hit it ä Radiosensitive normal anatomy; plan to miss it plan to miss it ä Surgery ä Plan to remove it ä Plan to miss it ä During surgery, view where it is & effect of treatment & effect of treatment ä Radiology ä View it to judge its pathology PD MRAT2 T1Contrast
3
MIDAG@UNC Image Guided Planning of Radiotherapy ä Planning in 3D ä Extracting normal anatomy ä Extracting tumor ä Planning beam poses ä Planning in 3D ä Extracting normal anatomy ä Extracting tumor ä Planning beam poses
4
MIDAG@UNC Object Representation in Medical Image Analysis ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy ä Fuse multimodality images (3D/3D) for planning ä Verify patient placement (3D/2D) ä Surgery ä Fuse multimodality images (3D/3D or 2D) for planning ä Fuse preoperative (3D) & intraoperative (2D) images ä Radiology ä Fuse multimodality images (3D/3D) for diagnosis ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy ä Fuse multimodality images (3D/3D) for planning ä Verify patient placement (3D/2D) ä Surgery ä Fuse multimodality images (3D/3D or 2D) for planning ä Fuse preoperative (3D) & intraoperative (2D) images ä Radiology ä Fuse multimodality images (3D/3D) for diagnosis
5
MIDAG@UNC Object Representation in Medical Image Analysis ä Shape & Volume Measurement ä Make physical measurement ä Radiotherapy ä Measure effect of therapy on tumor ä Radiology, Neurosciences ä Use measurement in science of object development ä Find how probable an object is ä Radiology, Neurosciences ä Use measurement as quantitative input to diagnosis ä Use measurement in science of object development ä Use as prior in object extraction ä E.g., extract the kidney shaped object ä Shape & Volume Measurement ä Make physical measurement ä Radiotherapy ä Measure effect of therapy on tumor ä Radiology, Neurosciences ä Use measurement in science of object development ä Find how probable an object is ä Radiology, Neurosciences ä Use measurement as quantitative input to diagnosis ä Use measurement in science of object development ä Use as prior in object extraction ä E.g., extract the kidney shaped object
6
MIDAG@UNC Object Shape & Volume Measurement: Infant Ventricle from 3D U/S (Gerig, Gilmore) Neurofibromatosis (Gerig, Greenwood)
7
MIDAG@UNC Object Extraction (Segmentation) ä Approach 1: preanalyze, then fit to model ä Neurosurgery (MR Angiogram), Radiology (CT) ä Vessels, ribs, bronchi, bowel via tube skeletons ä Cardiology (3D Ultrasound) ä Geometry via clouds of medial atoms ä Fit appropriately labeled clouds to 3D LV model ä Cardiac Nuclear Medicine (2D Gated Blood Pool Cine) ä Extract LV, with previous frame providing model ä Extraction via deformable m-rep model ä Shape from extracted LV; analyze shape series ä Surgery, Radiation Oncology (Multimodality MRI) ä Extract tumor, using local shape characteristics ä Approach 1: preanalyze, then fit to model ä Neurosurgery (MR Angiogram), Radiology (CT) ä Vessels, ribs, bronchi, bowel via tube skeletons ä Cardiology (3D Ultrasound) ä Geometry via clouds of medial atoms ä Fit appropriately labeled clouds to 3D LV model ä Cardiac Nuclear Medicine (2D Gated Blood Pool Cine) ä Extract LV, with previous frame providing model ä Extraction via deformable m-rep model ä Shape from extracted LV; analyze shape series ä Surgery, Radiation Oncology (Multimodality MRI) ä Extract tumor, using local shape characteristics
8
MIDAG@UNC Extracting Trees of Vessels via Skeletons (Aylward, Bullitt)
9
MIDAG@UNC Presenting Ribs via Tube Skeletons (Aylward)
10
MIDAG@UNC Presenting Bronchi and Lung Vessels via Tube Skeletons (Aylward)
11
MIDAG@UNC Presenting Small Bowel via Tube Skeletons (Aylward)
12
MIDAG@UNC Presenting Blood Vessels Supplying a Tumor for Embolization (Bullitt) Full tree, 2D Subtree, 2D3D, from 2 poses
13
MIDAG@UNC Heart Model (G. Stetten)
14
MIDAG@UNC Statistical Analysis of Medial Atom Clouds (G. Stetten)
15
MIDAG@UNC sphere slabcylinder LV Tube Identified by Medial Atom Statistical Analysis (G. Stetten)
16
MIDAG@UNC sphere slabcylinder Mitral Valve Slab Identified by Medial Atom Statistical Analysis (G. Stetten)
17
MIDAG@UNC Automatic LV Extraction via Mitral Valve/LV Tube Axis (G. Stetten)
18
MIDAG@UNC Gated Blood Pool Cardiac LV Cine Shape Analysis (G. Clary) Example sequence 4-sided medial elliptical analysis
19
MIDAG@UNC Object Extraction (Segmentation) ä Approach 2: deform model to optimize reward for image match + reward for shape normality ä Radiation Oncology (CT or MRI) ä Abdominal, pelvic organs ä Deform m-reps model ä Neurosciences (MRI or 3D Ultrasound) ä Internal brain structures ä Spherical harmonics boundary model ä Deformable m-reps model ä Neurosurgery (CT) ä Vertebrae ä Approach 2: deform model to optimize reward for image match + reward for shape normality ä Radiation Oncology (CT or MRI) ä Abdominal, pelvic organs ä Deform m-reps model ä Neurosciences (MRI or 3D Ultrasound) ä Internal brain structures ä Spherical harmonics boundary model ä Deformable m-reps model ä Neurosurgery (CT) ä Vertebrae
20
MIDAG@UNC M- Reps for Medical Image Object Extraction and Presentation (Chen, Thall)
21
MIDAG@UNC Displacements from Figurally Implied Boundary Boundary implied by figural modelBoundary after displacements
22
MIDAG@UNC Vertebral M-reps Model
23
MIDAG@UNC Vertebral M-reps Model: Spinous Process Figure
24
MIDAG@UNC Cerebral Ventricle M-reps Model
25
MIDAG@UNC Extraction with Object Shape as a Prior Brain structures (Gerig)
26
MIDAG@UNC RegistrationRegistration ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy ä Fuse multimodality images (3D/3D) for planning ä Verify patient placement (3D/2D) ä Surgery ä Fuse multimodality images (3D/3D or 2D) for planning ä Fuse preoperative (3D) & intraoperative (2D) images ä Radiology ä Fuse multimodality images (3D/3D) for diagnosis ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy ä Fuse multimodality images (3D/3D) for planning ä Verify patient placement (3D/2D) ä Surgery ä Fuse multimodality images (3D/3D or 2D) for planning ä Fuse preoperative (3D) & intraoperative (2D) images ä Radiology ä Fuse multimodality images (3D/3D) for diagnosis
27
MIDAG@UNC Image Guided Delivery of Radiotherapy ä Patient placement ä Verification of plan via portal image ä Calculation of new treatment pose ä Patient placement ä Verification of plan via portal image ä Calculation of new treatment pose
28
MIDAG@UNC Finding Treatment Pose from Portal Radiograph and Planning DRR
29
MIDAG@UNC Medial Net Shape Models Medial nets, positions onlyMedial net
30
MIDAG@UNC Image Match Measurment of M-rep
31
MIDAG@UNC Registration Using Lung Medial Object Model : Reference Radiograph (Levine) Medial nets, positions onlyMedial net
32
MIDAG@UNC Radiograph/Portal Image Registration (Levine) Intensity Matching Relative to Medial Model Medial net
33
MIDAG@UNC Shape & Volume Measurement ä Find how probable an object is ä Training images; Principal components ä Global vs. global and local ä Correspondence ä Find how probable an object is ä Training images; Principal components ä Global vs. global and local ä Correspondence Hippocampi (Gerig)
34
MIDAG@UNC Modes of Global Deformation Training set: Mode 1: Mode 2: Mode 3: x = x mean + b 1 p 1 x = x mean + b 2 p 2 x = x mean + b 3 p 3
35
MIDAG@UNC Shape & Volume Measurement ä Shape Measurement ä Modes of shape variation across patients ä Measurement = amount of each mode ä Shape Measurement ä Modes of shape variation across patients ä Measurement = amount of each mode Hippocampi (Gerig)
36
MIDAG@UNC Multiscale Medial Model From larger scale medial net, interpolate smaller scale medial net and represent medial displacements From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b.
37
MIDAG@UNC Summary: What shape representation is for in medicine ä Analysis from images ä Extract the “anatomic object”-shaped object ä Register based on the objects ä Diagnose based on shape and volume Medical science via shape Shape and biology Shape-based diagnostic approaches Shape-based therapy planning and delivery approaches ä Analysis from images ä Extract the “anatomic object”-shaped object ä Register based on the objects ä Diagnose based on shape and volume Medical science via shape Shape and biology Shape-based diagnostic approaches Shape-based therapy planning and delivery approaches
38
MIDAG@UNC Shape Sciences ä Medicine ä Biology ä Geometry ä Statistics ä Image Analysis ä Computer Graphics ä Medicine ä Biology ä Geometry ä Statistics ä Image Analysis ä Computer Graphics
39
MIDAG@UNC The End
40
MIDAG@UNC Options for Primitives ä Space: x i for grid elements ä Landmarks: x i described by local geometry ä Boundary: (x i,normal i ) spaced along boundary ä Figural: nets of diatoms sampling figures ä Space: x i for grid elements ä Landmarks: x i described by local geometry ä Boundary: (x i,normal i ) spaced along boundary ä Figural: nets of diatoms sampling figures
41
MIDAG@UNC Figural Models ä Figures: successive medial involution ä Main figure ä Protrusions ä Indentations ä Separate figures ä Hierarchy of figures ä Relative position ä Relative width ä Relative orientation ä Figures: successive medial involution ä Main figure ä Protrusions ä Indentations ä Separate figures ä Hierarchy of figures ä Relative position ä Relative width ä Relative orientation
42
MIDAG@UNC Figural Models with Boundary Deviations ä Hypothesis ä At a global level, a figural model is the most intuitive ä At a local level, boundary deviations are most intuitive ä Hypothesis ä At a global level, a figural model is the most intuitive ä At a local level, boundary deviations are most intuitive
43
MIDAG@UNC Medial Atoms ä Imply boundary segments with tolerance ä Similarity transform equivariant ä Zoom invariance implies width-proportionality of ä tolerance of implied boundary ä boundary curvature distribution ä spacing along net ä interrogation aperture for image ä Imply boundary segments with tolerance ä Similarity transform equivariant ä Zoom invariance implies width-proportionality of ä tolerance of implied boundary ä boundary curvature distribution ä spacing along net ä interrogation aperture for image
44
MIDAG@UNC Need for Special End Primitives ä Represent ä non-blobby objects ä angulated edges, corners, creases ä still allow rounded edges, corners, creases ä allow bent edges ä But ä Avoid infinitely fine medial sampling ä Maintain tangency, symmetry principles ä Represent ä non-blobby objects ä angulated edges, corners, creases ä still allow rounded edges, corners, creases ä allow bent edges ä But ä Avoid infinitely fine medial sampling ä Maintain tangency, symmetry principles
45
MIDAG@UNC Coarse-to-fine representation For each of three levels Figural hierarchy For each figure, net chain, successively smaller tolerance For each net tile, boundary displacement chain For each of three levels Figural hierarchy For each figure, net chain, successively smaller tolerance For each net tile, boundary displacement chain
46
MIDAG@UNC Multiscale Medial Model From larger scale medial net Coarsely sampled Smooother figurally implied boundary Larger tolerance Interpolate smaller scale medial net Finer sampled More detail in figurally implied boundary Smaller tolerance Represent medial displacements From larger scale medial net Coarsely sampled Smooother figurally implied boundary Larger tolerance Interpolate smaller scale medial net Finer sampled More detail in figurally implied boundary Smaller tolerance Represent medial displacements
47
MIDAG@UNC Multiscale Medial/Boundary Model From medial net Coarsely sampled, smoother implied boundary Larger tolerance Represent boundary displacements along implied normals Finer sampled, more detail in boundary Smaller tolerance From medial net Coarsely sampled, smoother implied boundary Larger tolerance Represent boundary displacements along implied normals Finer sampled, more detail in boundary Smaller tolerance
48
MIDAG@UNC Shape Repres’n in Image Analysis ä Segmentation ä Find the most probable deformed mean model, given the image ä Probability involves ä Probability of the deformed model ä Probability of the image, given the deformed model ä Segmentation ä Find the most probable deformed mean model, given the image ä Probability involves ä Probability of the deformed model ä Probability of the image, given the deformed model
49
MIDAG@UNC Medialness: medial strength of a medial primitive in an image ä Probability of image | deformed model ä Sum of boundariness values ä at implied boundary positions ä in implied normal directions ä with apertures proportional to tolerance ä Boundariness value ä Intensity profile distance from mean (at scale) ä Probability of image | deformed model ä Sum of boundariness values ä at implied boundary positions ä in implied normal directions ä with apertures proportional to tolerance ä Boundariness value ä Intensity profile distance from mean (at scale)
50
MIDAG@UNC Shape Rep’n in Image Analysis ä Segmentation ä Find the most probable deformed mean model, given the image ä Registration ä Find the most probable deformation, given the image ä Shape Measurement ä Find how probable a deformed model is ä Segmentation ä Find the most probable deformed mean model, given the image ä Registration ä Find the most probable deformation, given the image ä Shape Measurement ä Find how probable a deformed model is
51
MIDAG@UNC Object Shape Representations for Medicine to Manufacturing Object Shape Representations for Medicine to Manufacturing ä Figural models, at successive levels of tolerance ä Boundary displacements ä Work in progress ä Segmentation and registration tools ä Statistical analysis of object populations ä CAD tools, incl. direct rendering ä … ä Figural models, at successive levels of tolerance ä Boundary displacements ä Work in progress ä Segmentation and registration tools ä Statistical analysis of object populations ä CAD tools, incl. direct rendering ä …
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.