Presentation is loading. Please wait.

Presentation is loading. Please wait.

Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group."— Presentation transcript:

1 MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina, USA Credits: Many on MIDAG, especially Daniel Fritsch, Andrew Thall, George Stetten, Paul Yushkevich Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina, USA Credits: Many on MIDAG, especially Daniel Fritsch, Andrew Thall, George Stetten, Paul Yushkevich

2 MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis

3 MIDAG@UNC What shape representation is for ä Analysis from images ä Extract the kidney-shaped object ä Register based on the pelvic bone shapes ä Extract shape information w/o model ä Synthesis ä Design the object ä Deform the object, with physical realism  Shape science  Shape and biology  Shape-based diagnosis ä Analysis from images ä Extract the kidney-shaped object ä Register based on the pelvic bone shapes ä Extract shape information w/o model ä Synthesis ä Design the object ä Deform the object, with physical realism  Shape science  Shape and biology  Shape-based diagnosis

4 MIDAG@UNC What shape representation is for ä Analysis from images ä Extract the kidney-shaped object ä Register based on the pelvic bone shapes ä Analysis from images ä Extract the kidney-shaped object ä Register based on the pelvic bone shapes

5 MIDAG@UNC What shape representation is for ä Synthesis ä Design the object ä Deform the object, with physical realism ä Synthesis ä Design the object ä Deform the object, with physical realism

6 MIDAG@UNC What shape representation is for  Shape science  Shape and biology  Shape-based diagnosis  Shape science  Shape and biology  Shape-based diagnosis Brain structures (Gerig)

7 MIDAG@UNC Shape Sciences ä Geometry ä The spatial layout: via primitives ä Landmarks ä Boundary places and orientations ä Medial places, figural sizes and orientations ä Space itself ä Statistics ä The average shape ä Modes of variation from the average ä Computer Graphics ä Image Analysis ä Geometry ä The spatial layout: via primitives ä Landmarks ä Boundary places and orientations ä Medial places, figural sizes and orientations ä Space itself ä Statistics ä The average shape ä Modes of variation from the average ä Computer Graphics ä Image Analysis

8 MIDAG@UNC Options for Primitives ä Space: x i for grid elements ä Landmarks: x i described by local geometry ä Boundary: (x i,normal i ) spaced along boundary ä Figural: nets of diatoms sampling figures ä Space: x i for grid elements ä Landmarks: x i described by local geometry ä Boundary: (x i,normal i ) spaced along boundary ä Figural: nets of diatoms sampling figures

9 MIDAG@UNC Primitives for shape representation: Landmarks  Sets of points of special geometry

10 MIDAG@UNC Primitives for shape representation: Boundaries  Boundary points with normals

11 MIDAG@UNC Object Representation by M-Reps

12 MIDAG@UNC Each M-figure Represented by Net of Medial Primitives

13 MIDAG@UNC Each M-figure Represented by Net of Medial Primitives

14 MIDAG@UNC Figural Models ä Figures: successive medial involution ä Main figure ä Protrusions ä Indentations ä Separate figures ä Hierarchy of figures ä Relative position ä Relative width ä Relative orientation ä Figures: successive medial involution ä Main figure ä Protrusions ä Indentations ä Separate figures ä Hierarchy of figures ä Relative position ä Relative width ä Relative orientation

15 MIDAG@UNC Primitives’ Desired Properties ä Geometry ä Intuitive: simple, global + local ä Efficiently deformable ä Easily extracted or created ä Spatial tolerance inherent ä Statistics ä Unimodality: normally distributed ä Via geometrical, tolerance-sensitive metric ä Geometry ä Intuitive: simple, global + local ä Efficiently deformable ä Easily extracted or created ä Spatial tolerance inherent ä Statistics ä Unimodality: normally distributed ä Via geometrical, tolerance-sensitive metric

16 MIDAG@UNC Figural Models with Boundary Deviations ä Hypothesis ä At a global level, a figural model is the most intuitive ä At a local level, boundary deviations are most intuitive ä Hypothesis ä At a global level, a figural model is the most intuitive ä At a local level, boundary deviations are most intuitive

17 MIDAG@UNC Union and Difference of M-figures

18 MIDAG@UNC Medial Primitives  x, (b,n) frame, r,  (object angle) ä Imply boundary segments with tolerance ä Similarity transform equivariant ä Zoom invariance implies width-proportionality of ä tolerance of implied boundary ä boundary curvature distribution ä spacing along net ä interrogation aperture for image  x, (b,n) frame, r,  (object angle) ä Imply boundary segments with tolerance ä Similarity transform equivariant ä Zoom invariance implies width-proportionality of ä tolerance of implied boundary ä boundary curvature distribution ä spacing along net ä interrogation aperture for image n

19 MIDAG@UNC 3D kidney model extracted from CT Four figure model of the kidneys Red represents indentation figures Four figure model of the kidneys Red represents indentation figures

20 MIDAG@UNC Need for Special End Primitives ä Represent ä non-blobby objects ä angulated edges, corners, creases ä still allow rounded edges, corners, creases ä allow bent edges ä But ä Avoid infinitely fine medial sampling ä Maintain tangency, symmetry principles ä Represent ä non-blobby objects ä angulated edges, corners, creases ä still allow rounded edges, corners, creases ä allow bent edges ä But ä Avoid infinitely fine medial sampling ä Maintain tangency, symmetry principles

21 MIDAG@UNC End Primitives Rounded end primitive in cross-section Corner primitive in cross-section

22 MIDAG@UNC Displacements from Figurally Implied Boundary Boundary implied by figural modelBoundary after displacements

23 MIDAG@UNC Coarse-to-fine representation   For each of three levels   Figural hierarchy   For each figure, net chain, successively smaller tolerance   For each net tile, boundary displacement chain   For each of three levels   Figural hierarchy   For each figure, net chain, successively smaller tolerance   For each net tile, boundary displacement chain

24 MIDAG@UNC Multiscale Medial Model   From larger scale medial net   Coarsely sampled   Smooother figurally implied boundary   Larger tolerance   Interpolate smaller scale medial net   Finer sampled   More detail in figurally implied boundary   Smaller tolerance   Represent medial displacements   From larger scale medial net   Coarsely sampled   Smooother figurally implied boundary   Larger tolerance   Interpolate smaller scale medial net   Finer sampled   More detail in figurally implied boundary   Smaller tolerance   Represent medial displacements

25 MIDAG@UNC Multiscale Medial Model   From larger scale medial net, interpolate smaller scale medial net and represent medial displacements   From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b.

26 MIDAG@UNC Multiscale Medial/Boundary Model   From medial net   Coarsely sampled, smoother implied boundary   Larger tolerance   Represent boundary displacements along implied normals   Finer sampled, more detail in boundary   Smaller tolerance   From medial net   Coarsely sampled, smoother implied boundary   Larger tolerance   Represent boundary displacements along implied normals   Finer sampled, more detail in boundary   Smaller tolerance

27 MIDAG@UNC Shape Rep’n in Image Analysis ä Segmentation ä Extract an object from image ä Registration ä Find geometric transformation that brings two images into alignment ä 3D/3D ä 3D/2D ä Shape Measurement ä Find how probable a shape is ä Segmentation ä Extract an object from image ä Registration ä Find geometric transformation that brings two images into alignment ä 3D/3D ä 3D/2D ä Shape Measurement ä Find how probable a shape is

28 MIDAG@UNC Shape Repres’n in Image Analysis ä Segmentation ä Find the most probable deformed mean model, given the image ä Probability involves ä Probability of the deformed model (prior) ä Probability of the image, given the deformed model (likelihood) ä Segmentation ä Find the most probable deformed mean model, given the image ä Probability involves ä Probability of the deformed model (prior) ä Probability of the image, given the deformed model (likelihood)

29 MIDAG@UNC Probability of a deformed model ä From training set ä via principal components analysis, coarse-to-fine ä -C * Geometric difference from typical shape ä From training set ä via principal components analysis, coarse-to-fine ä -C * Geometric difference from typical shape

30 MIDAG@UNC Medialness: medial strength of a medial primitive in an image ä Probability of image | deformed model ä Sum of boundariness values ä at implied boundary positions ä in implied normal directions ä with apertures proportional to tolerance ä Boundariness value ä Intensity profile distance from mean (at scale) ä statistical, based on training set ä Intensity differences ä via Gaussian derivatives ä Probability of image | deformed model ä Sum of boundariness values ä at implied boundary positions ä in implied normal directions ä with apertures proportional to tolerance ä Boundariness value ä Intensity profile distance from mean (at scale) ä statistical, based on training set ä Intensity differences ä via Gaussian derivatives

31 MIDAG@UNC Figurally implied boundaries and rendering, via 4-figure model

32 MIDAG@UNC 3D DSL Model Deformation Initial Position of Model in Target Image

33 MIDAG@UNC 3D DSL Model Deformation Figural Deformation Iteration 3

34 MIDAG@UNC 3D DSL Model Deformation with interfigural penalties Initial position After optimization

35 MIDAG@UNC Shape Repres’n in Image Analysis ä Registration ä Find the most probable deformation, given the image ä Registration ä Find the most probable deformation, given the image

36 MIDAG@UNC Shape Rep’n in Image Analysis ä Prior-free medial shape analysis ä Cores: height ridges of medialness (Pizer, Fritsch, Morse, Furst) ä Statistical analysis of medial diatoms (Stetten) ä Prior-free medial shape analysis ä Cores: height ridges of medialness (Pizer, Fritsch, Morse, Furst) ä Statistical analysis of medial diatoms (Stetten)

37 MIDAG@UNC Shape Rep’n in Image Analysis ä Cores: height ridges of medialness

38 M I P @ U N C

39 MIDAG@UNC Shape Rep’n in Image Analysis ä Statistical analysis of medial diatoms

40 MIDAG@UNC sphere slabcylinder

41 MIDAG@UNC sphere slabcylinder

42 MIDAG@UNC sphere slabcylinder

43 MIDAG@UNC

44 MIDAG@UNC sphere slabcylinder

45 MIDAG@UNC sphere slabcylinder

46 MIDAG@UNC

47 MIDAG@UNC Shape Rep’n in CAD/CAM Shape Rep’n in CAD/CAM ä Stock figural models ä Deformation tools: large scale ä Coarse-to-fine specification ä Figural connection tools ä Direct rendering, according to display needs ä Stock figural models ä Deformation tools: large scale ä Coarse-to-fine specification ä Figural connection tools ä Direct rendering, according to display needs

48 MIDAG@UNC Deformation in CAD/CAM Deformation in CAD/CAM

49 MIDAG@UNC Shape Rep’n in CAD/CAM Shape Rep’n in CAD/CAM ä Design models for image analysis

50 MIDAG@UNC Medial Object Shape Representations for Image Analysis & Object Synthesis ä Figural models, at successive levels of tolerance ä Boundary displacements ä Work in progress ä Segmentation and registration tools ä Statistical analysis of object populations ä CAD tools, incl. direct rendering ä Connection relative critical manifolds ä … ä Figural models, at successive levels of tolerance ä Boundary displacements ä Work in progress ä Segmentation and registration tools ä Statistical analysis of object populations ä CAD tools, incl. direct rendering ä Connection relative critical manifolds ä …

51 MIDAG@UNC Application: Image guided planning & delivery of radiotherapy ä Planning in 3D ä Extracting normal anatomy ä Extracting tumor ä Planning beam poses ä Patient placement ä Verification of plan via portal image ä Planning in 3D ä Extracting normal anatomy ä Extracting tumor ä Planning beam poses ä Patient placement ä Verification of plan via portal image

52 MIDAG@UNC Finding Treatment Pose from Portal Radiograph and Planning DRR

53 MIDAG@UNC Medial Net Shape Models Medial nets, positions onlyMedial net

54 MIDAG@UNC Integrated Medialness vs. Pose Offset

55 MIDAG@UNC

56 MIDAG@UNC Representing Boundary Displacements ä Along figurally implied boundary normals ä Coarse-to-fine ä Captures along-boundary covariance ä Useful for rendering ä Along figurally implied boundary normals ä Coarse-to-fine ä Captures along-boundary covariance ä Useful for rendering

57 MIDAG@UNC Summing Medialness on Medial Net via Medial Weighting Function

58 MIDAG@UNC CT Slice of Kidneys in Abdomen

59 MIDAG@UNC Object Shape Brain structures (Gerig)

60 MIDAG@UNC Geometric aspects : Transformations  Euclidean:  translation and rotation  Similarity:  translation, rotation, zoom  Affine  Euclidean:  translation and rotation  Similarity:  translation, rotation, zoom  Affine

Download ppt "Medial Object Shape Representations for Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group."

Similar presentations

Face Recognition Sumitha Balasuriya.

Face Recognition Sumitha Balasuriya.
Image Registration  Mapping of Evolution. Registration Goals Assume the correspondences are known Find such f() and g() such that the images are best.

Image Registration  Mapping of Evolution. Registration Goals Assume the correspondences are known Find such f() and g() such that the images are best.
Active Shape Models Suppose we have a statistical shape model –Trained from sets of examples How do we use it to interpret new images? Use an “Active Shape.

Active Shape Models Suppose we have a statistical shape model –Trained from sets of examples How do we use it to interpret new images? Use an “Active Shape.
Learning deformable models Yali Amit, University of Chicago Alain Trouvé, CMLA Cachan.

Learning deformable models Yali Amit, University of Chicago Alain Trouvé, CMLA Cachan.
Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image.

Medical Image Registration Kumar Rajamani. Registration Spatial transform that maps points from one image to corresponding points in another image.
Stochastic Systems Group Some Rambling (Müjdat) and Some More Serious Discussion (Ayres, Junmo, Walter) on Shape Priors.

Stochastic Systems Group Some Rambling (Müjdat) and Some More Serious Discussion (Ayres, Junmo, Walter) on Shape Priors.
November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.

November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.
Independent Component Analysis Personal Viewpoint: Directions that maximize independence Motivating Context: Signal Processing “Blind Source Separation”

Independent Component Analysis Personal Viewpoint: Directions that maximize independence Motivating Context: Signal Processing “Blind Source Separation”
1/20 Using M-Reps to include a-priori Shape Knowledge into the Mumford-Shah Segmentation Functional FWF - Forschungsschwerpunkt S092 Subproject 7 „Pattern.

1/20 Using M-Reps to include a-priori Shape Knowledge into the Mumford-Shah Segmentation Functional FWF - Forschungsschwerpunkt S092 Subproject 7 „Pattern.
System Challenges in Image Analysis for Radiation Therapy Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University.

System Challenges in Image Analysis for Radiation Therapy Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University.
Geometric Morphometrics

Geometric Morphometrics
Extended Gaussian Images

Extended Gaussian Images
These improvements are in the context of automatic segmentations which are among the best found in the literature, exceeding agreement between experts.

These improvements are in the context of automatic segmentations which are among the best found in the literature, exceeding agreement between experts.
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.

Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
Edge Detection. Our goal is to extract a “line drawing” representation from an image Useful for recognition: edges contain shape information –invariance.

Edge Detection. Our goal is to extract a “line drawing” representation from an image Useful for recognition: edges contain shape information –invariance.
1 Skeletal Models Correspondence: Improved Interface, Organization, and Speed Clients: Prof. Stephen Pizer Comp.

1 Skeletal Models Correspondence: Improved Interface, Organization, and Speed Clients: Prof. Stephen Pizer Comp.
Fast, Multiscale Image Segmentation: From Pixels to Semantics Ronen Basri The Weizmann Institute of Science Joint work with Achi Brandt, Meirav Galun,

Fast, Multiscale Image Segmentation: From Pixels to Semantics Ronen Basri The Weizmann Institute of Science Joint work with Achi Brandt, Meirav Galun,
Multiscale Profile Analysis Joshua Stough MIDAG PI: Chaney, Pizer July, 2003 Joshua Stough MIDAG PI: Chaney, Pizer July, 2003.

Multiscale Profile Analysis Joshua Stough MIDAG PI: Chaney, Pizer July, 2003 Joshua Stough MIDAG PI: Chaney, Pizer July, 2003.
Medical Image Synthesis via Monte Carlo Simulation James Z. Chen, Stephen M. Pizer, Edward L. Chaney, Sarang Joshi Medical Image Display & Analysis Group,

Medical Image Synthesis via Monte Carlo Simulation James Z. Chen, Stephen M. Pizer, Edward L. Chaney, Sarang Joshi Medical Image Display & Analysis Group,
Clustering on Image Boundary Regions for Deformable Model Segmentation Joshua Stough, Stephen M. Pizer, Edward L. Chaney, Manjari Rao Medical Image Display.

Clustering on Image Boundary Regions for Deformable Model Segmentation Joshua Stough, Stephen M. Pizer, Edward L. Chaney, Manjari Rao Medical Image Display.

Similar presentations

Ads by Google