1. Medial Techniques for Automating Finite Element Analysis Jessica Crouch

2. Motivation Deformation Modeling Aim: Model soft tissue deformation Applications include –Medical simulation, surgical planning –Tomotherapy –Non-rigid registration of 3D medical images

3. Motivation Physically based deformable models Partial differential equations (PDEs) model the deformable behavior of materials –Establish stress / strain relationship Finite element method solves PDEs for discretized object models

4. Motivation Applications of FEM in Medical Imaging Non-rigid registration –Prostate – Bharatha, Hirose, et al. –Brain – Ferrant, Warfield, et al. –Breast – Azar Motion tracking –Heart wall – Papademetris, Shi, et al. Simulation –Facial surgery – Chabanas and Payan –Liver surgery – Cotin, Delingette, Ayache –Childbirth – Lapeer and Prager

5. Motivation Finite Element Method (FEM) Model geometric properties –Discretize space with a mesh composed of Nodes Elements –Boundary fitted

6. Motivation Finite Element Method (FEM) Model physical properties –Choose equations & coefficients that describe the material's deformability Assemble the finite element system of equations

7. Motivation FEM for Medical Image Applications Steps include –Segmentation –Mesh creation –Equation and coefficient selection –Boundary condition specification Deforming forces, displacements –Solution Labor Intensive, Computationally Intensive Automate using m-rep model framework

8. Planning Image –Imaging probe deforms prostate Intra-operative image –Prostate is relatively undeformed Motivation Prostate Registration Problem

9. Thesis Statement –M-rep based multiscale mesh generation, –M-rep derived boundary conditions, and –Multiscale solution of a finite element system of equations are techniques that improve the automation and efficiency of finite element analysis as it is applied to medical imaging applications and to the prostate brachytherapy application in particular.

10. Outline Motivation & Overview FEM model construction  M-reps & image segmentation –Mesh construction –Finite element system of equations Boundary conditions Solution Results for phantom prostate image registration Conclusions & Future Work

11. FEM Model Construction Automation of FEM for Imaging M-rep models –Medially based solid models –Provide wealth of shape information Global Local –Facilitate segmentation, meshing, boundary condition, and solution steps of FEM

12. FEM Model Construction M-rep Models Objects are decomposed into parts based on medial sheet branching Each branch of a medial sheet is represented by a figure Hierarchical tree of figures is organized by branching structure

13. FEM Model Construction M-rep Models A figure consists of –Single medial sheet –Functions defined on the medial sheet Radius Boundary direction vectors Boundary displacement vectors (small) Frame

14. FEM model construction M-rep Models’ Discrete Representation Each figure sampled by lattice of medial atoms Lattice structure provides (u,v) coordinate system on medial sheet

15. FEM Model Construction M-rep Model Visualization Adjusting the m-rep parameters stored in each atom affects the model’s geometry

16. FEM Model Construction M-rep Model Visualization A multi-figure m-rep object consists of multiple parts, each represented by a separate medial sheet A row of hinge atoms connects a subfigure to its host figure

17. FEM Model Construction M-rep Object Coordinate System (u,v,t,  ) coordinates parameterize an m-rep model Rotating, scaling, deforming an m- rep model changes its (u,v,t,  )  (x,y,z) mapping

18. FEM Model Construction M-rep Based Image Segmentation Pablo program –Builds a new m-rep model or –Adjusts an existing m-rep to fit an object in a 3D image Optimizes atoms (medial sheet position, radius function, boundary function, etc.) to maximize –image match –expected geometry Works well with clear boundaries, still being improved

19. FEM Model Construction M-rep Segmentation Demonstration

20. Outline Motivation & Overview FEM model construction M-reps & image segmentation  Mesh construction –Finite element system of equations Boundary conditions Solution Results for phantom prostate image registration Conclusions & Future Work

21. FEM Model Construction Mesh Construction Requirements Element choices – Shape Tetrahedra Hexahedra (preferred) Pyramids, wedges, etc.

22. FEM Model Construction Mesh Construction Requirements Elements must not be overly skewed Element size should fit the –Geometric detail of an object region –Solution precision needed in an object region Meshes typically must be seamless –Element face compatibility

23. FEM Model Construction Mesh Construction Top-down approach to hexahedral mesh design Based on m-rep models Mesh generated in m-rep object coordinate system, then mapped to world space

24. Step 1: Construct a sampling grid on the (u,v) parameter plane of the medial surface –Spacing depends on object radius, and is chosen to give elements approx. equal edge lengths in all directions FEM Model Construction Mesh Construction: Single Figure

25. Step 2: Compute coordinates for other layers of nodes, using illustrated meshing pattern. –Result is desirable hexahedral mesh. FEM Model Construction Mesh Construction: Single Figure

26. Step 3: Optimize node locations to improve element shapes –Objective function is based on the determinant of the Jacobian of the element shape function f( , ,  ) = (x,y,z) FEM Model Construction Mesh Construction: Single Figure

28. Quantitative evaluation of mesh quality Histograms of det(J) for prostate mesh elements Left: pre-optimization Right: post-optimization FEM Model Construction Mesh Construction: Single Figure

29. Mesh of single figure prostate m-rep model

30. FEM Model Construction Mesh Construction: Single Figure 5 object male pelvis m-rep model mesh

31. Mesh of space exterior to m-rep modeled objects necessary –To transmit forces between separate objects –To compute a smooth deformation field surrounding a modeled object Surrounding space meshed with –Pyramid layer on top of hexahedral elements –Tetrahedra fill remaining volume of interest generated by CUBIT FEM Model Construction Mesh Construction: Single Figure

32. Pyramid and tetrahedral elements for space external to m-rep model

33. FEM Model Construction Mesh Construction: Single Figure

34. Must ensure smooth, compatible connection between host figure and subfigure mesh elements FEM Model Construction Mesh Construction: Multi-Figure

35. Achieve compatibility by 1)Designing a host figure’s mesh so that the mesh lines along its surface fit the footprint of a subfigure 1)Designing a transition mesh pattern that fits between the main bodies of the host and subfigure meshes FEM Model Construction Mesh Construction: Multi-Figure

36. Compute host / subfigure intersection in terms of –Host figure object coordinates –Subfigure object coordinate FEM Model Construction Mesh Construction: Multi-Figure

37. Host mesh design: –Fit subfigure footprint with Cartesian type surface mesh –Complete the surface mesh –Interpolate interior nodes between the surface nodes FEM Model Construction Mesh Construction: Multi-Figure

38. The mesh transition region must adjust the number of rows and columns in the mesh pattern as well as switch between different mesh pattern topologies. FEM Model Construction Mesh Construction: Multi-Figure Avoid:

39. Subfigure transition mesh is template based –Template patterns chosen based on the mesh patterns defined For the subfigure footprint on the host surface Through a cross-section of the subfigure FEM Model Construction Mesh Construction: Multi-Figure

40. Template patterns assembled in m-rep coordinate space, then mapped to world space FEM Model Construction Mesh Construction: Multi-Figure

41. Outline Motivation & Overview FEM model construction M-reps & image segmentation Mesh construction  Finite element system of equations Boundary conditions Solution Results for phantom prostate image registration Conclusions & Future Work

42. Many constitutive models available –Linear elastic –Hyperelastic –Viscoelastic –Viscous Fluid Linear elasticity chosen for prostate registration experiment Methodology applies equally well for other constitutive models FEM Model Construction Finite Element Equations

43. Linear elastic model –Stress, , is proportional to strain, . –Linear elastic PDE: Elastic constants –Young’s modulus –Poisson’s ratio FEM Model Construction Finite Element Equations

46. Solution to the PDE is approximated on the mesh using element interpolation functions Result is a linear system of equations The full system of equations is singular FEM Model Construction Finite Element Equations

47. FEM Model Construction Mesh Construction: Single Figure

48. Outline Motivation & Overview FEM model construction M-reps & image segmentation Mesh construction Finite element system of equations  Boundary conditions Solution Results for phantom prostate image registration Conclusions & Future Work

49. Boundary conditions take the form of force vectors or displacement vectors applied to mesh nodes Displacement type boundary conditions allow a finite element system of equations to be reduced –The displacement of at least one node must be specified The reduced system of equations is non-singular and solvable FEM Model Construction Boundary Conditions

50. Force vectors or displacement vectors are not available directly from images An image pair provides information about changes in boundary shape FEM Model Construction Boundary Conditions

51. Use pair of m-rep segmentations to generate displacement type boundary conditions M-rep correspondences are based on the shared coordinate system of a pair of m-rep models FEM Model Construction Boundary Conditions

52. M-rep generated surface displacement vectors FEM Model Construction Boundary Conditions

53. M-rep correspondences are not necessarily physical correspondences, so boundary condition optimization was tested –Surface correspondences were varied –Potential energy of the deformation was minimizedPotential energy FEM Model Construction Boundary Conditions

55. Optimization had a negligible effect on phantom prostate deformation result Unoptimized m-rep generated boundary displacements are sufficiently accurate for prostate image registration Problems with larger deformations might benefit from boundary condition optimization

56. Outline Motivation & Overview FEM model construction M-reps & image segmentation Mesh construction Finite element system of equations Boundary conditions  Solution Results for phantom prostate image registration Conclusions & Future Work

57. For a 3D mesh with N nodes a 3N×3N system of equations is produced Reduced system is reduced by the number of boundary conditions Solution options: –Direct solution methods O(N 3 ) –Use iterative method with sparse matrix, get O(N 2 ) –Use conjugate gradient iterative solver for better convergence – possibly as good as O(N 9/8 ) FEM Model Construction Solution

58. To improve solution accuracy, subdivide mesh elements Add nodes at the midpoints of edges, quad faces, and hex volumes FEM Model Construction Solution

59. Subdivision with Euclidean world coordinates –refines the solution –does not change the model’s geometric accuracy Subdivision with m-rep object coordinates –refines the solution –refines the mesh geometry FEM Model Construction Solution

60. Mesh Subdivision Subdivision with m-rep object coordinates –Improved smoothness –Mesh geometry more closely approximates m-rep implied boundary with each subdivision

61. Mesh Subdivision Subdivided prostate mesh –3 levels

62. Multiscale 5 object pelvis mesh FEM Model Construction Solution

63. Mesh Subdivision Mesh size grows quickly with subdivision Subdivision –Improves the resolution of the model –Increases solution time Prostate mesh node and element counts for each subdivision level:

64. With iterative solution methods, an initial solution guess is required Use coarse mesh solution to predict solution on a finer mesh Interpolation performed in m-rep object coordinates rather than world coordinates FEM Model Construction Solution

65. Coarse-to-fine solution strategy improves solution efficiency FEM Model Construction Solution

66. Outline Motivation & Overview FEM model construction M-reps & image segmentation Mesh construction Finite element system of equations Boundary conditions Solution  Results for phantom prostate image registration Conclusions & Future Work

67. Planning Image –Imaging probe deforms prostate Intra-operative image –Prostate is relatively undeformed Results Prostate Image Registration Avg. seed movement: 9.4 mm Avg. movement of bottom plane of seeds: 11.6 mm

68. Results Prostate Image Registration

70. Red: Inflated image Gray: Computed deformation applied to uninflated image Results Prostate Image Registration

71. Seed centers –Blue: segmented from inflated image –Green: segmented from uninflated image, then moved by the computed deformation

72. Results Prostate Image Registration Results are averages for 75 seeds that were manually segmented in uninflated and inflated probe images. The computed deformation was applied to uninflated seed positions to map them into the inflated image. The difference between mapped seed centers and seed positions identified in the inflated image was measured. Segmentation error cannot be separated from these error estimates

73. Results Prostate Image Registration Image resolution in x and y directions:.7mm Image resolution in z direction: 3 mm Resolution limits segmentation accuracy, so a larger error estimate is expected for the z direction

74. Results Prostate Image Registration Registration accuracy for the bottom plane of seeds is particularly important and is analyzed separately

75. Results Prostate Image Registration Sensitivity to segmentation error was evaluated by perturbing the prostate model

76. Results Hex / Tet mesh comparison Tet mesh constructed with CUBIT from the surface tiles of the hex mesh

77. Results Hex / Tet mesh comparison Hex mesh accuracy is better Accuracy gap is largest in the direction with the most deformation

78. Outline Motivation & Overview FEM model construction M-reps & image segmentation Mesh construction Finite element system of equations Boundary conditions Solution Results for phantom prostate image registration  Conclusions & Future Work

79. Meshing –Automatic hexahedral mesh generation from m-rep models Boundary Conditions –Automatic displacement boundary conditions generated from a pair of m-rep segmentations Conclusions Summary: Claims

80. Solution –Resolution adjustable with m-rep coordinate subdivision –Efficiency improvement by predicting solution on a fine mesh based on solution from a coarser mesh Prostate Phantom Results –Seed prediction error on the order of the segmentation error / image resolution

81. Conclusions Automated Process To register images A & B by deforming image A:

82. Thesis Statement –M-rep based multiscale mesh generation, –M-rep derived boundary conditions, and –Multiscale solution of a finite element system of equations are techniques that improve the automation and efficiency of finite element analysis as it is applied to medical imaging applications and to the prostate brachytherapy application in particular.

83. Conclusions Future Work Now: –Local subdivision –Apply to other parts of anatomy –Use more sophisticated material models Long term: –Use database of deformable organ models to further automate the creation of individualized simulations

84. Acknowledgments Steve Pizer Committee members: Ed Chaney, Guido Gerig, Sarang Joshi, Carol Lucas, and Julian Rosenman MIDAG members MSKCC collaborators Family & friends