Frank Heckel1, Olaf Konrad2, Heinz-Otto Peitgen1 Fast and Smooth Interactive Segmentation of Medical Images Using Variational Interpolation Frank Heckel1, Olaf Konrad2, Heinz-Otto Peitgen1 1 Fraunhofer MEVIS, Germany 2 MeVis Medical Solutions, Germany

Overview Introduction Variational Interpolation Interactive Segmentation using Variational Interpolation Conclusion Outlook

Introduction Automatic segmentation Good reproducibility Not always possible Manual segmentation Time-consuming Bad reproducibility Semi-automatic segmentation Automatic algorithm which requires user interaction Interactive segmentation: Algorithm reacts on user in almost real-time (calculation times <1s)

Introduction Manual segmentation by drawing contours Goal: Interpolation of a 3D mask from a few (user drawn) contours Conditions: Parallel or arbitrary oriented contours Smooth (at least C1-continuous) Fast (interactive calculation times) Surface shall contain all contour points “Extrapolation” character Solution: Variational Interpolation

Variational Interpolation Radial basis function 3D extension of thin-plate spline interpolation Object is defined by an implicit function: Constraints for f(x): Smooth (C1- or C2-continuous) Contains all contour points “Surface constraints” The normal on the surface corresponds to the normal in each contour point “Normal constraints” Surface constraint Normal constraint

Variational Interpolation Solution Solve the linear system that describes f(x) to get the weights wj Matrix is symmetric  Bunch-Kaufmann algorithm (LAPACK) Global interpolation scheme

Interactive Segmentation using Variational Interpolation Voxelization Iteratively evaluate f(x) only on the surface (surface tracking similar to marching cubes) Start points: first point of each contour (unless it is already part of a reconstructed surface) Fill the surface slice-wise to get a segmentation of the whole object(s)

Interactive Segmentation using Variational Interpolation Segmentation process Draw / Delete / Edit Contours Calculate Segmentation Check Result

Interactive Segmentation using Variational Interpolation Performance Speedup Solving the linear system is O(n³) Fast LAPACK implementation is essential Support 64-bit, SSE, Multi-Threading Intel Math Kernel Library (MKL) AMD Core Math Library (ACML) Voxelization is O(nm) Multi-threading can be used Current implementation does not scale very well Threads CLAPACK MKL SP 1 52.75s 10.01s 5.27 2 - 5.74s 9.19 4 3.57s 14.78 8 2.89s 18.25 Threads Voxelization SP 1 4.15s 1.00 2 2.49s 1.67 4 1.92s 2.16 8 2.13s 1.95 Metastasis example* 17 contours, n=6246 constraints, m=23541 surface voxels Test system: 2x Intel Xeon X5550 (2.66GHz, Turbo Boost and Hyper-Threading disabled), 12 GB RAM, Windows 7 64-bit, MKL 10.2.2

Interactive Segmentation using Variational Interpolation Performance Quality preserving constraint reduction to become interactive Points that don’t contribute to the overall geometry are removed Measure: Angle and length of adjacent edges q=1.0 q=0.2 Quality n LAPACK Voxelization Total SP Overlap 1.0 6246 2.89s 1.92s 5.13s 1.00 100% 0.5 3608 0.75s 1.18s 2.08s 2.47 99.94% 0.2 1410 0.08s 0.55s 0.67s 7.66 99.32% 0.1 710 0.02s 0.36s 0.41s 12.51 97.05% Test system: 2x Intel Xeon X5550 (2.66GHz, Turbo Boost and Hyper-Threading disabled), 12 GB RAM, Windows 7 64-bit, MKL 10.2.2 (8 threads), Voxelization with 4 threads

Interactive Segmentation using Variational Interpolation Robustness Self-intersecting contours Sign of normal constraint needs to be modified Additional start point for surface tracking at intersection Reduced contours Additional constraints might have to be inserted for long line segments

Interactive Segmentation using Variational Interpolation Limitations Memory consumption is O(n²) Performance for large objects not interactive by now Voxelization becomes bottleneck Liver example (14 contours, 169300 surface voxels): Quality n LAPACK Voxelization Total Overlap 1.0 13796 23.42s 28.95s 53.28s 100% 0.5 7916 4.94s 16.79s 21.94s 99.5% 0.2 3170 0.5s 7.3s 7.82s 98.74% 0.1 1594 0.1s 8.3s 8.41s 97.68% Test system: 2x Intel Xeon X5550 (2.66GHz, Turbo Boost and Hyper-Threading disabled), 12 GB RAM, Windows 7 64-bit, MKL 10.2.2 (8 threads), Voxelization with 4 threads

Interactive Segmentation using Variational Interpolation Limitations Normals are calculated in 2D (based on one contour) Contradictory contours More complex self-intersections Low number of constraints

Conclusion Accurate and smooth surface reconstruction Contours can be arbitrarily oriented Plausible extrapolation Interactive for small objects if: Fast LAPACK implementation is used (MKL or ACML) Voxelization is parallelized The number of constraints is reduced Purely geometrical Independent of modality or scanning parameters Wide range of segmentation tasks Can be easily extended to 4D (needs different radial basis functions)

Outlook Reduction of voxelization time More advanced parallelization GPU? Further improve robustness Investigate local interpolation schemes E.g., compactly supported radial basis functions by Morse et al.* Algorithm is available in MeVisLab 2.1 (“CSOConvertTo3DMask”) Uses only CLAPACK * Bryan S. Morse et al., “Interpolating implicit surfaces from scattered surface data using compactly supported radial basis functions”, ACM SIGGRAPH Courses, 2005

Thanks to: Thank you! frank.heckel@mevis.fraunhofer.de

Variational Interpolation Effect of the normal constraints: G. Turk and J. F. O’Brien, “Modelling with implicit surfaces that interpolate”, ACM Transactions on Graphics, Vol. 21(4), 2002

Variational Interpolation Difference to shape-based interpolation (distance transform): Distance transform Variational interpolation G. Turk and J. F. O’Brien, “Shape transformation using variational implicit functions”, Proceedings of SIGGRAPH’99, 1999