1. Computerized Craniofacial Reconstruction using CT-derived Implicit Surface Representations D. Vandermeulen, P. Claes, D. Loeckx, P. Suetens Medical Image Computing (ESAT-Radiology) S. De Greef, G. Willems Centre of Forensic Odontology K.U.Leuven, Faculties of Medicine and Engineering Louvre Seminar March 29 2006

2. Cranio-facial reconstruction: What?

3. Manual Craniofacial Reconstruction subjective/ artistic talent Lots of expertise, both explicit (documented) and implicit Errors by inconsistencies in application Misalignment of LM on the skull time consuming Only few reconstructions possible TAYLOR, K. T. 2001. Forensic Art and Illustration. CRC Press LLC. Introduction Manual Computer Bias Dense LM Data Method Results Discussion Conclusion

4. Computer-based reconstructions Small number of tissue thickness measurement landmarks (LMs) Independent soft tissue thickness and facial surfaces Template-bias: face interpolation with a single facial template (either generic or gender/ancestry/age matched) http://www.cs.ubc.ca/nest/imager/contributions/katrinaa/recon.html Introduction Manual Computer Bias Dense LM Data Method Results Discussion Conclusion

5. Removing template bias Using statistical facial templates to remove bias (Claes et al.) Using combined statistical model of facial template and soft tissue thicknesses (Claes et al.) Fitting Algorithm... Database Introduction Manual Computer Bias Dense LM Data Method Results Discussion Conclusion

6. Surface and sparse Landmark-based Craniofacial Reconstruction

7. Dense Landmark model Warp W target Reference skull Reference skin Warped skin Warped skull Introduction Manual Computer Bias Dense LM Data Method Results Discussion Conclusion

8. Statistical dense LM model ….. Template database target Introduction Manual Computer Bias Dense LM Data Method Results Discussion Conclusion

9. CT scan + Simultaneous visualisation hard & soft tissues - Using ionising radiation PM-CT can be used as golden standard? dehydration! in-vivo CT on control population? only by lowering radiation dose! Volumetric Template Data : CT Introduction Data MR CT LD-CT Method Results Discussion Conclusion

10. Volumetric Template Data : MRI MRI scan + Excellent visualisation of soft tissues - Bone details lost Introduction Data MR CT LD-CT Method Results Discussion Conclusion

11. Low-Dose CT Decrease radiation dose to acceptable (?) level while retaining sufficient quality for diagnosis, therapy or image-based measurements Starting from clinical multi-slice spiral CT protocol (Siemens Sensation 16 (Erlangen, Germany)) by lowering the X-ray source current and voltage and increasing the pitch. Measured effective radiation dose: 0.18 mSv i.o. 1.5 mSv Measuring image quality: thickness differences smaller than a voxel (<0.5 mm). Introduction Data MR CT LD-CT Method Results Discussion Conclusion

12. CT image preprocessing Acquisition and conversion from DICOM to Analyze Noise reduction using edge preserving filtering Metal artifact removal Segmentation of skin and bone surfaces by (hysteresis) thresholding and mathematical morphology Implicit Surface representation by signed Distance Transformation Introduction Data Method Results Discussion Conclusion

13. Implicit Functions for Object Representations and Transformations: a tentative tutorial Dirk Vandermeulen Medical Image Computing Seminar January 17, 2003

14. 3-D example Copyright FarField Technology Ltd.

15. … Shape Morphing Alternative: interpolate the smooth implicit functions! Example: f(x) = signed distance G. Turk and J. F. O Brien, Shape Transformation using Variational Implicit Functions, Siggraph 99 f 1 (x)>0 f 2 (x)>0 t.f 1 (x)+(1-t).f 2 (x)>0 0 t 1 Shape Transformation Using Variational Implicit Functions, Greg Turk James F. OBrien, ACM Siggraph99

16. Implicit Surface Representation Signed Distance Transform (sDT) Introduction Data Method MAR sDT Warping Reconstruction Results Discussion Conclusion

17. Craniofacial reconstruction: method Warp W target Reference skullWarped skull Warp W Reference skin Warped skin Introduction Data Method MAR sDT Warping Reconstruction Results Discussion Conclusion

18. Warping method (D.Loeckx et al.) Represent warping by tensor-product B-Spline Free Form Deformation (FFD) Introduction Data Method MAR sDT Warping Reconstruction Results Discussion Conclusion

19. Warping method (D.Loeckx et al.) Regularization of FFD by Volume-preserving penalty

20. Example: template skull to target skull warping

21. Example: extrapolation to template skin warping =?

22. Example: extrapolation to template skin warping

23. Skin Surface Reconstruction Construct (weighted) average of warped skin sDTs Introduction Data Method MAR sDT Warping Reconstruction Results Discussion Conclusion

24. Example

25. Validation Given only small-sized database (N=20), how to separate into test and validation subsets? N-fold Cross-Validation or Leave-one-out CV: –For i=1:NrSubjects Reconstruct Subject i from all other subjects in Database Compare Result to ground truth of i Evaluate Error

26. Qualitative Validation Qualitative: 3D reconstructions vs subset of database (face pool comparisons) Introduction Data Method Results Discussion Conclusion

27. Face pool comparisons

28. Quantitative Validation I Calculate distances between reconstructed and ground truth surface Introduction Data Method Results Discussion Conclusion |d| = 1.6 ± 1.2 mm

29. Quantitative Validation I Gather error statistics over all subjects Define M ( 500) test points on a reference head surface Find corresponding points on all surfaces by non-rigid surface-based warping (Claes et al.) Evaluate error (distance from reconstructed surface to real surface) at test points: mean, std, etc…

30. Quantitative Validation I: results Average (1.9mm)Std (1.7mm)

31. Quantitative Validation II Not reconstruction accuracy but recognition accuracy Based on similarity measure between surfaces or, in this case, M reference points p i (same as before) on the surfaces S Use coordinate-system free representation (invariant to translation/rotation) of surface S Euclidean Distance Matrix E S : E S (i,j) = ||p i -p j ||

32. Quantitative Validation II How to measure similarity between two surfaces S1 and S2? Compare E S1 to E S2 : e.g. –Sum of Squared Differences: ||E S1 -E S2 || = i,j>i ( E S1 (i,j) – E S2 (i,j) ) 2 /L >= 0 –(normalized) Cross-Correlation Invariance to scaling/size by normalizing EDM with size factor, e.g. geometric mean N S (i,j) = E S (i,j) / (E S ), with (E S ) = ( ij E S (i,j) ) 1/L

33. Quantitative Validation II Generate Classification Rank Matrix 1 7 20 11 5 4 16 13 3 12 6 15 19 17 8 9 2 18 14 10 2 6 15 12 19 16 3 5 17 7 11 4 8 20 1 13 18 10 14 9 3 5 4 16 11 12 7 6 13 15 17 19 2 20 8 1 14 9 10 18 13 4 3 14 9 8 16 12 17 5 10 1 11 7 2 15 6 20 19 18 7 3 11 5 20 1 16 4 6 19 2 12 15 13 17 18 8 9 14 10 6 15 19 11 2 7 16 12 3 20 5 1 17 13 4 18 8 14 10 9 5 1 13 6 9 3 14 7 4 16 2 8 11 17 12 20 10 15 19 18 8 10 14 4 13 17 12 3 9 7 16 15 6 2 1 5 11 20 19 18 9 7 4 14 8 13 10 3 1 17 5 12 6 16 11 20 15 2 19 18 10 8 14 4 13 17 12 7 9 3 16 15 6 1 2 11 5 20 19 18 11 5 6 3 20 1 16 19 12 4 15 13 2 7 17 18 8 14 10 9 12 15 3 16 19 6 2 17 4 20 11 8 5 13 1 7 10 18 14 9 4 13 14 7 17 8 3 16 10 1 11 15 12 6 9 5 20 2 19 18 14 13 8 4 10 9 7 17 3 16 12 1 6 15 11 5 2 20 19 18 6 12 2 3 15 20 11 16 13 5 17 4 1 8 19 7 10 18 14 9 16 3 19 12 6 11 4 5 13 17 2 1 15 7 20 8 10 14 18 9 17 13 12 3 8 4 16 10 2 14 15 6 19 7 11 5 1 20 9 18 18 6 19 5 12 11 16 1 2 20 15 3 17 4 13 7 8 10 14 9 6 16 12 2 19 3 5 11 17 1 15 20 18 4 13 7 8 10 14 9 20 1 11 5 6 12 3 15 16 7 17 4 19 13 2 18 8 14 10 9 Correct Rank 1 Classification: 14/20 (13/20) Correct Rank <= 2 Classification: 16/20 (14/20)

34. Algorithmic improvements Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts Non-rigid registration –fine tuning of regularization parameters to improve skull-skull matching and extrapolation quality –Alternative deformation models Statistical Deformation Models (based on sDT or original CT of database) –Combination with local models (e.g. nose (De Greef)) –Combination with point/surface model (Claes) Weighted averaging of sDT –Weights ~ skull overlap –Weights ~ class similarity (gender, age, BMI) –Spatially varying weights Statistical Modes of Variation Introduction Data Method Results Discussion Conclusion

35. Metal Artifacts Introduction Data Method Results Discussion Conclusion

36. Metal Artifact Removal Introduction Data Method MAR sDT Warping Reconstruction Results Discussion Conclusion

37. Algorithmic improvements Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts Non-rigid registration –fine tuning of regularization parameters to improve skull-skull matching and extrapolation quality, again using leave-one-out cross-validation –Alternative deformation models Statistical Deformation Models (based on sDT or original CT of database) –Combination with local models (e.g. nose (De Greef)) –Combination with point/surface model (Claes) Weighted averaging of sDT –Weights ~ skull overlap –Weights ~ class similarity (gender, age, BMI) –Spatially varying weights Statistical Modes of Variation Introduction Data Method Results Discussion Conclusion

38. Algorithmic improvements Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts Non-rigid registration –fine tuning of regularization parameters to improve skull-skull matching and extrapolation quality –Alternative deformation models Statistical Deformation Models (based on sDT or original CT of database) –Combination with local models (e.g. nose (De Greef)) –Combination with point/surface model (Claes) Weighted averaging of sDT –Weights ~ skull overlap –Weights ~ class similarity (gender, age, BMI) –Spatially varying weights Statistical Modes of Variation Introduction Data Method Results Discussion Conclusion

39. Attribute-modulated reconstruction All reconstructions so far made with all data in the database, irrespective of gender, age and BMI! sDT = i w i sDT i, w i = 1/N

40. Attribute-weighted interpolation How to bias reconstruction to specific attribute values? (k-)Nearest Neighbour? Problem: small database, hence weak statistical model (PCA, PLS, …) Solution(?): Shape by Example –Given attribute values (gender, age, BMI) p i and q of template subjects i and target subject, resp. –Find weight w i (q) to apply to sDT i in weighted average sDT = I w i (q) sDT i, w i (p j ) ij –Determined using RBF smoothest approximation

41. Example

42. AllFemales onlyMales only AWI Females+BMI Ground truth

43. Algorithmic improvements Metal Artifact Reduction: using morphology (opening) only in slices ~ artifacts Non-rigid registration –fine tuning of regularization parameters to improve skull-skull matching and extrapolation quality –Alternative deformation models Statistical Deformation Models (based on sDT or original CT of database) Weighted averaging of sDT –Weights ~ skull overlap –Weights ~ class similarity (gender, age, BMI) –Spatially varying weights Statistical Modes of Variation Introduction Data Method Results Discussion Conclusion

44. Proof of concept of volumetric cranio-facial reconstruction Validation procedure required on a representative database Metal Artifact Reduction is required Missing Data problem using deformation model Comments? Introduction Data Method Results Discussion Conclusion