1. Automatic Discovery of Connections between Vietnamese’s Anthropometric Features – Master Thesis Dinh Quang Huy Iba Laboratory

2. Contents Background and Motivation 1 Our Approach 2 Evaluation and Result 3 Conclusion and Future Work 4 2

3. Facial Reconstruction  Recreating the face of an unidentified individual from their skeletal remains.  Forensic: determining the face of victims in the murders  Archaeology: verifying the remains of historic figures.  Anthropology: approximating the look of early hominid forms. 3

4. Facial Reconstruction Techniques  Three methods  2D reconstruction  Manual 3D reconstruction  Computer-aided 3D Reconstruction 4

5. 2D Reconstruction  Requires a forensic artist who draws the picture  Usually requires cooperation between artist and forensic anthropologist  Reconstruction process  Tissue depth markers are fixed on the skull  Taking photograph  Draw the portrait 5

6. Manual 3D Reconstruction  Clay modeling approach.  Landmark dowels are placed on the predefined landmarks on the skull  The lengths of these dowels are the tissue depth at the dowel’s landmark.  Clay is then filled between the dowels to build up the tissue depth of the unknown person  Effectiveness of clay reconstruction depends wholly on the skill of the sculptor 6

7. Computer-aided 3D Reconstruction  Manual approaches have limitations such as time consuming and sculptor experience dependence  Most computer-aided 3D reconstruction systems use the same process  The skull is scanned to obtain its 3D model  Facial landmarks on the skull are detected or manually selected  Tissue depths at these landmarks are referenced  Based on these landmarks and tissue depths, the face is built by interpolation or morphing 7

8. Soft tissue thickness prediction  The critical issue is soft tissue thickness prediction  All known facial reconstruction systems collect tissue depths at predefined landmarks of a population and use the average value in the reconstruction.  There are some methods to collect tissue depths information  Invasive technique: needle technique  Non-invasive techniques: ultrasound, MRI, CT 8

9. Problem and Motivation  Lack of a suitable method for soft tissue thickness prediction  The models generated automatically by the systems are often modified manually by experts to have the final model.  Do facial features such as eye and nose shapes depend on the skull?  Many researchers have been working to prove that these features depend on the bones. 9

10. Contents Background and Motivation 1 Our Approach 2 Evaluation and Result 3 Conclusion and Future Work 4 10

11. Our Approach  We treat the soft tissue thickness prediction issue the missing data problem  The tissue depths are the target  The skull shape is the input  One solution for the missing data problem is to  Collect data of pairs of input and target  Apply machine learning techniques to obtain a model that best describes input-target relationship 11

12. Data Description: Target Data  Soft tissue thicknesses at facial landmarks  landmarks are believed to contain the most information about the skull shapes 12  Facial features: distance between 2 pupils, nose height, and nose length.

13. Data Description: Input Data  Distances between facial landmarks  These distances must be measured from the skull itself. In the case of facial reconstruction, only these distances are known 13

14. Data Description (cont) Input (17 fields)Target (38 fields) Cranial height Cranial width … Vertex thickness Nose length … Data 1137140…6.546… Data 2127145…5.843… Data 3132.4145…5.846.9… ………………… Data98129.4139…4.142.7… 14 Need to predict these information These information can be measured from any discovered skull

15. Data Collecting  Available techniques  Invasive: needle technique  Non-invasive: ultrasound, MRI, CT  Needle technique can not be used on living people, so its accuracy is very low  Needle technique can not be used to measure distances between facial landmarks, either.  Among the non-invasive techniques, CT is the best option  CT image is in high quality and the distances between facial landmarks can be measured visually. 15

16. Data Collecting (cont) 16

17. Data Collecting (cont)  Collected data of 220 Vietnamese candidates  98 males, 122 females  Age varies from 17 to 82  Weight varies from 38kg to 75kg 17

18. Discovery of Relationships  Two methods  Linear Regression Fitting a linear equation to the observed data Can only discover one to one relationships  Artificial Neural Networks Non-linear nature Work in two modes, training and testing 18

19. Applying Linear Regression  Linear Regression is applied for all pairs of input-target  For each target field, the pair with best performance (in term of MSE) is chosen 19

20. Neural network approach  This structure can represent any functional relationship between inputs and outputs if the hidden layer has enough neurons  two-layer feed forward network  tan-sigmoid transfer function in the hidden layer  linear transfer function in the output layer  Every target field has its own neural network prediction model 20

21. Contents Background and Motivation 1 Our Approach Evaluation and Result 3 Conclusion and Future Work 4 2 21

22. Evaluation  Ten-fold cross validation  Linear Regression and Neural Network techniques were implemented  The results are compared with the ‘average method’ 22

23. Prediction Result 23

24. Reconstruction Result 24

25. Reconstruction Result (cont) 25

26. Contents Background and Motivation 1 Our Approach 2 Evaluation and Result 3 Conclusion and Future Work 4 26

27. Conclusions  Proposed an approach for automatic discovery of relationships between anthropometric features  Critical in facial reconstruction systems  Evaluation results show that the approach has better performance than the ‘average method’ which is used in most facial reconstruction systems  Although the data is collected from Vietnamese, the study can be applied for any race 27

28. Publications 1.Quang Huy Dinh, Thi Chau Ma, The Duy Bui, Trong Toan Nguyen, Dinh Tu Nguyen. “Facial soft tissue thicknesses prediction using anthropometric distances.” Studies in Computational Intelligence (351/2011), page 117-126, Springer, (2001) 2.Thi Chau Ma, Dinh Tu Nguyen, Quang Huy Dinh, The Duy Bui. “3D facial reconstruction system from skull for Vietnamese.” Proceedings of the 3rd International Conference on Knowledge and Systems Engineering, page 120-127, IEEE, (2011). 28

30. Linear equation lists 30 N#OUTPUTINPUTLINEAR EQUATIONMSE 1vertexcranial breadthy = -0.038041x + 10.82511.1230 2trichioncranial heighty = 0.07284x - 4.64471.1500 3glabellaforehead width (ft-ft)y = 0.073272x - 2.24821.0456 4nasioncranial heighty = 0.070439x - 5.0220.8300 5rhinionmolar-molary = -0.036784x + 4.18170.3635 6pronasalebase nose lengthy = 0.34191x + 5.69066.7684 7nose_lengthn-rhy = 1.1274x + 27.573311.8050 8subnasalebn-bny = -0.3371x + 22.56464.7101 9upper_lip_bordermolar-molary = 0.12137x + 5.07324.5644 10lower_lip_bordercranial heighty = 0.086747x + 1.78372.9479 11stomionbase facial length (ba-pr)y = -0.072432x + 10.8432.0073 12metalal-aly = 0.18113x + 3.56323.7307 13metonal-aly = 0.13342x + 1.07832.1684 14opisthooranional-aly = 0.14536x - 0.0195731.5853 15exocanthion_(R)cranial heighty = 0.052174x - 3.23080.7072 16exocanthion_(L)cranial heighty = 0.054596x - 3.47050.7609 17endocanition_(R)cranial heighty = 0.10877x - 8.82662.2855 18endocantion_(L)cranial heighty = 0.13074x - 11.59922.1918 19pupil-pupilex-exy = 0.71282x - 5.94724.6317

31. Linear equation lists (cont) 31 N#OUTPUTINPUTLINEAR EQUATIONMSE 20supraobital_(R)en-eny = 0.083979x + 2.67810.5880 21supraobital_(L)al-aly = 0.097903x + 1.06740.5861 22infraobital_(R)cranial heighty = 0.051155x - 2.40111.2943 23infraobital_(L)cranial heighty = 0.055695x - 2.99221.0309 24zygomatic_arch_(R)n-rhy = 0.070673x + 3.490.8013 25zygomatic_arch_(L)base cranial length (n-ba)y = -0.049215x + 9.34950.7973 26zygomatic_(R)base facial length(ba-pr)y = -0.042995x + 8.5560.7596 27zygomatic_(L)base facial length(ba-pr)y = -0.051253x + 9.35690.8386 28porion_(R)bn-bny = 0.1512x + 4.58233.0310 29porion_(L)bn-bny = 0.16467x + 4.13362.1702 30gonion_(R)al-aly = 0.088104x + 0.530220.9684 31gonion_(L)nasal projectiony = 0.10235x + 3.50320.8845 32alare_(R)al-aly = 0.18611x + 1.09061.7071 33alare_(L)al-aly = 0.2063x + 0.330531.6000 34lateral_nasal_(R)al-aly = 0.23816x - 2.07831.4152 35lateral_nasal_(L)al-aly = 0.2648x - 3.10831.3845 36nose_heightbase nose lengthy = 0.17297x + 15.41493.7506 37bucal_(R)bn-bny = 0.31662x + 6.925412.2240 38bucal_(L)bn-bny = 0.31206x + 7.020912.3088

32. MSE values for the three methods 32 N#OUTPUTAVGLRNN 1vertex1.19141.0625 0.8928 2trichion1.29451.0877 1.0664 3glabella1.2074 1.0110 1.0706 4nasion0.96990.7571 0.7220 5rhinion0.3886 0.3400 0.3797 6pronasale7.96216.0558 5.2456 7nose_length21.862110.8344 8.7059 8subnasale6.3008 4.3927 4.6878 9upper_lip_border4.94684.3581 3.7205 10lower_lip_border3.16742.7312 2.4167 11stomion2.21931.8766 1.8168 12metal4.10073.4298 3.3625 13meton2.3685 1.9901 2.0885 14opisthooranion1.89091.5124 1.1001 15exocanthion_(R)0.7884 0.6635 0.7084 16exocanthion_(L)0.8609 0.7121 0.8459 17endocanition_(R)2.58042.0950 1.7213 18endocantion_(L)2.67792.0706 2.0099 19pupil-pupil10.8380 4.4587 4.9687

33. MSE values for the three methods (cont) 33 N#OUTPUTAVGLRNN 20supraobital_(R)0.66890.5533 0.4556 21supraobital_(L)0.68590.5340 0.4986 22infraobital_(R)1.40381.2479 1.0475 23infraobital_(L)1.1147 0.9573 1.1920 24zygomatic_arch_(R)0.8485 0.7432 1.6805 25zygomatic_arch_(L)0.8857 0.7400 0.7982 26zygomatic_(R)0.83260.6982 0.5635 27zygomatic_(L)0.9557 0.7722 1.3729 28porion_(R)3.3546 2.7241 2.9786 29porion_(L)2.55522.0471 1.7367 30gonion_(R)1.05210.9333 0.8245 31gonion_(L)0.9360 0.8330 1.5443 32alare_(R)2.09651.6396 1.5934 33alare_(L)2.03421.5304 1.4494 34lateral_nasal_(R)1.9751 1.4220 1.5541 35lateral_nasal_(L)2.09081.3537 1.3495 36nose_height4.1012 3.5995 4.5687 37bucal_(R)13.6992 11.2034 12.2837 38bucal_(L)13.9451 11.6959 11.7598