Automatic Discovery of Connections between Vietnamese’s Anthropometric Features – Master Thesis Dinh Quang Huy Iba Laboratory

Contents Background and Motivation 1 Our Approach 2 Evaluation and Result 3 Conclusion and Future Work 4 2

Facial Reconstruction  Recreating the face of an unidentified individual from their skeletal remains.  Forensic: determining the face of victims in the murders  Archaeology: verifying the remains of historic figures.  Anthropology: approximating the look of early hominid forms. 3

Facial Reconstruction Techniques  Three methods  2D reconstruction  Manual 3D reconstruction  Computer-aided 3D Reconstruction 4

2D Reconstruction  Requires a forensic artist who draws the picture  Usually requires cooperation between artist and forensic anthropologist  Reconstruction process  Tissue depth markers are fixed on the skull  Taking photograph  Draw the portrait 5

Manual 3D Reconstruction  Clay modeling approach.  Landmark dowels are placed on the predefined landmarks on the skull  The lengths of these dowels are the tissue depth at the dowel’s landmark.  Clay is then filled between the dowels to build up the tissue depth of the unknown person  Effectiveness of clay reconstruction depends wholly on the skill of the sculptor 6

Computer-aided 3D Reconstruction  Manual approaches have limitations such as time consuming and sculptor experience dependence  Most computer-aided 3D reconstruction systems use the same process  The skull is scanned to obtain its 3D model  Facial landmarks on the skull are detected or manually selected  Tissue depths at these landmarks are referenced  Based on these landmarks and tissue depths, the face is built by interpolation or morphing 7

Soft tissue thickness prediction  The critical issue is soft tissue thickness prediction  All known facial reconstruction systems collect tissue depths at predefined landmarks of a population and use the average value in the reconstruction.  There are some methods to collect tissue depths information  Invasive technique: needle technique  Non-invasive techniques: ultrasound, MRI, CT 8

Problem and Motivation  Lack of a suitable method for soft tissue thickness prediction  The models generated automatically by the systems are often modified manually by experts to have the final model.  Do facial features such as eye and nose shapes depend on the skull?  Many researchers have been working to prove that these features depend on the bones. 9

Contents Background and Motivation 1 Our Approach 2 Evaluation and Result 3 Conclusion and Future Work 4 10

Our Approach  We treat the soft tissue thickness prediction issue the missing data problem  The tissue depths are the target  The skull shape is the input  One solution for the missing data problem is to  Collect data of pairs of input and target  Apply machine learning techniques to obtain a model that best describes input-target relationship 11

Data Description: Target Data  Soft tissue thicknesses at facial landmarks  landmarks are believed to contain the most information about the skull shapes 12  Facial features: distance between 2 pupils, nose height, and nose length.

Data Description: Input Data  Distances between facial landmarks  These distances must be measured from the skull itself. In the case of facial reconstruction, only these distances are known 13

Data Description (cont) Input (17 fields)Target (38 fields) Cranial height Cranial width … Vertex thickness Nose length … Data …6.546… Data …5.843… Data … … ………………… Data … … 14 Need to predict these information These information can be measured from any discovered skull

Data Collecting  Available techniques  Invasive: needle technique  Non-invasive: ultrasound, MRI, CT  Needle technique can not be used on living people, so its accuracy is very low  Needle technique can not be used to measure distances between facial landmarks, either.  Among the non-invasive techniques, CT is the best option  CT image is in high quality and the distances between facial landmarks can be measured visually. 15

Data Collecting (cont) 16

Data Collecting (cont)  Collected data of 220 Vietnamese candidates  98 males, 122 females  Age varies from 17 to 82  Weight varies from 38kg to 75kg 17

Discovery of Relationships  Two methods  Linear Regression Fitting a linear equation to the observed data Can only discover one to one relationships  Artificial Neural Networks Non-linear nature Work in two modes, training and testing 18

Applying Linear Regression  Linear Regression is applied for all pairs of input-target  For each target field, the pair with best performance (in term of MSE) is chosen 19

Neural network approach  This structure can represent any functional relationship between inputs and outputs if the hidden layer has enough neurons  two-layer feed forward network  tan-sigmoid transfer function in the hidden layer  linear transfer function in the output layer  Every target field has its own neural network prediction model 20

Contents Background and Motivation 1 Our Approach Evaluation and Result 3 Conclusion and Future Work

Evaluation  Ten-fold cross validation  Linear Regression and Neural Network techniques were implemented  The results are compared with the ‘average method’ 22

Prediction Result 23

Reconstruction Result 24

Reconstruction Result (cont) 25

Contents Background and Motivation 1 Our Approach 2 Evaluation and Result 3 Conclusion and Future Work 4 26

Conclusions  Proposed an approach for automatic discovery of relationships between anthropometric features  Critical in facial reconstruction systems  Evaluation results show that the approach has better performance than the ‘average method’ which is used in most facial reconstruction systems  Although the data is collected from Vietnamese, the study can be applied for any race 27

Publications 1.Quang Huy Dinh, Thi Chau Ma, The Duy Bui, Trong Toan Nguyen, Dinh Tu Nguyen. “Facial soft tissue thicknesses prediction using anthropometric distances.” Studies in Computational Intelligence (351/2011), page , Springer, (2001) 2.Thi Chau Ma, Dinh Tu Nguyen, Quang Huy Dinh, The Duy Bui. “3D facial reconstruction system from skull for Vietnamese.” Proceedings of the 3rd International Conference on Knowledge and Systems Engineering, page , IEEE, (2011). 28

Linear equation lists 30 N#OUTPUTINPUTLINEAR EQUATIONMSE 1vertexcranial breadthy = x trichioncranial heighty = x glabellaforehead width (ft-ft)y = x nasioncranial heighty = x rhinionmolar-molary = x pronasalebase nose lengthy = x nose_lengthn-rhy = x subnasalebn-bny = x upper_lip_bordermolar-molary = x lower_lip_bordercranial heighty = x stomionbase facial length (ba-pr)y = x metalal-aly = x metonal-aly = x opisthooranional-aly = x exocanthion_(R)cranial heighty = x exocanthion_(L)cranial heighty = x endocanition_(R)cranial heighty = x endocantion_(L)cranial heighty = x pupil-pupilex-exy = x

Linear equation lists (cont) 31 N#OUTPUTINPUTLINEAR EQUATIONMSE 20supraobital_(R)en-eny = x supraobital_(L)al-aly = x infraobital_(R)cranial heighty = x infraobital_(L)cranial heighty = x zygomatic_arch_(R)n-rhy = x zygomatic_arch_(L)base cranial length (n-ba)y = x zygomatic_(R)base facial length(ba-pr)y = x zygomatic_(L)base facial length(ba-pr)y = x porion_(R)bn-bny = x porion_(L)bn-bny = x gonion_(R)al-aly = x gonion_(L)nasal projectiony = x alare_(R)al-aly = x alare_(L)al-aly = x lateral_nasal_(R)al-aly = x lateral_nasal_(L)al-aly = x nose_heightbase nose lengthy = x bucal_(R)bn-bny = x bucal_(L)bn-bny = x

MSE values for the three methods 32 N#OUTPUTAVGLRNN 1vertex trichion glabella nasion rhinion pronasale nose_length subnasale upper_lip_border lower_lip_border stomion metal meton opisthooranion exocanthion_(R) exocanthion_(L) endocanition_(R) endocantion_(L) pupil-pupil

MSE values for the three methods (cont) 33 N#OUTPUTAVGLRNN 20supraobital_(R) supraobital_(L) infraobital_(R) infraobital_(L) zygomatic_arch_(R) zygomatic_arch_(L) zygomatic_(R) zygomatic_(L) porion_(R) porion_(L) gonion_(R) gonion_(L) alare_(R) alare_(L) lateral_nasal_(R) lateral_nasal_(L) nose_height bucal_(R) bucal_(L)