Hu Li Moments for Low Resolution Thermal Face Recognition By, Naser zaeri Arab open university UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

Outline Introduction Hu Li Moments Experimental Work Conclusion UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

Introduction [1/3] Visible-spectrum images have high variability because they are produced by reflection in surfaces has strong dependence on luminosity spatial distribution of the light sources Light reflected from human faces also varies depending on the skin colour of people from different ethnic groups.  This may cause great difficulties in recognizing the face in applications such as outdoor surveillance tasks. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

Introduction [2/3] Recently, researchers have investigated the use of thermal imagery for face recognition with good results. In thermal imagery of human tissue (including the face), where two objects with different temperatures are in contact (e.g. vessel and surrounding tissue), heat conduction creates a smooth temperature gradient at the common boundary. The anatomical information imaged by this technology involves features believed to be unique to each person and can be considered as a “discriminant fingerprint.” UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

Introduction [3/3] Advantages: Eye is not sensitive in IR range. Illumination can be used in a more flexible way. Not vulnerable to facial skin and expressions. However, limited work has focused on adapting the technology to low resolution applications. In real situations, things become more complicated. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

HU LI MOMENTS [1/3] In this work, we propose a method that describes the thermal face images by a set of measurable quantities called invariants. It is important to choose the proper class of invariants. Invariance to translation can be achieved by shifting the class such that its centroid coincides with the origin of the coordinate system. In this case, we have the central geometric moments 𝜇 𝑝𝑞 = 𝑥=0 𝑀−1 𝑦=0 𝑁−1 (𝑥− 𝑥 ) 𝑝 (𝑦− 𝑦 ) 𝑞 𝑓(𝑥,𝑦) Note: f(x, y) is the image function and M, N are image dimensions, and 𝑥 =m10/m00 , 𝑦 =m01/m00 are the coordinates of the class centroid. And that: 𝑚 𝑝𝑞 = 𝑥=0 𝑀−1 𝑦=0 𝑁−1 𝑥 𝑝 𝑦 𝑞 𝑓(𝑥,𝑦)

HU LI MOMENTS [2/3] Scaling invariance is obtained by proper normalization of each moment. Researchers normalize most often by a proper power of μ00 : 𝜂 𝑝𝑞 = 𝜇 𝑝𝑞 µ 00 𝜔 , 𝜔= 𝑝+𝑞 2 +1 where 𝜂 𝑝𝑞 is called normalized central geometric moment. A further advanced and more complicated invariant moments are the Hu Li invariants, where they attract great attention for object description and consecutive classification. Therefore, we decided to describe every thermal face image with a feature vector 𝐚 consisting of the 12 Hu Li invariants. That is, the feature vector 𝐚 is given by: 𝐚=[ 𝜑 1 , 𝜑 2 , 𝜑 3 , 𝜑 4 , 𝜑 5 , 𝜑 6 , 𝜑 7 , 𝜑 8 , 𝜑 9 , 𝜑 10 , 𝜑 11 , 𝜑 12 ] UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

HU LI MOMENTS [3/3] Where, 𝜑 1 = 𝜂 20 + 𝜂 02 𝜑 2 = ( 𝜂 20 − 𝜂 02 ) 2 +4 𝜂 11 2 𝜑 3 = ( 𝜂 30 −3 𝜂 12 ) 2 +(3 𝜂 21 − 𝜂 03 ) 2 𝜑 4 = ( 𝜂 30 + 𝜂 12 ) 2 +( 𝜂 21 + 𝜂 03 ) 2 𝜑 5 = 𝜂 30 −3 𝜂 12 𝜂 30 + 𝜂 12 [( 𝜂 30 + 𝜂 12 ) 2 −3 𝜂 21 + 𝜂 03 ) 2 + 3 𝜂 21 − 𝜂 03 𝜂 21 + 𝜂 03 [3( 𝜂 30 + 𝜂 12 ) 2 − 𝜂 21 + 𝜂 03 ) 2 𝜑 6 = 𝜂 20 − 𝜂 02 ( 𝜂 30 + 𝜂 12 ) 2 −( 𝜂 21 + 𝜂 03 ) 2 +4 𝜂 11 ( 𝜂 30 + 𝜂 12 )( 𝜂 21 + 𝜂 03 ) 𝜑 7 = 3 𝜂 21 −𝜂 03 𝜂 30 + 𝜂 12 [( 𝜂 30 + 𝜂 12 ) 2 −3 𝜂 21 + 𝜂 03 ) 2 − 𝜂 30 − 3𝜂 12 𝜂 21 + 𝜂 03 [3( 𝜂 30 + 𝜂 12 ) 2 − 𝜂 21 + 𝜂 03 ) 2 𝜑 8 = 𝜂 40 + 𝜂 22 + 𝜂 04 𝜑 9 = ( 𝜂 40 − 𝜂 04 ) 2 +4 ( 𝜂 31 − 𝜂 13 ) 2 𝜑 10 =( 𝜂 40 −6 𝜂 22 + 𝜂 04 ) 2 +16 ( 𝜂 31 − 𝜂 13 ) 2 𝜑 11 =( 𝜂 40 −6 𝜂 22 + 𝜂 04 ) 2 ( 𝜂 40 − 𝜂 04 ) 2 +4 ( 𝜂 31 − 𝜂 13 ) 2 +16 𝜂 40 − 𝜂 04 +( 𝜂 31 + 𝜂 13 )( 𝜂 31 − 𝜂 13 ) 𝜑 12 =( 𝜂 40 −6 𝜂 22 + 𝜂 04 ) 2 𝜂 40 − 𝜂 04 2 +4 𝜂 31 − 𝜂 13 2 −16 𝜂 40 − 𝜂 04 +( 𝜂 31 + 𝜂 13 )( 𝜂 31 − 𝜂 13 )

EXPERIMENTAL WORK [1/5] We have built a database consisting of 1500 frontal images for 20 different subjects taken in different sessions. Participants were asked to express three different emotions with their faces: neutral, anger, and smiling. We evaluated the technique on images with four different resolutions of probe thermal images, beside the original (300) high resolution images. The original high resolution images are of size 180×160. Images at a particular resolution are obtained by down-sampling the original images to the required resolution (namely, 90×80, 45×40, 22×20, and 18×16). We end up with 5 space resolutions × 300 images per resolution, giving a total of 1500 images. All subsets contain random images with eyeglasses.

EXPERIMENTAL WORK [2/5] We refer to a 90×80 image as an image of 0.5 resolution, as the original resolution image has been down-sampled by 50% (of its rows and columns.) Similarly, the 45×40, 22×20, and 18×16 have been referred to as 0.25 (quarter), 0.125 (one-eighth), and 0.1 (one-tenth) resolution, respectively. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

EXPERIMENTAL WORK [3/5] The face image is first divided into components where the local characteristics and features are obtained. Then, each moment is trained on a determined cluster (component) of thermal images in the training database. In our work, the image is divided into 16 equal-size and non-overlapped components. Below, an example of a 16-component thermal image at 0.25 resolution. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

EXPERIMENTAL WORK [4/5] After that, the results obtained from the similarity measures from the feature vectors for the different components are fused together at a second stage using “voting” fusion to achieve the final similarity score and decide whether the input face image belongs to a given class. Figure on the right shows a 3D temperature distribution for a16-component scheme. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

EXPERIMENTAL WORK [5/5] As the results reveal, the technique has shown impressive and stable performance. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

CONCLUSION In this work, we presented a new technique for low resolution face recognition by the virtue of thermal face characteristics and using Hu Li moment invariants. The technique has shown remarkable performance with an average recognition rate of ~94% at the various resolutions. Our future work will consider different resolutions at various poses. UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018

Thank you .. Any Questions? UKSim-AMSS 20th International Conference on Modelling & Simulation Cambridge University, 27 - 29 March 2018