Human Activity Recognition, Biometrics and Cybersecurity Mohamed Abdel-Mottaleb, Ph.D. Image Processing and Computer Vision Department of Electrical and Computer Engineering University of Miami

Human Activity Recognition

Results Sampled frames from a sequence with several activities We collected 81 video sequences of single activities for training HMMs. The algorithm has been tested on a set of sixteen sequences of continuous complex activities consisting of totally 66 single activities. 59 single activities were properly segmented and recognized from the sixteen video clips. This means that our system has an accuracy rate of 89%. The results demonstrate the efficiency of the algorithm with respect to segmentation and recognition voting results for the complex activity

Pathologic Gait Identification Pathologic Gait Identification

Data collection Camera positions within the reconstruction volume A motion-capturing system composed of eight M-Cam cameras (Vicon 512, Vicon Motion Systems, Lake Forest, CA) that record the spatial positions of the markers throughout the whole gait cycles. A motion-capturing system composed of eight M-Cam cameras (Vicon 512, Vicon Motion Systems, Lake Forest, CA) that record the spatial positions of the markers throughout the whole gait cycles.

Data collection Marker set used in the current study we are interested in assessing the walking gait based on motion symmetry, we just use the data from symmetrical markers for our purpose. We also use the data from marker CLAV in our work. we are interested in assessing the walking gait based on motion symmetry, we just use the data from symmetrical markers for our purpose. We also use the data from marker CLAV in our work.

Biometrics The use of unique human characteristics to positively identify a person The use of unique human characteristics to positively identify a person Biometrics has attracted much attention due to its potential in a broad range of applications Biometrics has attracted much attention due to its potential in a broad range of applications Biometrics offers reliable technique to answer the enormous need in society Biometrics offers reliable technique to answer the enormous need in society 7

8 Fully Automated Face Normalization and Single Sample Face Recognition in Unconstrained Environments

9 Our proposed face normalization technique. (a) Estimated landmarks using the flexible mixture of trees. (b) Triangularization of the initial mesh created by the estimated landmarks. (c) Fitting active appearance model on the face. (d) Result of the piecewise affine warping into the frontal mesh. Single Sample Face Recognition in the Wild We proposed a pose normalization technique that synthesizes a frontal face from rotated faces.

10 Single Sample Face Recognition in the Wild Experimental Results on the LFW Database Publications: M. Haghighat, M. Abdel-Mottaleb, “Fully Automatic Face Normalization and Single Sample Face Recognition in Unconstrained Environments,” Expert Systems With Applications, vol. 47, pp , (Impact Factor: 2.57)

Ear Recognition 11

Shape from Shading 12  Shape from shading is ideal for 3D surface reconstruction of texture-less, quasi-Lambertian objects such as the ear.

Gender Recognition from Ear Images Ear Detection Ear Detection Feature Extraction: Gabor Wavelets Feature Extraction: Gabor Wavelets Feature Reduction: PCA Feature Reduction: PCA Building The Dictionary (Matrix A) Building The Dictionary (Matrix A) Sparse Representation Sparse Representation Majority Voting Majority Voting 13 R. Khorsandi and M. Abdel-Mottaleb, “Ear Biometrics and Sparse Representation based on Smoothed L0 Norm”, Accepted in International Journal of Pattern Recognition and Artificial Intelligence, 2014

14 Cloud-Based Privacy-Preserving Biometric Identification (CloudID) Biometrics as a key attribute: Advantages: high level of security precise identification not being transferable mobility impossibility to forge user friendliness Face as a biometric: Additional Advantages: no need for subject’s interaction no need for additional hardware can be done from long distance and anonymously applications in surveillance systems Cons: changes in illumination, pose, expression, image occlusion changes by aging similarities between people irretrievable identity theft CloudID uses a k-d tree structure to create encrypted search queries. Applying this structure in the core of a searchable encryption technique helps the system not only to quantize the biometric features but also to handle the variations in the biometric data.

15 Trusted Party Cloud DB Query(TK,C) Anti- Spoofing Face Detection Feature Extraction Dimensionality Reduction Quantization (k-d tree) Generate Token ( TK ) Create Encrypted Predicate Cloud Cloud-Based Privacy-Preserving Biometric Identification (CloudID) CloudID is the first cloud-based biometric identification system. It links the confidential information of the users to their biometrics and stores it in an encrypted fashion. Making use of a searchable encryption technique, biometric identification is performed in encrypted domain to make sure that the cloud provider or potential attackers do not gain access to any sensitive data or even the contents of the individual queries. M. Haghighat, S. Zonouz, M. Abdel-Mottaleb, "CloudID: Trustworthy Cloud-based and Cross-Enterprise Biometric Identification", Expert Systems With Applications, 42, pp , June 2015 (selected as one of the top 26 biometrics research papers across Elsevier in )

Medical Applications Nursing Applications? Nursing Applications? Home Personal Assistant Home Personal Assistant Helping the Visually Impaired Helping the Visually Impaired Medical Applications Medical Applications Psychological assessment of patients from video clips showing their interactions Psychological assessment of patients from video clips showing their interactions Study of Autistic Children from Video Study of Autistic Children from Video