1. Robodog Frontal Facial Recognition AUTHORS GROUP 5: Jing Hu EE ’05 Jessica Pannequin EE ‘05 Chanatip Kitwiwattanachai EE’ 05 DEMO TIMES: Thursday, April 21st, 2005 9:30AM, 10AM, 1PM and 1:30PM Skin Color Analysis ADVISORS: Dr. Daniel Lee Dr. Nabil Fahrat Gray Scale Image First Crop Feature Extraction Second Crop Rotation Trained Neural Network Picture from Dog Facial Database Recognition Algorithm Dog Reaction Minimum Error University of Pennsylvania Department of Electrical and Systems Engineering ABSTRACT: Facial Recognition has drawn a significant amount of attention in the research area in the past few years. There is an increasing interest in the implementation of facial recognition systems because of the emerging demands of more efficient security systems. The ability to take into account differences in lighting conditions, facial orientation and background objects is crucial for the implementation of a successful system. Many different approaches of the problem have been developed over the past two decades. So far, each proposed method has different comparative advantages and disadvantages. With the chosen approach of this project, the face region is first extracted from the original picture using skin color analysis. The facial features are then generated from the face region. By doing so, the background noise can be eliminated, thus increasing the recognition accuracy and decrease the computation volume of the system. The facial features are then fed into a neural network to overcome image distortion due to lighting condition, facial expression and orientation of the face. Finally, in order to enhance the role of human-robot interaction for which recognition is a crucial capability, the Sony’s Aibo Dog is used as the interface to the system. Recognition Results Feature Vector Three most prominent SIFT Features of random picture from database Neural Network Pass features through the Network once and adjust the weights Wij and Wjk Final Weights Wij and Wjk Repeat for 150,000 iterations After all iterations Neural Network Training Convergence of Neural Network weights during training For each person, 400 pictures are stored in a database, with different facial orientations or expressions. At each iteration, a picture is taken at random from the database and passed once through the neural network. The network is trained over 150,000 iterations, which allows on average each picture to be trained 75 times. First Image: Min Error1 < 0.005? Second Image: Min Error2 < 0.005? Third Image: Min Error3 < 0.005? Yes No Yes Min Error1 No Recognition Yes Recognition Min Error belongs to the same person as in 1 st Image? No Yes Recognition Yes Min Error belongs to the same person as in 1 st or 2 nd Image? No Start over with new group of pictures Min Error2 Min Error3Does Min belong to pair of pictures? Take Minimum of 3 Errors No Is Min Error of single Picture < 0.05? Yes Sample Set Recognition No Yes Recognition Algorithm a group of pictures is analyzed. At each stage, a measure of the minimal error helps determine if an immediate conclusion is possible or whether an additional picture is required. The neural network outputs five errors, each measuring the level of similarity with one of the individuals trained by the network. The lower the error, the closer the person is to one of the trained individuals. Therefore, the smallest error indicates who was recognized in a given picture. In order to improve the robustness of the system, the following algorithm is implemented. Instead of determining recognition based on a single picture, Graphical User Interface JPJHCHFRSG 95% 88% 85% 84% 86% 88%95%85%84% 87.6% Rate of Success Convergence of 1 element in the Wij matrix Neural Network Outline with three layers Convergence of 1 element in the Wjk matrix Convergence of 1 element in the Delta Wij matrix Convergence of 1 element in the Delta Wjk matrix Note: The Delta Wij and Delta Wjk matrices correspond to the adjustment to the Weights performed at each iteration. After successful training, these numbers should converge to zero, as shown on the above graph. Special Thanks to: Paul Vernaza