1. LOGO On Person Authentication by Fusing Visual and Thermal Face Biometrics Presented by: Rubie F. Vi ñ as, 方如玉 Adviser: Dr. Shih-Chung Chen, 陳世中

2. www.themegallery.com Company Logo Outline I.Abstract II.Introduction III.Method Overview IV.Empirical Evaluation V.Conclusion

3. www.themegallery.com Company Logo I.Abstract  Recognition Algorithms  Use data obtained by imaging faces in the thermal spectrum are promising in achieving invariance to extreme illumination changes that are often present in practice.  Paper  Analyze Performance –Recently: Proposed face recognition algorithm » Combination: Visual and Thermal modalities by “Decision Level Fusion”.  Examine: Effects of the proposed data preprocessing in each domain. Contribution to improved recognition of different types of features Importance of prescription glasses detection –Recognition Vs. Verification »1-to-N matching »1-to-1 matching  Discuss: Significance of the results –Identify a number of limitations of the current state-of-the-art face recognition algorithm. –Propose promising directions for future research.

4. www.themegallery.com Company Logo II.Introduction  Optical Imager  Sensor used in face biometric systems. Why? –Availability –Low-cost  Function Captures the light reflectance of the face surface in the visible spectrum. Visible Spectrum  Problem Ambient Light

5. www.themegallery.com Company Logo II.Introduction (cont.)  Advantages of Thermal Spectrum Vs. Visible Spectrum  Invariance  Ambient Illumination changes Thermal Infrared Sensor –Measures heat energy radiation »Emitted by the face Appearance-based face recognition algorithms –Applied to thermal IR imaging –Consistent Performance »Under various lighting conditions and facial expressions. Visual Spectrum Thermal Spectrum

6. www.themegallery.com Company Logo III.Method Overview System Overview 1. Data Preprocessing and Registration 2. Glasses Detection 3. Fusion

7. www.themegallery.com Company Logo

8. www.themegallery.com Company Logo III.Method Overview (cont.)  Set-up Matching (Feature Fusion)  Similarity Score Two images using only a single modality (visual or thermal) Constant-Weighted Summation: –Where: »p v  visual score »p t  thermal score »p m  mouth score »p e  eyes score »p h  holistic score »w m  mouth weighing constant Visual Spectrum  0.0Thermal Spectrum  0.1 »w e =1-w h -w m  eyes weighing constant »w h  holistic weighing constant Visual Spectrum  0.7Thermal Spectrum  0.8 Note:Choice of values are recommendations from the Arandjelovic System

9. www.themegallery.com Company Logo III.Method Overview (cont.)  Set-up Matching (Modality Fusion)  Similarity Score Visual and Thermal data Joint Similarity  Where: p f  Fusion score w v =w v (p v )  Visual weighing function w t  Thermal weighing function

10. www.themegallery.com Company Logo III.Method Overview (cont.)  Visual Weighing Function Estimation w v =w v (p v ) i.Estimate –Probability that w v is the optimal weighting given –Iterative Algorithm –Unknown Image vs. Offline Gallery ii.Compute w(p v ) –Maximum Posteriori iii.Make an analytic fit –Obtain marginal Distribution

11. www.themegallery.com Company Logo III.Method Overview (cont.)  Prescription Glasses  Thermal Image exact opposite of the Visual Image Thermal Spectrum –Dark patches  Practical Importance US (year 2000) ~ 96 million people (34% total population) – Wear prescription glasses  System Appearance Distortion that glasses cause in thermal imagery –Help recognize by detecting their presence. –Subject »Not wearing glasses  use all parameters »Wearing Glasses  w e =0 (thermal image)

12. www.themegallery.com Company Logo IV.Empirical Evaluation  Old System to Evaluate  Dataset 02: IRIS Thermal/Visible Face Database Subset of the Object Tracking and Classification Beyond the Visible Spectrum (OTCBVS) database1 Available for download –http://www.cse.ohio-state.edu/OTCBVS-BENCH/http://www.cse.ohio-state.edu/OTCBVS-BENCH/ Contents –29 individuals »11 roughly matching poses *Visual and Thermal spectra and large illumination variations

13. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.)  Algorithm  Trained Use all images – Single illumination – 3 Salient facial features could be detected  Result 7-8 Visual Images 6-7 Thermal Images

14. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.)  After evaluation  Needs Performance improvement Solution –Band-pass and Self-Quotient Image Filters  Recognition Performance Use –Individual local features and fusion with holistic face appearance  Importance Prescription Glasses

15. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.)  Algorithm Performance  Evaluate 1-to-N matching scenario –Test data »One of the people in the training set –Recognition »Associating it with the closest match 1-to-1 matching scenario (Verification) –To know if the person is the same –Performance was quantified »Looking at the true positive admittance rate »Threshold  1 admitted intruder in 100

16. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.)  Results Summary of 1-to-N matching results Poor Performance Improved Performance (35%) Improved Performance (47%)

17. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.)

18. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.) Holistic representations Receiver-Operator Characteristics (ROC) Visual (blue) and thermal (red) spectra.

19. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.)  Low recognition rate of Raw Thermal Imagery  Two main limitations of this modality Inherently lower discriminative power Occlusions caused by prescription glasses –Addition:Glasses Detection Module Little help Benefit *Steering away from misleadingly good matches between any two people wearing glasses Limitation * Discriminative region of the face is lost. Modest Improvement  Optimal band-pass filtering in thermal imagery

20. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.) Isolated local features Receiver-Operator Characteristics (ROC) Visual (blue) and thermal (red) spectra

21. www.themegallery.com Company Logo IV.Empirical Evaluation (cont.) Glasses detection results Inter- and intra- class similarities

22. www.themegallery.com Company Logo V.Conclusion  Analyzed and Empirically evaluated  Recently proposed system Personal identification –Face biometrics from visual and thermal imagery  Results  Filters can be used Greatly improve recognition accuracy in both domains  Little improvement is seen Inclusion of local feature-based patches  Proposed Algorithm  Detection of glasses Very reliable across individuals and different imaging conditions  Fusion of visual and thermal imagery Very promising in practical applications Two Modes –Offer discriminative power in complementary ways

23. www.themegallery.com Company Logo V.Conclusion (cont.)  Verification Setup  Evaluated method 80% correct admittance rate (1% intruder admittance) –Problem »Much lower rate than that required in most practical applications *Making the system useful only in the pre-screening process  Much better performance was achieved in 1-to-N matching evaluation High correct identification rate of 97% was obtained –Using »Small number of training images  5-7 » Presence of large illumination changes

24. www.themegallery.com Company Logo V.Conclusion (cont.)  Recommendations  Further use of the thermal spectrum Segmenting out the glasses –Holistic appearance can still be used in matching »Challenge: Presence of extreme poses *Glasses merge with the background with more profile views  Different representation of local appearance Possibly offer further benefit with large pose changes

25. www.themegallery.com Company Logo THANK YOU!!!