1. 以區域二元圖樣與部分比對為基礎之 人臉辨識 Face Recognition with Local Binary Patterns and Partial Matching Presenter : 施佩汝 Advisor : 歐陽明 教授 1

2. Outlines Motivation Implementation Result Conclusion 2

3. MOTIVATION 3

4. Motivation 4

5. Publication Che-Hua Yeh, Pei-Ruu Shih, Kuan-Ting Liu, Yin-Tzu Lin, Huang-Ming Chang, Ming Ouhyoung. A Comparison of Three Methods of Face Recognition for Home Photos. ACM Siggraph, poster, 2009. 5

6. Problem Statement 6

7. Main Contribution Improve Local Binary Patterns by using Partial Matching Metric Better Performance in Home Photos 7

8. IMPLEMENTATION 8

9. System Overview Pre-Processing Build Descriptor Images Descriptors Clustering Calculate LBP Build Descriptor 9

10. System Overview Pre-Processing Build Descriptor Images Descriptors Clustering Calculate LBP Build Descriptor 10

11. Pre-Processing 11

12. System Overview Pre-Processing Build Descriptor Images Descriptors Clustering Calculate LBP Build Descriptor 12

13. System Overview Pre-Processing Build Descriptor Images Descriptors Clustering Calculate LBP Build Descriptor 13

14. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 589921 54 86 671213 1 14

15. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 11 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 15

16. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 16

17. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 1 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 17

18. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 1 0 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 18

19. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 1 00 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 19

20. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 1 100 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 20

21. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 11 100 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 21

22. Local Binary Patterns [PAMI2006] An operator to encode the relationship of a pixel and its neighbors. 589921 54 86 671213 110 11 100 Z1Z1 Z2Z2 Z3Z3 Z8Z8 Z0Z0 Z4Z4 Z7Z7 Z6Z6 Z5Z5 LBP = 11010011 22

23. System Overview Prepared-Works Build Descriptor Images Descriptors Clustering Calculate LBP Build Descriptor 23

24. Facial Image Descriptor They use Spatially Enhanced Histogram in original Local Binary Pattern. [PAMI2006] 24

25. Local Patches We sample a patch for every s pixels. There are S patches for one image. ss 25

26. Spatial Block [CVPR2007] We use three concentric circles to describe a patch. 26

27. Descriptor Build a descriptor for one face. 27

28. System Overview Pre-Processing Build Descriptor Images Descriptors Clustering Calculate LBP Build Descriptor 28

29. System Overview Pre-Processing Build Descriptor Images Descriptors Complete-Linkage Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering 29

30. System Overview Pre-Processing Build Descriptor Images Descriptors Complete-Linkage Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering 30

31. Similarity They use the weighted Chi-Square distance in original Local Binary Pattern. [PAMI2006] 31

32. Partial Matching [ICCV2009] Step1: – Compute the similarity of each patch from one image with the nearby patches in another image., Image 1: I (1) Image 2: I (2) 32

33. Partial Matching [ICCV2009] Step2: – Sort the similarities of all patches. – d αS is the similarity of I (1) to I (2). 33

34. Partial Matching [ICCV2009] Step3: – Calculate the similarity of I (2) to I (1) 34

35. Partial Matching [ICCV2009] Step4: – Use the maximum of two similarity 35

36. System Overview Pre-Processing Build Descriptor Images Descriptors Complete-Linkage Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering 36

37. Hierarchical Clustering Build a tree-based hierarchical taxonomy (dendrogram) from a set of documents. Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008. 37

38. Hierarchical Clustering Clustering obtained by cutting the dendrogram at a desired level: each connected connected component forms a cluster. Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008. 38

39. Hierarchical Complete-Linkage Clustering Similarity of the “furthest” points. Makes “tighter,” spherical clusters that are typically preferable. Material Selected from Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, 2008. 39

40. Performance Optimization 4 threads in Quad-Core system Pre-Processing Build Descriptor Images Descriptors Complete-Link Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering 40

41. Performance Optimization 4 threads in Quad-Core system – 3 times faster than single thread. 73 minutes to 24 minute for 309 images. Pre-Processing Build Descriptor Images Descriptors Complete-Link Clustering Calculate LBP Build Descriptor Compute all the similarities Clustering 41

42. RESULT 42

43. FERET Result fa:gallery, 994 images fb: alternative facial expression, 992 images dup1: the photos taken after later, 736 images dup2: the photos taken at least one year after the gallery, 228 images 43

44. FERET Result 44 Accuracyfbdup1dup2 LBP [PAMI2006] 95.67%59.92%45.61% Our Result98.89%71.33%68.42% Time ※ Registerfbdup1dup2 LBP [PAMI2006] 46 seconds102 seconds67 seconds21 seconds Our Result4 minutes224 minutes168 minutes5 minutes ※ The time results are computed in multithreads version.

45. Experiments Home Photo Dataset I – 309 images, 5 subjects Home Photo Dataset II – 838 images, 8 subjects 45

46. Evaluation Cluster Number Unknown Number Pair-wise Precision Pair-wise Rand Index Executing time 46

47. Unknown Number The number of clusters which contain only one component. 47

48. Precision/Rand Index 48 Assigned Same Cluster Different Clusters Ground Truth Same Cluster tpfn Different Clusters fptn

49. Dataset-I Result #Clusters#UnknownPrecision Rand Index Time ※ Picasa Web947399.92%0.82981610 seconds Picasa PC9975100%0.8650023 minutes LBP [PAMI2006] 1003190.39%0.8118564 seconds LID_PM [ICCV2009] 993798.78%-11 minutes Our Result1003999.46%0.81629024 minutes LID+PM [Chang2010] 1004399.24%-25 minutes LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block ※ The time results are computed in multithreads version. 49

50. Dataset-I Result 50 LBPOur Result

51. Dataset-I Result Wrong clustering result in LBP The clusters in our result 51

52. Dataset-II Result #Clusters#UnknownPrecision Rand Index Time ※ Picasa Web1954799.49%0.87622910 seconds Picasa PC253150100%0.88849210 minutes LBP [PAMI2006] 2534591.70%0.7026117 seconds LID_PM [ICCV2009] 2537997.88%-51 minutes Our Result2536499.59%0.871187163 minutes LID+PM [Chang2010] 2538999.82%-136 minutes LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block ※ The time results are computed in multithreads version. 52

53. Dataset-II Result 53 LBPOur Result

54. Dataset-II Result Wrong clustering result in LBP Clusters in our result 54

55. Demo Face Recognition with web camera 55

56. CONCLUSION 56

57. Conclusion LBP is an efficient algorithm for face recognition. Partial Matching is good for the different facial expression or different illumination in facial images. Our system has better performance than LBP. 57

58. Future Works Improve performance by GPU. Use other extension of LBP combined with Partial Matching. 58

59. Thank You for Your Attention! 59

60. Result LBP: Local Binary Pattern, PM: Partial Matching, SB: Spatial Block 60