1. Real Time Face Monitoring Ajesh Sen Thapa(04) Biraj Subedi(13) Diwas Poudel(16) Pravin Sangraula(32) Yogesh Gautam(48)

2. Introduction  This project deals with real time face detection and recognition.  Advanced topics In computer vision and pattern recognition.  Face Detection determines the face in the image.  Face Recognition recognize person whose face is in the image.  Real time face monitoring monitors face via video streaming through ip camera.

3. Objectives  To detect and recognize face using c#.  To provide advance security system.

4. Literature Review  New and hot topic for Nepal.  Very useful for security purposes.  Hjelmas and Low conducted feature based and image based approach on face detection and identification.  Feature based deals with various features of face.  Image Based deals with detecting multiple faces with cluster-intensive backgrounds.

5. HTTP Live Streaming  HTTP stands for Hypertext Transfer Protocol.  HTTP functions as a request-response protocol in the client-server computing model.  HTTP Live Streaming uses a conventional web server to distribute audio visual content and requires specific software to fit into the proper format transmission in the real time.

6. HTTP Live Streaming Architecture 1. Servers 1. Encoder(H.264,MP3) 2. Segmenter(MPEG-Ts) 2. Distributor -Accepts request and delivers resources. 3. Client -request and download all the files

7. Implementation of HLS Protocol for IP Camera

8. Stream segmented  Image/video data of IP is transmitted to the server through CGI  To support HLS protocol, it is required to do continuous generation of MPEG that is encoded using H.264 codec

9. Media encoder  The media encoder converts the generated raw H.264 stream file to a segmented MPEG-2.Ts file

10. Playlist Generation  The encoder continuously add the conversion for live streaming of HLS protocol.

11. Web server  The segmented media and the information of media to be played are transmitted to the client.  The HLS server receives a file transmission request from a client through a HTTP and provides a response.

12. Result of HLS streamming

13. Features of HLS  Adaptability Enables a player to adapt to the bandwidth of network.  Availability Master playlist can list duplicate variant streams hosted by different servers. So a player can swap to different servers which grants a degree of high availability to serving streams.  Using Fragmented MP4 Using fragmented mp4 files allows content to be played in HLS without the need to multiplex it into MPEG-2 Transport stream.

14. Implementation of HLS protocol for IP camera

15. Face Detection Face detection is a computer technology being used in a variety of applications that identifies human faces in digital images. Face detection is not face recognition.

16. Viola–Jones object detection  It is the first real time object detection method.  It can be trained to detect a variety of object.  It was motivated primarily by the problem of face detection

17. Characteristic  Robust – very high detection rate (true-positive rate) & very low false-positive rate always.  Real time – For practical applications at least 2 frames per second must be processed.  Face detection only (not recognition) - The goal is to distinguish faces from non-faces (detection is the first step in the recognition process)

18. Stage of Viola–Jones Detection Algorithm  Haar Feature Selection (Digital image feature)  Creating an Integral Image  Adaboost Training  Cascading Classifiers

19. Example of Haar-like Feature Haar value = ∑(pixel in white region) - ∑(pixel in black region) Haar like feature is a rectangular regions. The different of the pixel value in each region is used to detect subsection of an image.

20. How Haar-like Features works? Orginal image ∑pixel on black region = 16,672 ∑pixel on white region = 41,600 Haar value = 41,600 - 16,672= 24,928 ∑pixel on black region = 29,536 ∑pixel on white region = 32,864 Haar value = 32,864- 29,536= 3328

21. Integral Image

22. Integral Image Example Sum((1,1),(3,2)) = 7+8+9+12+13+14 = 63 Sum((1,1),(3,2)) = integral(3,2) + integral(0,0)- integral(0,2)-integral(3,0) =90+1-18-10 =63

23. Adaboost classifier General classifier uses all feature to classify which takes high computation time. But adaboost uses one feature to create “weak” classifier and combines it to create “strong” classifier

24. Cascade architecture Only 0.01% of window contains positive faces. 2 feature can detect with 50% false positive rate. So cascade architecture uses a layer of architecture with increasing features. 2 features23 features10 features

25. Face Recognition 1) A method of identifying a face as known or unknown based on the face present in database. 2) Principal Component Analysis with Eigen faces, 3) Linear Discriminate Analysis, 4) Elastic face matching, 5) Geometric approach.

26.  Why PCA ? 1) simplest,easiest 2) good to start 3) It works on noisy data Face of people has similar view,aren’t randomly distributed (less variation)

27. Implementation of PCA Algorithm for Face recognition  Training the Recognizer.  Recognizing an unknown face.

28. Training the Recogniter 1. Create training set of faces.  900 image and of 40*40 dimension

29. Each face vector will be of 1D denotes by as  2. Convert face image of training set to face vector

31. 3. Calculate average of face image. 4. Subtract average face vector image from each face vector. 5. Compute the Covariance matrix. (1600*900)*(900*1600) Reject (900*1600)*(1600*900)

32. 6. Compute Eigen vector for covariance matrix. PCA analysis Top 20 eigen face (for 900 faces) For 900 faces we get 900 eigen vector(V) and eigen value. 7.Getting Eigen faces will get 900 eigen face for equivalent original 900 faces of Dataset.

33. 8. Projection of training samples into Eigen face space.

34. PCA for recognizing unknown face  Step 1: Input the image of the unknown person.  Step 2: Convert the input image to a face vector

35.  Step 3 :Normalize this face vector  Step 4: Represent the face as a weight vector of the input image. There are k weights associated with the k Eigen vectors.

36.  Step 7 : Calculate distance between the weight vector of the input image and the weight vector of the training set. For this we have used Euclidean Distance(D).  So, we will get total of 900 distance value. The smallest distance is considered to be the face match result.

38. Limitation of Eigen Faces Approach  Variation in lighting condition.  Differences in Pose.  Expression  Covariance matrix is difficult to be calculated efficiently.

39. Database Structure  SQLite Database doesn’t require server to store the information so it was easily implemented in C#.  Our Application stores details of a person like name, level etc. Person Status Path Level Name ID Fig: ER Diagram for Person entity with its attribute

40. Log Date Hour Minute Time Year User Month Fig: E-R Diagram for Log Table with its attributes

41. Proposed System IP Camera(mobile) HTTPVideo MJPEGFrame SelectionFace DetectionRecognition Data Base

42. Features of our Application  Allows users to save person after recognizing faces with name,level and status.  Keeps the log record of each detected faces and display according to the time.  Enables users to edit their profile.  Monitors Face with remote camera.

43. Software platform  C#  SQLite database  Visual Studio

44. References  An Overview of Principal Component Analysis Sasan Karamizadeh1, Shahidan M. Abdullah1, Azizah A. Manaf1, Mazdak Zamani1, Alireza Hooman2  Department of Computer Science and Engineering National Institute of Technology Rourkela Rourkela – 769008, IndiaFace Recognition Using PCA and Eigen Face Approach by Abhishek Singh and Saurabh Kumar  http://fewtutorials.bravesites.com  https://en.wikipedia.org/wiki/Facial_recognition_and_detection  https://en.wikipedia.org/wiki/Haar-like_feature  https://onionesquereality.wordpress.com  M. Turk and A. Pentland, "Eigenfaces for recognition ", J Cog.  Neuroscience, vol. 3, no. I, pp. 71-86, 1991.  Papageorgiou, Oren and Poggio, "A general framework for object detection", International Conference on Computer Vision, 1998.  Viola and Jones, “Robust real-time object detection”, IJCV 2001.

45. Thank you….