2. -Sridhar Godavarthy

3. A Little Background: Blink A Lot More Background: Strain as a Soft Forensic Evidence Facial Recognition Culprits Human anatomy as a feature Strain Measurement Micro expression Detection using Strain Patterns Challenges Sample Strain patterns References

4. A Little Background: Blink A Lot More Background: Strain as a Soft Forensic Evidence Facial Recognition Culprits Human anatomy as a feature Strain Measurement Micro expression Detection using Strain Patterns Challenges Sample Strain patterns References

5. A Little Background

6. Why are some people brilliant decision makers? How do some people act upon instincts? Why are we unable to explain some decisions?

7. Great decision makers are not ones that process the most information Malcolm Gladwell’s ‘The statue that didn’t look right’ They are those who have perfected the art of “Thin Slicing” Filtering out the very few factors that matter.

9. A Little Background: Blink A Lot More Background: Strain as a Soft Forensic Evidence Facial Recognition Culprits Human anatomy as a feature Strain Measurement Micro expression Detection using Strain Patterns Challenges Sample Strain patterns References

10. V.Manohar, D.B.Goldgof, S.Sarkar,Y.Zhang Some slides have been adapted from the Authors’ presentation A Lot More Background

11. Face recognition has made huge advances Picasa’s Web Albums Sony’s “say cheese”( or is it CHEERS) detection “Almost” perfect Picasa still confuses between closely related faces Canon almost always never detects my face Some say - might be because of my hair ;-) Has anyone used the Lenovo Face ID? Because they use static images Could be supplemented for better performance.

12. Illumination Camouflage(Makeup/glasses) Facial Hair Expressions The Solution: Use methods based on Human Anatomy

13. Iris scan Retina scan Skull X-ray Disadvantage Require Specialized equipment Intrusive Proposed Alternative Skin and tissues of the face

14. Different materials have different elasticity Elasticity can be modeled Known Calculate Authentic Author Slide

15. What is Facial Strain? Strain on soft tissue when expressions are made. Anatomical method Uses a pair of frames to measure deformation

16. Why Facial Strain? As it is a difference, it is independent of all the earlier mentioned culprits(ICHE)

17. ‘Visual Pattern’ is unique to every face. Easily quantifiable by ‘elasticity’ Hard to measure – non-linear, inverse equations Can be represented by strain pattern under specific boundary conditions Is unique to a person.

18. Contact strain measurement equipment is already available. Cannot be used if we are looking to identify people at a Casino/Airport Did I mention the actual applications of this paper Soft forensics based on surveillance videos

19. Two major steps 1. Obtain motion field between two frames 2. Compute strain image from above Motion field.

20. Feature Based Need to identify features – Difficult! Features may be ill defined( when camouflaged) Usually requires manual intervention Produces a sparse motion field Produce Good correspondence in large motion Optical Flow based Fully automated Dense Motion field. Requires constant illumination

21. Observed motion over sequential image frames Adapted Author Slide

22. 3D Strain Ideal No high speed equipment available to capture range images 2D Strain Well – not much of a choice Authors could use existing data.

23. Variation of displacement values obtained from optical flow Calculated by taking the derivative of each pixel  Sobel operator (central difference) Authentic Author Slide

24. Finite Element Method Forward modeling when Dirichlet condition is satisfied Good at handling irregular shapes Computationally expensive This method is an approximation to the solution Finite Difference Method Strain, a tensor, can be expressed derivatives of the displacement vector This can be approximated by a Finite Difference Method. Very efficient when carried out on a regular grid. This method is an approximation to the differential equation

25. Finite Strain tensor Cauchy tensor

26. Motion is mostly vertical Strain pattern is dominated by its normal components The strain magnitudes are scaled to gray levels White = highest strain Black = lowest strain It is now a pattern matching problem.

27. Motion field : Based on Optical flow Strain Type: 2-D Computation: Finite Difference Method

29. Strain Magnitude is now 1-D Use PCA to perform matching

30. Experiments performed on Normal light Low light Shadow light Regular face Camouflaged face Frontal view Profile view Neutral expression Open mouth

31. Subject may not perform the expression to the same extent every time Experiments repeated on shorter, subsampled videos

32. Strain measurement seems to be logically correct We do not discuss the PCA and hence the recognition results as they are outside the scope of this discussion.( But they were good) Acts as a supplement to existing recognition methods.

33. A Little Background: Blink A Lot More Background: Strain as a Soft Forensic Evidence Facial Recognition Culprits Human anatomy as a feature Strain Measurement Micro expression Detection using Strain Patterns Challenges Sample Strain patterns References

35. Macro Expressions: Large movement Smile Talking Shaking head Micro expressions Raising eyebrow Fast blinking

37. Supplement lie detection Very little noise As part of a general discussion Bond might not have lost even the first time!

38. 12345n6 a. 1-2 b. 3-4 c. 1-4 d. 1-3 e. 4-6 f. 1-6FrameStrain a1-2100 b3-4200 c1-4300 d1-3200 e4-6200 f1-6400

39. Macro ExpressionMicro ExpressionNoise

40. Small movements are inevitable Macro expressions also possible Eyes always blink. Need to detect changes in speed of blinking Need to identify the frames to be used Solution: Normalize

41. V.manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a Soft Forensic Evidence", IEEE Workshop on Applications of Computer Vision (WACV'07),pp 42-42 Vasant Manohar, Matthew Shreve, Dmitry Goldgof and Sudeep Sarkar, "Finite Element Modeling of Facial Deformation in Videos for Computing Strain Pattern", International Conference on Pattern Recognition, Dec. 2008 Matthew A. Shreve, Shaun J. Canavan, Yong Zhang, John R. Sullins, and Rupali Patil, "Imaging And Characterization Of Facial Strain In Long Video Sequences",xxxx Malcolm Gladwell,” Blink: The Power of Thinking Without Thinking”, Back Bay Books (April 3, 2007)

42. Sridhar Godavarthy Dept. Of Computer Science and Engineering University of South Florida sgodavar@cse.usf.edu