- Sridhar Godavarthy. Expressions Microexpressions FACS Evolutionary Psychology Proposed method Outline Video Face Detection, Alignment and Splitting.

Slides:



Advertisements
Similar presentations
+ Sridhar Godavarthy July 01, 2010 Defense of a Masters Thesis Computer Science and Engineering University of South Florida.
Advertisements

Activity Recognition Aneeq Zia. Agenda What is activity recognition Typical methods used for action recognition “Evaluation of local spatio-temporal features.
Computer Vision Optical Flow
A New Block Based Motion Estimation with True Region Motion Field Jozef Huska & Peter Kulla EUROCON 2007 The International Conference on “Computer as a.
3D Computer Vision and Video Computing 3D Vision Topic 4 of Part II Visual Motion CSc I6716 Fall 2011 Cover Image/video credits: Rick Szeliski, MSR Zhigang.
Announcements Quiz Thursday Quiz Review Tomorrow: AV Williams 4424, 4pm. Practice Quiz handout.
Optical Flow Methods 2007/8/9.
May 2004Motion/Optic Flow1 Motion Field and Optical Flow Ack, for the slides: Professor Yuan-Fang Wang Professor Octavia Camps.
Announcements Project1 artifact reminder counts towards your grade Demos this Thursday, 12-2:30 sign up! Extra office hours this week David (T 12-1, W/F.
Object Detection and Tracking Mike Knowles 11 th January 2005
Visual motion Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys.
CS 6825: Motion Part 2 – Optical Flow Recall: Optical Flow is an approximation of the 2D motion field. is an approximation of the 2D motion field. Motion.
Motion Computing in Image Analysis
Optical Flow Estimation
Visual motion Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys.
Motion Estimation Today’s Readings Trucco & Verri, 8.3 – 8.4 (skip 8.3.3, read only top half of p. 199) Numerical Recipes (Newton-Raphson), 9.4 (first.
Linearizing (assuming small (u,v)): Brightness Constancy Equation: The Brightness Constraint Where:),(),(yxJyxII t  Each pixel provides 1 equation in.
COMP 290 Computer Vision - Spring Motion II - Estimation of Motion field / 3-D construction from motion Yongjik Kim.
Image Primitives and Correspondence
3D Rigid/Nonrigid RegistrationRegistration 1)Known features, correspondences, transformation model – feature basedfeature based 2)Specific motion type,
Matching Compare region of image to region of image. –We talked about this for stereo. –Important for motion. Epipolar constraint unknown. But motion small.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
Optical Flow Digital Photography CSE558, Spring 2003 Richard Szeliski (notes cribbed from P. Anandan)
3D Computer Vision and Video Computing 3D Vision Topic 8 of Part 2 Visual Motion (II) CSC I6716 Spring 2004 Zhigang Zhu, NAC 8/203A
Motion and optical flow Thursday, Nov 20 Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys, S. Lazebnik.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Irfan Essa, Alex Pentland Facial Expression Recognition using a Dynamic Model and Motion Energy (a review by Paul Fitzpatrick for 6.892)
My Project Title -Sridhar Godavarthy Contents A Little Background: Blink A Lot More Background: Strain as a Soft Forensic Evidence – Facial Recognition.
TP15 - Tracking Computer Vision, FCUP, 2013 Miguel Coimbra Slides by Prof. Kristen Grauman.
Linearizing (assuming small (u,v)): Brightness Constancy Equation: The Brightness Constraint Where:),(),(yxJyxII t  Each pixel provides 1 equation in.
Optical Flow Donald Tanguay June 12, Outline Description of optical flow General techniques Specific methods –Horn and Schunck (regularization)
The Brightness Constraint
Tzu ming Su Advisor : S.J.Wang MOTION DETAIL PRESERVING OPTICAL FLOW ESTIMATION 2013/1/28 L. Xu, J. Jia, and Y. Matsushita. Motion detail preserving optical.
The Measurement of Visual Motion P. Anandan Microsoft Research.
Micro expression Detection using Strain Patterns - Sridhar Godavarthy Based on V.Manohar, D.B. Goldgof, S.Sarkar, Y. Zhang, "Facial Strain Pattern as a.
Visual motion Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys.
Course 9 Texture. Definition: Texture is repeating patterns of local variations in image intensity, which is too fine to be distinguished. Texture evokes.
1 Ying-li Tian, Member, IEEE, Takeo Kanade, Fellow, IEEE, and Jeffrey F. Cohn, Member, IEEE Presenter: I-Chung Hung Advisor: Dr. Yen-Ting Chen Date:
Uses of Motion 3D shape reconstruction Segment objects based on motion cues Recognize events and activities Improve video quality Track objects Correct.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #16.
Motion Analysis using Optical flow CIS750 Presentation Student: Wan Wang Prof: Longin Jan Latecki Spring 2003 CIS Dept of Temple.
Effective Optical Flow Estimation
3D Imaging Motion.
Optical Flow. Distribution of apparent velocities of movement of brightness pattern in an image.
1 Motion Analysis using Optical flow CIS601 Longin Jan Latecki Fall 2003 CIS Dept of Temple University.
-Sridhar Godavarthy A Little Background: Blink A Lot More Background: Strain as a Soft Forensic Evidence Facial Recognition Culprits Human anatomy as.
Features, Feature descriptors, Matching Jana Kosecka George Mason University.
Colour and Texture. Extract 3-D information Using Vision Extract 3-D information for performing certain tasks such as manipulation, navigation, and recognition.
Motion / Optical Flow II Estimation of Motion Field Avneesh Sud.
Optical flow and keypoint tracking Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Image Motion. The Information from Image Motion 3D motion between observer and scene + structure of the scene –Wallach O’Connell (1953): Kinetic depth.
MOTION Model. Road Map Motion Model Non Parametric Motion Field : Algorithms 1.Optical flow field estimation. 2.Block based motion estimation. 3.Pel –recursive.
Motion tracking TEAM D, Project 11: Laura Gui - Timisoara Calin Garboni - Timisoara Peter Horvath - Szeged Peter Kovacs - Debrecen.
Under Guidance of Mr. A. S. Jalal Associate Professor Dept. of Computer Engineering and Applications GLA University, Mathura Presented by Dev Drume Agrawal.
Motion and optical flow
Motion and Optical Flow
3D Vision Topic 4 of Part II Visual Motion CSc I6716 Fall 2009
The Brightness Constraint
Image Primitives and Correspondence
Range Imaging Through Triangulation
The Brightness Constraint
The Brightness Constraint
Announcements more panorama slots available now
Announcements Questions on the project? New turn-in info online
Coupled Horn-Schunck and Lukas-Kanade for image processing
Announcements more panorama slots available now
Optical flow Computer Vision Spring 2019, Lecture 21
Optical flow and keypoint tracking
Presentation transcript:

- Sridhar Godavarthy

Expressions Microexpressions FACS Evolutionary Psychology Proposed method Outline Video Face Detection, Alignment and Splitting Motion Field Optic Flow Optic Strain Datasets Current Work Future Work Results Initial WACV Optic Flow Thresholding

Expressions Microexpressions FACS Evolutionary Psychology

Primary means of social emotion conveyance Non verbal Conveys emotional state Voluntary or involuntary Minuscule differences in muscle movement

6 primary expressions(Not all clearly distinguishable) Can you identify the ones in this picture?

What are microexpressions? Subtle movements of the human face Usually caused when attempting to mask a macro- expression Quick enough to be completed within the blink of an eye Last from 1/25 th to 1/5 th of a second Restricted to certain muscles(regions) of the face Almost impossible to fake

Examples: Raising an eyebrow Shrugging of shoulders Pout of lips Fast blinking of eye Non Examples: Talking Smiling Laughing Anger

For compartmentalization and categorization of human expressions 32 Action Units (with muscle involvement) 14 Action Descriptors ( ! -do-) Can be used to code any possible expression Widely used in pain recognition and facial expression simulations

1000 page manual. Devised and written by one man. Requires extensive training. Some success in automating [ Bartlett et. al 1999].

Study of everything we discussed until now The child of ONE man - Paul Ekman. Over thirty years of research One of the world’s leading experts on lying. About 2 dozen books and innumerable articles Developed FACS Scientific Advisor to “Lie to Me” Co creator of Microexpression Training Tool (METTx)

Expressions Microexpressions FACS Evolutionary Psychology Proposed method Outline Video Face Detection, Alignment and Splitting Motion Field Optic Flow Optic Strain Datasets Current Work Future Work Results Initial WACV Optic Flow Thresholding

Face Detection & Translation Optical Strain Split into ROI Motion Field Estimation (Optical Flow) Eye Detection/Alignment Thresholding for period, strain Combine and count ROI

Face Detection & Translation Optical Strain Split into ROI Motion Field Estimation (Optical Flow) Eye Detection/Alignment Thresholding for period, strain Combine and count ROI

Video is a collection of individual images also known as frames In reality: spatial and temporal compression using properties of the scene. Any video can be decoded into a series of frames. 24/30 frames per second of video.

The science of encoding a video in a manner such that Minimum number of bits are used Motion compensated prediction can be performed from a previous frame.

Face Detection & Translation Optical Strain Split into ROI Motion Field Estimation (Optical Flow) Eye Detection/Alignment Thresholding for period, strain Combine and count ROI

Face Detection & Translation Optical Strain Split into ROI Motion Field Estimation (Optical Flow) Eye Detection/Alignment Thresholding for period, strain Combine and count ROI

2D vector field of velocities of the image points induced by the relative motion.

Feature-based methods Extract visual features (corners, textured areas) and track them over multiple frames Sparse motion fields, but more robust tracking Suitable when image motion is large (10 s of pixels) Direct methods Directly recover image motion at each pixel from spatio- temporal image brightness variations Dense motion fields, but sensitive to appearance variations Suitable for video and when image motion is small

Def: Optical Flow is the apparent motion of brightness patterns in the image Ideally, same as the motion field Have to be careful: apparent motion can be caused by lighting changes without any actual motion

Brightness constancy Under most circumstance, the apparent brightness of moving objects remain constant Optical Flow Equation Relation of the apparent motion with the spatial and temporal derivatives of the image brightness Aperture problem Only the component of the motion field in the direction of the spatial image gradient can be determined The component in the direction perpendicular to the spatial gradient is not constrained by the optical flow equation Key assumptions Brightness constancy: projection of the same point looks the same in every frame Small motion: points do not move very far Spatial coherence: points move like their neighbors

Constant Flow Method Assumption: the motion field is well approximated by a constant vector within any small region of the image plane Solution: Least square of two variables (u,v) from NxN Equations – NxN (=5x5) planar patch Condition: A T A is NOT singular (null or parallel gradients) Weighted Least Square Method Assumption: the motion field is approximated by a constant vector within any small region, and the error made by the approximation increases with the distance from the center where optical flow is to be computed Solution: Weighted least square of two variables (u,v) from NxN Equations – NxN patch Affine Flow Method Assumption: the motion field is well approximated by a affine parametric model u T = Ap T +b (a plane patch with arbitrary orientation) Solution: Least square of 6 variables (A,b) from NxN Equations – NxN planar patch

Face Detection & Translation Optical Strain Split into ROI Motion Field Estimation (Optical Flow) Eye Detection/Alignment Thresholding for period, strain Combine and count ROI

Different materials have different elasticity Elasticity can be modeled Known Calculate

What is Facial Strain? Strain on soft tissue when expressions are made. Anatomical method Uses a pair of frames to measure deformation

Finite Element Method Forward modeling when Dirichlet condition is satisfied Good at handling irregular shapes Computationally expensive This method is an approximation to the solution Finite Difference Method Strain, a tensor, can be expressed derivatives of the displacement vector This can be approximated by a Finite Difference Method. Very efficient when carried out on a regular grid. This method is an approximation to the differential equation

Finite Difference Method Compute spatial derivatives from discrete points. Forward Difference Method Central Difference Method Richardson extrapolation

Optical FlowOptical Strain

Expressions Microexpressions FACS Evolutionary Psychology Proposed method Outline Video Face Detection, Alignment and Splitting Motion Field Optic Flow Optic Strain Datasets Current Work Future Work Results Initial WACV Optic Flow Thresholding

Name#videos~ #SequencesTotal Political USF(30)8-9~250 Found Videos4(+10)1-2~20 ASL1TBGt-

Eye detection/face alignment to accommodate head movement/rotation. Automatic thresholding Dataset collection Testing on “interesting” videos Trying a different Optical Flow( Black and Anandan) Run expression detection to remove Macro expressions first.

Micro expression recognition

Expressions Microexpressions FACS Evolutionary Psychology Proposed method Outline Video Face Detection, Alignment and Splitting Motion Field Optic Flow Optic Strain Datasets Current Work Future Work Results Initial WACV Optic Flow Thresholding

Images resized non-uniformly for presentation

~22 frames ~5 frames

P. Ekman and W. Friesen. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, 1978 Malcolm Gladwell,” Blink: The Power of Thinking Without Thinking”, Back Bay Books (April 3, 2007) G. Donato, M. Bartlett, J. Hager, P. Ekman, and T. Sejnowski. Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10):974–989, 1999

Sridhar Godavarthy Dept. Of Computer Science and Engineering University of South Florida