Video Surveillance for Human Emotion Identification(VSHEI) Anku Adhikari, Hao Wu and Lingyong Wang (aadhikr2, haowu11, lwang84@illlinois.edu) Introduction In this project we propose VSHEI, a system to extract heart pulse from human facial videos, that is suitable for varied environments and tolerant to natural motion and movements. We highly improve results from comparative works that use Eulerian Color Magnification and demonstrate usage of the results for emotion analysis by heart rate variability detection. Goals: Design a system able to extract human pulse under conditions of normal movement and environmental dynamics. Investigate best methods to pre-process the video for facial tracking and stabilization. Use current methods that magnify changes in human face video frames and improve and customize them. Extract clear pulse signals and eliminate environmental noise. Demonstrate the application of the extracted pulse information and multiple video footage for detecting heart rate variability and diagnosing heart rate change indicating emotion/physiological states. Implementation Results System Framework: Raw Video Face Tracking Output Video Stabilization Output Eulerian Magnification Output Extracted Pulse Waves Pulse Rate, Diagnosis Face Tracking: Signal Processing: Video Stabilization: Emotion Analysis: Contribution  VSHEI has much improved results compared to Eulerian Color Magnification method(MIT) with results that are sharper, less noisy and more accurate. It can extract heart pulses in different scenarios of lighting, indoor/outdoor condition, skin type and movement levels and video at different distance and facial angle from camera. It preprocesses to stabilize the face in the video and handle different levels of motion in the human subject. It is able to face track changes introduced by 2D motion in all direction and slight 3D face rotation and motion. It uses feature points to tracking human faces. This approach out performs another algorithm that uses skin color tone. VSHEI pulse rate and waveforms can be used to analyze pulse and facial changes for other human physiology applications directly. Pulse rate = 61.87bpm State: Resting state Diagnosis: physiological stress not detected Evaluation Experimental Videos Description Level1(Motion test) Stable subject Level2(Motion test) Slight movement Level3(Motion test) Subject moving forward Low HR State (Variability test) Subject before exercise High HR State(Variability test) State Subject immediately after exercise Evaluation done using 7 samples of experimental video data. Wide variety of video shot samples collected for experimental analysis in conditions of: Indoor and Outdoor lighting conditions Different subjects and skin tones Different types of movement Different levels of physical exertions Different types of background noise conditions Future Work Further optimization for real-time applications. VSHEI can be applied for other applications that investigate physiological changes in the body: criminal investigation, athlete training, medical monitoring, etc.