1. Emotion Recognition using the GSR Signal on Android Devices Shuangjiang Li

2. Outline Emotion Recognition The GSR Signal Preliminary Work Proposed Work Challenges Discussion

3. Emotion Recognition Human-Computer Interaction Speech recognition Gesture/Action recognition Facial expression recognition Emotion recognition … Affective Computing ( Picard @MIT Media Lab around late 90s)

4. Emotion Recognition Physiological Signals

5. Source: http://biomedikal.in/2011/05/important-physiological-signals-in-the-body/http://biomedikal.in/2011/05/important-physiological-signals-in-the-body/

6. The GSR Signal Galvanic Skin Response (GSR) measuring the electrical conductance of the skin due to the response of the skin and muscle tissue to external and internal stimuli, the conductance can vary by several microsiemens (unit of ohm). GSR is highly sensitive to emotions (fear, anger, startle response, etc.) http://en.wikipedia.org/wiki/Skin_conductance

7. The GSR Signal GSR Sensor SHIMMER ( Sensing Health with Intelligence, Modularity, Mobility and Experimental Reusability ) Platform The goal of SHIMMER is to provide an extremely compact extensible platform for long-term wearable sensing in both connected and disconnected settings using proven system building blocks. a highly extensible wireless sensor platform SHIMMER firmware is based on TinyOS Data transmit via Bluetooth Can sense EMG, ECG, GSR, etc. Support Matlab, LabView, Android, C#/.Net etc. http://shimmer.sourceforge.net/ http://www.shimmer-research.com/ Adrian Burns, SHIMMER: An Extensible Platform for Physiological Signal Capture, IEEE EMBS, 2010

9. Preliminary Work Emotion recognition based on the GSR signal Four emotion categories: amusement, fear, relax, sadness Using GSR + Accelerometer signal Preprocessing Using supported accelerometer data Denoising using median filter Data rescaling and normalization Feature Extraction 6 statistical features + 10 time domain features + 4 frequency domain features + feature selection (SFFS)

11. Preliminary Work Recognition rate KNN 10-fold cross-validation Subject dependent / single subject

12. Proposed Work Emotion recognition on Android Devices Android GUI for reading GSR sensor data GSR data preprocessing GSR data classification Sequential learning

13. Challenge Emotion signal tend to very noisy. Emotion signal generally lacks ground truth and emotion is very subjective. Recognition algorithms on Android devices should be light weight Dealing with sequential data

14. Discussion Q&A