1. IMOTIONS EMOTION RECOGNITION RW Taggart, M Dressler, S Khan, P Kumar JF Coppola, C Tappert CSIS 692/481 Pace University May 5, 2016

2. IMOTIONS  Biometric research platform  IMotions Inc. Founded in 2005 and based in Copenhagen, Denmark  Can combine eye tracking, facial expression analysis, EEG, GSR, EMG ECG  Investigating facial expression recognition & Galvanic Skin Response (GSR)  Trying to correlate these detected expressions with emotions (happiness, sadness, confusion, anger, fear)  And measure skin/heart rate responses to a certain stimuli Source: https://cdn.imotions.com/wp-content/uploads/2016/02/Screenshot-iMotions.jpghttps://cdn.imotions.com/wp-content/uploads/2016/02/Screenshot-iMotions.jpg

3. SLACK  Remote Team collaboration  Separation of thoughts through “Channels”  Search for previous conversations  “Pin” relevant posts or discussions for simple retrieval  Easy for customer to stay in sync with the team  Agile project development with standup, playback, & retrospective

4. EXPERIMENT  Goals:  Assess usability and validity of iMotions software as a biometrics platform  Assess accuracy of GSR in heart rate/skin conductance results  Automatically process collected video recordings and GSR recordings through iMotions software  Data Collection: - Facial Responses  Collection of 5 videos played consecutively, with brief pauses, to induce facial reactions.  Total of 5 videos, 30 – 40 sec. each, Total time of just 4-5 min with breaks  Due to IT resource limitations, project was confined to only project participants  “Bulk” upload of videos into iMotions software - Galvanic Skin Response  Sync and configure Shimmer device with iMotions to collect GSR with heart rate  Total of 5 genres of music, 30 sec samples of each, total of 4-5 min with breaks  Experiment was only limited to one user due to multiple connection hassles and accessibility

5. COLLECTION PLATFORM  Remote Team, local software  Forced to create a data collection platform  “iMotionsDataCollector” is a web-based framework to allow remote capture of responses to external stimuli  Built on MEAN Stack  AngularJS, Express, node.js, mongodb  Hosted on IBM Bluemix  Used Microsoft OneDrive cloud storage for hosting video data  Available open source:  Hosted: https://data-collector.mybluemix.nethttps://data-collector.mybluemix.net  Code Repository: github.com/rwtaggart/iMotionsDataCollectorgithub.com/rwtaggart/iMotionsDataCollector

6. DATA PROCESSING  Bucketing Emotions:  iMotions has a recognition “confidence” metric in 5 areas of emotions based on a recorded baseline  These 5 facets will be compared against an expected reaction based on the stimuli given  There are three valence levels associated to each emotion  Remote Desktop Control:  Can take over the iMotions laptop remotely  Allows all team members to contribute and perform data analysis

7. FINDINGS  Remote use of iMotions  Easy import/export of data manually  Non-trivial to automate data processing  Non-trivial to record and analyze data from remote location  Powerful FACET video processing  Easy to understand, detailed attributes  External Stimuli should be engaging or interactive if possible, to illicit true emotional response  Correlation Analysis  External analysis engines, such as R, are required for higher- level analysis  GSR  Heart rate results were more accurate compared to skin response  Response time was little to nothing  However, hardware is expensive and use is limited to iMotions computer

8. SUMMARY  Great start to long term research project at Pace University  Need Pace to invest IT resources for large data collection servers  Research use of iMotions APIs to automate data collection & processing

9. QUESTIONS?