WearSys 2018 Keystroke Inference Using Ambient Light Sensor on Wrist-Wearables: A Feasibility Study Mohd Sabra, Anindya Maiti Murtuza Jadliwala Wichita State University, Wichita, KS, USA University of Texas San Antonio, San Antonio, TX, USA Hello everyone, I am very excited to be able to present TITLE at WearSys 2018. I am Mohd Sabra just graduated as an undergraduate at Wichita State Univeristy and will be a Phd student next fall at UTSA. N Sunday 10th June, 2018. Munich, Germany

Premise Inferred Keystrokes Environmental ambient lights Ambient Light Sensor on watch Typing sensitive data Smartwatches are recently becoming a popular trend, infact according to Consumer Technology Association, the sales of smartwatchs have been doubling every year snice. 4 years. currently selling 141 million units. The reason why some many people use smartwatches is due how versitial and useful a smart watch is. For example Some of the many usasages a smartwatch provides are fitness monitoring, health monitoring, and personal assistance . These smartwatches have sensors inside of them to improve the user expernice. One of the sensor is an ambient light sensor. The main role of the ambient light sensor is to detect if it is day or night , to increase/decrease screen brightness and some apps implement features such as night reading. In our paper we had a question we wanted to answer. “Can an adversary uses this sensor in an unorxdiaxte way to infer the keystrokes?”. If an adversary was able to control and place envirmoental lights , could the adversary infer the keystrokes while a user is typing with the smartwatch – on lets say ATM keypad? Smartwatch people wear everywhere Have an ambient light sensor for day and light, battery saving mode In the wrong enovirment could be used to infer the data Inferred Keystrokes

Related Work: Keystroke Inference Attacks Acoustic Emanations Asonov Et Al. [1] & Berger Et Al. [3] Surface Vibration Emanations Barisani Et Al. [2] & Marquardt Et Al. [14] Motion Emanations Wang Et Al. [20] & Liu Et Al. [10] Using sound to detect different keystroke in keyboard Using keyboard vibration that is caught by the phone to detect keystrokes Using motion sensors to detect motions of hand

Related Work: Ambient Light Sensor as a Side-Channel Spreitzer et al. [18]: Smartphone ambient light sensor to detect smartphone keystroke. Holmes et al. [7]: Light emanation from computer screen to determine screen distance from smartwatch. Using smartphone Sensor to get smartphone keystroke study of light to get distance from screen to smartwatch

Our Contributions Use of external light sources around the ATM. Optimization using timing analysis. Complete attack framework for inferring ATM PIN codes.

Attack Summary 1) Have the target user Install a malicious app on the smartwatch. 2) Set up a special light environment around the target (desired ATM). 3) Train a supervised learning-based classifier to infer keystrokes from ambient light data. 4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

Attack Summary 1) Have the target user Install a malicious app on the smartwatch. 2) Set up a special light environment around the target (desired ATM). 3) Train a supervised learning-based classifier to infer keystrokes from ambient light data. 4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

Malicious App Objective: Collect light intensity (lux) values and timestamp data from smartwatch and covertly transport it to the adversary. Ways to get the app on the target: Direct Access Social Engineering Trojan application masqueraded as a legitimate application It all starts with having a malicious app in the target smartwatch. Once the app is there, Ambient light sensor is a zero-permission sensor!

Attack Summary 1) Have the target user Install a malicious app on the smartwatch. 2) Set up a special light environment around the target (desired ATM). 3) Train a supervised learning-based classifier to infer keystrokes from ambient light data. 4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

Background about Light Lux value: The amount of lumen per square meter. Distance : The further the light source from the ambient light, the lower lux read. Angle : The more perpendicular the sensor angle is to the light source, the higher lux value would be detected. Orientation : Light sources should be in an asymmetric fashion to ideally create unique lux values for different keys.

Light Settings:1 vs 2 light sources 1 dimensional : Hard to distinguish between change of sensor location or angle. Blind Spots : Not all angles covered 2 dimensional : Easier to distinguish between change of sensor location or angle. Blind Spots : Not all angles covered The lux reading is 1 dimenisal, for example it can be 100 lux or 200 lux. But it also depends on the distance. Because you only have 1 light source, it is hard to detect between a change of angle or change of distance. Also the light source will not cover all 180 degree reotation the user would do , thus the light source have blind spots, spots the sensor would not detect the setup enivorment 1 1 2

Light Settings:3+ light sources Blind Spots : No more blind spots. 3 Light Sources: 2 1 3 There is insignificant change (sometimes negatively) and more complex to model. 4+ Light Sources:

Attack Summary 1) Have the target user Install a malicious trojan app on the smartwatch. 2) Set up a special light environment around the target (desired ATM). 3) Train a supervised learning-based classifier to infer keystrokes from ambient light data. 4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

Training Machine Learning Classifiers Train classifier Data Collection Feature Extraction Lux before/after keystroke, average/median/min/max lux Training Machine Learning Classifiers SVM, Random Forest, K-NN, Decision Tree, Naïve Bayes

Attack Summary 1) Have the target user Install a malicious app on the smartwatch. 2) Set up a special light environment around the target (desired ATM). 3) Train a supervised learning-based classifier to infer keystrokes from ambient light data. 4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes. VVVV

Experimental setup Training data: Adversary collected 1200 random ATM digits keystroke Environment : Training and testing environment are the same Participants : 14 Smartwatch: Sony Smartwatch 3 ATM Keypad: Wincor Nixdorf EPPv5 (Digit replica using tablet) Test data: 40 random 4-digit PIN codes

Result Trends: Sequential Duplicate Digits:

Results for Sequential Duplicate Digits (1-1) (0-0) (3-3) (4-4) etc. No lux change (with right environment) 100% detection accuracy 27.1% of all 4-digit PIN have at least 1 sequential duplicate A single duplicate digit PIN such as 1-1-1-1 reduces search space to 10

Result Trends: Sequential Duplicate Digits Speed is ~constant during typing

Constant Speed Distance between buttons are never changing. Distance = Time * Speed Proportional Distance = ~ Proportional Time The target will always end at a “Enter” button. Backtracking can be used to reduce search space.

Time Analysis Get Euclidean distance between buttons Create table dividing every unique distance pairs Integrate test distance pairs Get time between consecutive keystrokes Divide every unique time pair Starting from Enter button, backtrack PIN Get possible distance pairs

Attack Summary Trained Classifiers Typing Sequential Duplicates Detection Timing Analysis Feature Extraction Module Apply Classification Techniques Mark and Remove Duplicates Raw lux values and Timestamps Reduce Search Space

1 Keystroke vs. 4-Digit PIN 1 Keystroke max accuracy = 65% 0.65^4 = 17.8% accuracy (best case) 4-digit PIN with reduced search space: 62% accuracy (significant improvement)

Limitations Smartwatch angle Interruption during typing Low ambient light sensor precision Coarse grained give lux in steps of 4.

Conclusion With suitable light environment, up to 65% accuracy in inferring individual keystrokes. After reducing search space using timing analysis, up to 62% accuracy in inferring 4-digit ATM PIN codes. Unregulated ambient light sensor on smartwatches can be exploited as an effective side-channel attack vector.