1. WearSys 2018
Keystroke Inference Using Ambient Light Sensor on Wrist-Wearables: A Feasibility Study
Mohd Sabra, Anindya Maiti
Murtuza Jadliwala
Wichita State University, Wichita, KS, USA
University of Texas San Antonio, San Antonio, TX, USA
Hello everyone, I am very excited to be able to present TITLE at WearSys I am Mohd Sabra just graduated as an undergraduate at Wichita State Univeristy and will be a Phd student next fall at UTSA. N
Sunday 10th June, Munich, Germany

2. Premise Inferred Keystrokes Environmental ambient lights
Ambient Light Sensor on watch
Typing sensitive data
Smartwatches are recently becoming a popular trend, infact according to Consumer Technology Association, the sales of smartwatchs have been doubling every year snice. 4 years. currently selling 141 million units. The reason why some many people use smartwatches is due how versitial and useful a smart watch is. For example Some of the many usasages a smartwatch provides are fitness monitoring, health monitoring, and personal assistance . These smartwatches have sensors inside of them to improve the user expernice. One of the sensor is an ambient light sensor. The main role of the ambient light sensor is to detect if it is day or night , to increase/decrease screen brightness and some apps implement features such as night reading. In our paper we had a question we wanted to answer. “Can an adversary uses this sensor in an unorxdiaxte way to infer the keystrokes?”. If an adversary was able to control and place envirmoental lights , could the adversary infer the keystrokes while a user is typing with the smartwatch – on lets say ATM keypad?
Smartwatch people wear everywhere
Have an ambient light sensor for day and light, battery saving mode
In the wrong enovirment could be used to infer the data
Inferred Keystrokes

3. Related Work: Keystroke Inference Attacks
Acoustic Emanations Asonov Et Al. [1] & Berger Et Al. [3]
Surface Vibration Emanations Barisani Et Al. [2] & Marquardt Et Al. [14]
Motion Emanations Wang Et Al. [20] & Liu Et Al. [10]
Using sound to detect different keystroke in keyboard
Using keyboard vibration that is caught by the phone to detect keystrokes
Using motion sensors to detect motions of hand

4. Related Work: Ambient Light Sensor as a Side-Channel
Spreitzer et al. [18]:
Smartphone ambient light sensor to detect smartphone keystroke.
Holmes et al. [7]:
Light emanation from computer screen to determine screen distance from smartwatch.
Using smartphone Sensor to get smartphone keystroke study of light to get distance from screen to smartwatch

5. Our Contributions Use of external light sources around the ATM.
Optimization using timing analysis.
Complete attack framework for inferring ATM PIN codes.

6. Attack Summary
1) Have the target user Install a malicious app on the smartwatch.
2) Set up a special light environment around the target (desired ATM).
3) Train a supervised learning-based classifier to infer keystrokes from ambient light data.
4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

7. Attack Summary
1) Have the target user Install a malicious app on the smartwatch.
2) Set up a special light environment around the target (desired ATM).
3) Train a supervised learning-based classifier to infer keystrokes from ambient light data.
4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

8. Malicious App
Objective: Collect light intensity (lux) values and timestamp data from smartwatch and covertly transport it to the adversary.
Ways to get the app on the target:
Direct Access
Social Engineering
Trojan application masqueraded as a legitimate application
It all starts with having a malicious app in the target smartwatch. Once the app is there,
Ambient light sensor is a zero-permission sensor!

9. Attack Summary
1) Have the target user Install a malicious app on the smartwatch.
2) Set up a special light environment around the target (desired ATM).
3) Train a supervised learning-based classifier to infer keystrokes from ambient light data.
4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

10. Background about Light
Lux value: The amount of lumen per square meter. Distance : The further the light source from the ambient light, the lower lux read. Angle : The more perpendicular the sensor angle is to the light source, the higher lux value would be detected. Orientation : Light sources should be in an asymmetric fashion to ideally create unique lux values for different keys.

11. Light Settings:1 vs 2 light sources
1 dimensional : Hard to distinguish between change of sensor location or angle.
Blind Spots : Not all angles covered
2 dimensional : Easier to distinguish between change of sensor location or angle.
Blind Spots : Not all angles covered
The lux reading is 1 dimenisal, for example it can be 100 lux or 200 lux. But it also depends on the distance. Because you only have 1 light source, it is hard to detect between a change of angle or change of distance. Also the light source will not cover all 180 degree reotation the user would do , thus the light source have blind spots, spots the sensor would not detect the setup enivorment 1 1 2

12. Light Settings:3+ light sources
Blind Spots : No more blind spots.
3 Light Sources: 2 1 3
There is insignificant change (sometimes negatively) and more complex to model.
4+ Light Sources:

13. Attack Summary
1) Have the target user Install a malicious trojan app on the smartwatch.
2) Set up a special light environment around the target (desired ATM).
3) Train a supervised learning-based classifier to infer keystrokes from ambient light data.
4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.

14. Training Machine Learning Classifiers
Train classifier
Data Collection
Feature Extraction
Lux before/after keystroke, average/median/min/max lux
Training Machine Learning Classifiers
SVM, Random Forest, K-NN, Decision Tree, Naïve Bayes

15. Attack Summary
1) Have the target user Install a malicious app on the smartwatch.
2) Set up a special light environment around the target (desired ATM).
3) Train a supervised learning-based classifier to infer keystrokes from ambient light data.
4) Collect ambient light data from the target user by means of the malicious app and employ the classifier to infer keystrokes.
VVVV

16. Experimental setup
Training data: Adversary collected 1200 random ATM digits keystroke
Environment : Training and testing environment are the same
Participants : 14
Smartwatch: Sony Smartwatch 3
ATM Keypad: Wincor Nixdorf EPPv5 (Digit replica using tablet)
Test data: 40 random 4-digit PIN codes

18. Result Trends:
Sequential Duplicate Digits:

19. Results for Sequential Duplicate Digits
(1-1) (0-0) (3-3) (4-4) etc.
No lux change (with right environment)
100% detection accuracy
27.1% of all 4-digit PIN have at least 1 sequential duplicate
A single duplicate digit PIN such as reduces search space to 10

20. Result Trends: Sequential Duplicate Digits
Speed is ~constant during typing

21. Constant Speed Distance between buttons are never changing.
Distance = Time * Speed
Proportional Distance = ~ Proportional Time
The target will always end at a “Enter” button.
Backtracking can be used to reduce search space.

22. Time Analysis Get Euclidean distance between buttons
Create table dividing every unique distance pairs
Integrate test distance pairs
Get time between consecutive keystrokes
Divide every unique time pair
Starting from Enter button, backtrack
PIN
Get possible distance pairs

23. Attack Summary Trained Classifiers Typing
Sequential Duplicates Detection
Timing Analysis
Feature Extraction Module
Apply Classification Techniques
Mark and Remove Duplicates
Raw lux values and Timestamps
Reduce Search Space

25. 1 Keystroke vs. 4-Digit PIN
1 Keystroke max accuracy = 65%
0.65^4 = 17.8% accuracy (best case)
4-digit PIN with reduced search space:
62% accuracy (significant improvement)

26. Limitations Smartwatch angle Interruption during typing
Low ambient light sensor precision
Coarse grained give lux in steps of 4.

27. Conclusion
With suitable light environment, up to 65% accuracy in inferring individual keystrokes.
After reducing search space using timing analysis, up to 62% accuracy in inferring 4-digit ATM PIN codes.
Unregulated ambient light sensor on smartwatches can be exploited as an effective side-channel attack vector.