1. Anindya Maiti, Murtuza Jadliwala, Jibo He Igor Bilogrevic
(Smart)Watch Your Taps: Side-Channel Keystroke Inference Attacks using Smartwatches
Anindya Maiti, Murtuza Jadliwala, Jibo He
Igor Bilogrevic
December 27, 2018

2. Problem Statement
Is it Possible to Infer What is Being Typed on the Phone Based on Wrist Movements Observable by the Smartwatch?
December 27, 2018

3. Motivations Side-Channel Attacks
We can’t turn off access to accelerometer and gyroscope sensors. All applications have access to these two critical sensors by default.
Permissions allows control of access to data directly sensed by the sensors, but not to information that can be indirectly inferred from the sensors!
sca
Side-Channel Attacks
December 27, 2018

4. The Idea Smartphone Smartwatch
Capture motion by sampling the accelerometer (collect linear accelerometer samples).
December 27, 2018

5. Further Investigation1 9
Averages of 30 keystrokes each
More activity on Y and Z axis, than X axis.
Tap on each number on the keypad produces a characteristically unique motion on the wrist! We used this observation in our attack.
December 27, 2018

6. Attack Setup
An attacker installs a malicious application on the victim’s smartwatch through social engineering (e.g. Trojan horse, pretexting, baiting, phishing, etc.) or by gaining physical access to the smartwatch.
Installed malicious application is used to remotely gather motion activity from the sensors of the victim’s smartwatch.
Actual attack is executed “offline”.
Attacker packages the malicious application as a useful application, such as lets say a fitness tracker application. Also keep in mind that operation system makers of the watch can become potential attackers.
December 27, 2018

7. The Attack Detect keystrokes. Extract Features.
Train classification models using appropriate supervised- learning algorithms and labeled training data.
Simple Linear Regression (SLR)
Random Forests (RF)
k-Nearest Neighbor (k-NN)
Use the trained classification models to infer the target’s key taps.
December 27, 2018

8. Experiments 1/2 12 participants aged between 19-32 years age.
A total of 300 keystrokes (30 per numeric key) per participant were collected.
67% used for training, 33% for testing.
For comparison with similar previous works using smartphone motion sensors [1][2], we carried out attack using linear accelerometer data from both the smartwatch and smartphone.
Owusu, Emmanuel, et al. "Accessory: Password Inference Using Accelerometers on Smartphones." Proceedings of the Twelfth Workshop on Mobile Computing Systems & Applications. ACM, 2012.
Miluzzo, Emiliano, et al. "TapPrints: Your Finger Taps Have Fingerprints." Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services. ACM, 2012.
December 27, 2018

9. Experiments 2/2 Samsung Gear Live smartwatch
Motorola XT1028 smartphone
Linear accelerometer of both the watch and phone sampled at 50 Hz.
December 27, 2018

10. Evaluation One vs. One: Training and test data from same participant.B
One vs. One: Training and test data from same participant.
One vs. Rest: Test data from one participant, training data from remaining 11 participants.
All vs. All: Training and test data combined from all 12 participant.
December 27, 2018

11. However, in typing scenario B attack on both devices were comparable.
Results
In non holding hand typing smartwatch performed better than smartphone.
However, in holding hand typing both were comparable. A B
Also, classification accuracy drops with reduction in sampling frequency.
In typing scenario A, attack on smartwatch performed better than smartphone.
However, in typing scenario B attack on both devices were comparable.
December 27, 2018

12. Conclusion
Experimental results validate that smartwatch motion sensors can be employed as effective side-channels to infer private information, such as numeric key taps.
The threat of wrist motion based keystroke inference can be amplified due to smartwatches.
December 27, 2018

13. Future Work
We further analyze the effect of combining motion data from both smartwatch and smartphone.
We are designing an attack framework for another popular typing scenario, where keystrokes events can’t be detected based on motion spikes.
Thank You! Questions?
December 27, 2018