Anindya Maiti, Murtuza Jadliwala, Jibo He Igor Bilogrevic (Smart)Watch Your Taps: Side-Channel Keystroke Inference Attacks using Smartwatches Anindya Maiti, Murtuza Jadliwala, Jibo He Igor Bilogrevic December 27, 2018

Problem Statement Is it Possible to Infer What is Being Typed on the Phone Based on Wrist Movements Observable by the Smartwatch? December 27, 2018

Motivations Side-Channel Attacks We can’t turn off access to accelerometer and gyroscope sensors. All applications have access to these two critical sensors by default. Permissions allows control of access to data directly sensed by the sensors, but not to information that can be indirectly inferred from the sensors! sca Side-Channel Attacks December 27, 2018

The Idea Smartphone Smartwatch Capture motion by sampling the accelerometer (collect linear accelerometer samples). December 27, 2018

Further Investigation 1 9 Averages of 30 keystrokes each More activity on Y and Z axis, than X axis. Tap on each number on the keypad produces a characteristically unique motion on the wrist! We used this observation in our attack. December 27, 2018

Attack Setup An attacker installs a malicious application on the victim’s smartwatch through social engineering (e.g. Trojan horse, pretexting, baiting, phishing, etc.) or by gaining physical access to the smartwatch. Installed malicious application is used to remotely gather motion activity from the sensors of the victim’s smartwatch. Actual attack is executed “offline”. Attacker packages the malicious application as a useful application, such as lets say a fitness tracker application. Also keep in mind that operation system makers of the watch can become potential attackers. December 27, 2018

The Attack Detect keystrokes. Extract Features. Train classification models using appropriate supervised- learning algorithms and labeled training data. Simple Linear Regression (SLR) Random Forests (RF) k-Nearest Neighbor (k-NN) Use the trained classification models to infer the target’s key taps. December 27, 2018

Experiments 1/2 12 participants aged between 19-32 years age. A total of 300 keystrokes (30 per numeric key) per participant were collected. 67% used for training, 33% for testing. For comparison with similar previous works using smartphone motion sensors [1][2], we carried out attack using linear accelerometer data from both the smartwatch and smartphone. Owusu, Emmanuel, et al. "Accessory: Password Inference Using Accelerometers on Smartphones." Proceedings of the Twelfth Workshop on Mobile Computing Systems & Applications. ACM, 2012. Miluzzo, Emiliano, et al. "TapPrints: Your Finger Taps Have Fingerprints." Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services. ACM, 2012. December 27, 2018

Experiments 2/2 Samsung Gear Live smartwatch Motorola XT1028 smartphone Linear accelerometer of both the watch and phone sampled at 50 Hz. December 27, 2018

Evaluation One vs. One: Training and test data from same participant. B One vs. One: Training and test data from same participant. One vs. Rest: Test data from one participant, training data from remaining 11 participants. All vs. All: Training and test data combined from all 12 participant. December 27, 2018

However, in typing scenario B attack on both devices were comparable. Results In non holding hand typing smartwatch performed better than smartphone. However, in holding hand typing both were comparable. A B Also, classification accuracy drops with reduction in sampling frequency. In typing scenario A, attack on smartwatch performed better than smartphone. However, in typing scenario B attack on both devices were comparable. December 27, 2018

Conclusion Experimental results validate that smartwatch motion sensors can be employed as effective side-channels to infer private information, such as numeric key taps. The threat of wrist motion based keystroke inference can be amplified due to smartwatches. December 27, 2018

Future Work We further analyze the effect of combining motion data from both smartwatch and smartphone. We are designing an attack framework for another popular typing scenario, where keystrokes events can’t be detected based on motion spikes. Thank You! Questions? December 27, 2018