1. Secure Unlocking of Mobile Touch Screen Devices by Simple Gestures – You can see it but you can not do it
Arjmand Samuel
Microsoft Research
Muhammad Shahzad Alex X. Liu
Dept. of Computer Science and Engineering
Michigan State University

2. Security Sensitive Information in Mobile Device
Muhammad Shahzad

3. PIN/Password based Authentication
Shoulder surfing
Smudge attack
Muhammad Shahzad

4. Gesture based Authentication (GEAT)
Not What they input but How they input
Resilient to
Should surfing attack
Smudge attack
Requires no extra hardware
Scientific foundation: human behavior tends to be consistent in same context.
J. A. Ouellete and W. Wood. Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior. Psychological Bulletin, 124(1):54-74, July 1998.
Muhammad Shahzad

5. Gestures for Authentication
Muhammad Shahzad

6. Data Collection and Analysis

7. Data Collection Recruited 50 volunteers
Ages between 19 and 55
students, faculty, corporate employees
Gave phones with data collection app to volunteers
Data collection app
Asked users to perform gestures shown on screen
Stored the samples in a cloud based storage
Muhammad Shahzad

8. 2 3 4 1 5 6 7 8

9. Gesture Features Stroke time Inter-stroke time Displacement magnitude
Displacement direction
Velocity magnitude
Velocity direction
Device Acceleration
Stroke time
Displacement Magnitude
Displacement Direction
Inter-Stroke time
Muhammad Shahzad

10. Stroke, Inter-stroke times
Muhammad Shahzad

11. Displacement Magnitude
Muhammad Shahzad

12. Velocity Magnitude
Volunteer 1
Volunteer 2
Muhammad Shahzad

13. Device Acceleration
Volunteer 1
Volunteer 2
Muhammad Shahzad

14. GEAT Working Mechanism

15. How GEAT works Collect training samples Generate classification model
Securely unlock the phone
Muhammad Shahzad

16. Classification Model Noise removal Features for classification
Classifier training and Gesture ranking
Muhammad Shahzad

17. Simple Moving Average (Low Pass Filter)
Noise Removal
Simple Moving Average (Low Pass Filter)
Muhammad Shahzad

18. Features for Classification
Features used
Stroke time
Inter-stroke time
Displacement magnitude
Displacement direction
Velocity magnitude
Velocity direction
Device Acceleration
Stroke based features
Sub-stroke based features
Muhammad Shahzad

19. Feature Selection
Selected
Discarded
Muhammad Shahzad

20. Classifier training Single class classification
Support Vector Distribution Estimation (SVDE)
RBF kernel
Grid search for optimal classifier parameters
Gesture Ranking
Muhammad Shahzad

21. Securely unlocking the device
Accepted
Rejected
Accepted
Majority Voting Decision: Accepted
Muhammad Shahzad

22. Handling Multiple Behaviors
Segregate the samples from different behaviors
Generate Minimum Variance Partitions
Agglomerative Hierarchical Clustering
Wards Linkage
Train classifiers for each cluster
Test an unknown sample against each cluster
Muhammad Shahzad

23. Experimental Evaluation

24. Accuracy Evaluation Single gesture Three gestures Avg EER Avg EER
4.8% with DA
6.8% without DA
Avg EER
1.7% with DA
3.7% without DA
Muhammad Shahzad

25. Multiple Behaviors
Muhammad Shahzad

26. Effect of System Parameters
Muhammad Shahzad

27. Conclusion Proposed a gesture based authentication scheme
Improves security and usability
Resilient to shoulder surfing attacks and smudge attacks
Handles multiple user behaviors
Evaluation through simulations and real world experiments
More in the paper
Detailed data analysis
Technical details of
extracting multiple behaviors
determining duration and locations of sub-strokes
classifier training
more evaluation
Muhammad Shahzad

28. Questions?
Muhammad Shahzad