uWave: Accelerometer-based personalized gesture recognition and its applications Tae-min Hwang

Index Introduction uWave algorithm design Prototype implementation Evaluation Discussion Conclusion

Introduction The majority of the past work has focused on detecting the contour of hand movement Vision-based methods are fundamentally limited by their hardware requirements and high computation load Wii remote has a “camera”(IR sensor) Detects motion by tracking the relative movement of IR transmitters mounted on the display Smart glove can recognize very fine gestures but require the user to wear a glove tagged with Multiple sensors

uWave algorithm design Fig. 1. uWave is based on acceleration quantization, template matching with DTW, and template adaptation Recognition process

uWave algorithm design Quantization of acceleration data Step 1. The time series of acceleration is temporally Compressed by an averaging window of 50 ms that Moves at a 30 ms step Step 2. The acceleration data is converted into one of 33 levels.

uWave algorithm design i i+2 i i i time Dynamic time warping

uWave algorithm design Dynamic time warping

uWave algorithm design Template adaptation First day template Second day template Dynamic time warping has each time templates

uWave algorithm design Template adaptation When correctly recognized 1st day Positive Update 2nd day

uWave algorithm design Template adaptation 1st day Negative Update 2nd day

Prototype implementation Wii remote three-axis accelerometer from Analog Devices, ADXL330 Range : -3g ~ 3g Lenovo T60 with 1.6GHz Core 2 Duo Time : less than 2ms The speed of uWave implemented in C on multiple platforms Tmobile MDA Pocket PC with Windows Mobile 5.0 and 195MHz TI OMAP processor Time : 4ms 16-bit microcontroller TI MSP430LF1611 Time : 300ms

Evaluation Accuracy with and without template adaptation Average accuracy 93.5% Average accuracy 98.4% Use Bootstrapping 8 participant, 10 time for 8 gestures, 7 days on 3 weeks

Evaluation Rejection Rate(%) Recognition Accuracy(%) Recognition accuracy and rejection rate

Evaluation Gesture-based 3D mobile user interface

Evaluation Gesture-based user authentication 5 participants per each groups

Evaluation Non-critical user authentication Template replacement is important to adapt to such variations What accuracy can uWave system achieve in recognizing users based on user-created ID gestures? How usable is it? In particular, how challenging is it to memorize and perform an ID gesture, in comparison to conventional text ID-based authentication? Since users are allowed to create their own gestures, what constraints in ID gesture selection can be employed to improve the accuracy and usability?

Evaluation Non-critical user authentication

Evaluation What tradeoffs between security and usability can the uWave-based solution achieve? How security can be jeopardized if the attacker sees the owner’s gesture performance, which can be much more visible than textual password entry? Critical user authentication Attackers in G see other`s gesture called visual disclosure 5 times to their own passwords 5 trials to other`s passwords

Evaluation Critical user authentication

Evaluation Critical user authentication

Discussion Limitations of uWave and gesture recognition based on accelerometers in general

Discussion Limitations of uWave and gesture recognition based on accelerometers in general

Discussion Limitations of uWave and gesture recognition based on accelerometers in general

Discussion Limitations of uWave and gesture recognition based on accelerometers in general

Discussion Limitations of uWave and gesture recognition based on accelerometers in general Improving critical authentication While our user evaluation show that accelerometer-based authentication works well for non-critical authentication in terms of both usability and accuracy, it apparently cannot provide the strict security required by critical authentication in many application

Conclusion - The core of uWave includes dynamic time warping (DTW) to measure similarities between two time series of accelerometer readings; quantization for reducing computation load - Experiments demonstrate that uWave achieves 98.6% accuracy - Applications show high recognition accuracy and recognition speed with different hardware features and system resources - There is a need for future research to cope with visual disclosure