1. Enabling Always-Available Input with Muscle-Computer Interfaces T. Scott Saponas University of Washington Desney S. Tan Microsoft Research Dan Morris Microsoft Research Ravin Balakrishnan University of Toronto Jim Turner Microsoft Corporation James A. Landay University of Washington

2. Mobile Computing Enables…

3. “How the computer sees us.” Igoe & O'Sullivan

4. Hands Busy Physically Active

9. Muscle-Computer Interfaces

10. Muscles Activate via Electrical Signal

11. Electrical Signal can be sensed by Electromyography (EMG)

12. EMG for Diagnostics, Prosthetics & HCI Jacobsen, et al. “Utah Arm”

13. Costanza, et al. “Intimate interfaces in action” EMG for Diagnostics, Prosthetics & HCI

14. Jacobsen, et al. “Utah Arm” Costanza, et al. “Intimate interfaces in action” Naik, et al. “Hand gestures” EMG for Diagnostics, Prosthetics & HCI

15. Jacobsen, et al. “Utah Arm” Costanza, et al. “Intimate interfaces in action” Wheeler & Jorgensen “Neuroelectric joysticks” Naik, et al. “Hand gestures” EMG for Diagnostics, Prosthetics & HCI

16. Finger Gestures Detected from Upper Forearm

17. Detecting Finger Gestures Challenging

18. Offline Classification of Finger Gestures on a Surface Saponas, et al. CHI 2008

19. Real-Time Classification of Free Space & Hands Busy Gestures Pinch Mug Bag

20. Bimanual Gesture + dominant hand gesture non-dominant hand squeeze

21. Sensor Placed on Upper Forearm

23. Stimulus / Response Training

24. Gesture Classification Technique 30 millisecond sample X 6 Sensors Support Vector Machine labeled training data user specific model machine learning

25. Gesture Classification Technique 30 millisecond sample Root Mean Square (RMS) ratios between channels Frequency Energy 10 Hz bands Phase Coherence ratios between channels X 6 Sensors Features Support Vector Machine labeled training data user specific model machine learning

26. Gesture Classification Technique 30 millisecond sample Root Mean Square (RMS) ratios between channels Frequency Energy 10 Hz bands Phase Coherence ratios between channels X 6 Sensors Features Support Vector Machine user specific model machine learning gesture classification

28. 12 Person Experiment Pinch Mug Bag

29. Training vs Testing in Several Postures Train Test LeftCenterRight Left78%72%57% Center70%79%74% Right68%73%74%

30. Posture Independent Pinching

31. Bag in Hand Better Recognized

32. Worked Well for Those Who “got it”

33. 80% Accurate with 70 Seconds Training

34. Portable Music Player Menus Some participants navigated menus easily Other participants found interaction difficult

35. Limitations of Current Technique Works best for SINGLE user SINGLE session Wired Sensors with Gel and Adhesive Sitting or Standing at a Desk in the Lab

36. Ongoing & Future Work Wireless Armband, Dry Electrodes, Cross-Session Models

37. Ongoing & Future Work Walking & Jogging Wireless Armband, Dry Electrodes, Cross-Session Models

38. Ongoing & Future Work Walking & Jogging Interactive Tabletops Wireless Armband, Dry Electrodes, Cross-Session Models

39. Enabling Always-Available Input with Muscle-Computer Interfaces T. Scott Saponas University of Washington Desney S. Tan Microsoft Research Dan Morris Microsoft Research Ravin Balakrishnan University of Toronto Jim Turner Microsoft Corporation James A. Landay University of Washington Thanks for Listening