Mobile HCI Presented by Bradley Barnes

Mobile vs. Stationary Desktop – Stationary Users can devote all of their attention to the application. Very graphical, detailed Use the keyboard and mouse for input

Mobile and Wearable Devices Users in motion – can’t devote all of attention to the application Limited screen real estate Input and output capabilities are restricted for users on the move

Mobile Device Interaction The interface for mobile and wearable devices continues to mimic those of desktop computers. New interaction techniques are needed to safely accommodate users on the move. Interaction should be subtle, discreet, and unobtrusive.

Mobile Interaction Methods Keyboard Touch Screen Speech Recognition Head motion, 3-D sound Eyeglass displays with Gestural Interaction

CHI2005 Paper Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller Using a mobile device in a social context should not cause embarrassment and disruption to the immediate environment.

Intimate Interfaces Discrete interfaces that allow control of mobile devices through subtle gestures in order to gain social acceptance. Must take into account the social context where the interaction will occur. Most interaction occurs around other people (bus, train, street, etc.)

Electromyographic Signals EMG signals are generated by muscle contraction. Signals are picked up by contact electrodes. Allows a definition of “subtle” or “motionless gestures” that can be used to issue commands to mobile devices. Can sense muscular activity not related to movement.

The System Armband controller recognizes gestures Signals transmitted via Bluetooth Compliant device receives signals and performs appropriate action. Can be PDA, mobile phone, etc. The armband works on all users: no calibration or training required.

When combined with eyeglass displays, the system becomes “hands free”. Can be operated when users are carrying items. Can be used in specific fields, such as maintenance, for assistance when the user’s hands are tied.

Design Process Iterative process centered on users. Three pilot studies and one formal Study. Pilot Study 1: Bicep is chosen muscle, and the gesture is defined as a brief unnoticeable contraction of the bicep.

Pilot Study 2: Refine the gesture definition, and create an algorithm for its detection. New subjects w/ variety of muscle volumes Gesture not fully described to subjects Compared EMG signals of gesture to those of normal activity. Algorithm detects peaks in the EMG signals.

Pilot Study 3: Fine tuning of the system, wanted to test for false positives and false negatives Consisted of new and returning users Worked with the algorithm until the number of false positives approached zero Also, they decided to try a gesture alphabet with two gestures. They are defined as two short contractions of different duration.

Formal Study: Validation of Results Pilot studies set up the system parameters by testing gestures on subjects who were not mobile. Conducted to assess the usability of EMG as a subtle interaction technique for mobile devices. Evaluated the system usability in a mobile context.

Formal Experiment Design 10 adult participants-Ages 23 to 34 Perform 5 walking tasks – one with no contraction to calculate misclassification rate, and other four with contractions of different durations. Subjects make laps around obstacles while doing the contractions

Familiarization sessions preceded the walking tasks. These involved standing and making contractions. The participants were prompted to contract by a MIDI piano tone delivered through the wireless headphones. System recognized contractions between 0.3 and 0.8 seconds, but the subjects did not know the duration of their contraction: only that the system recognized it.

Tasks 1) Walking, No Contractions – 10 laps 2) Standing, Familiarization, Generic Contractions 3) Walking, Stimulus-Response, Generic Contractions 4) Standing, Familiarization, Short Contractions

5) Walking, Stimulus-Response, Short Contractions 6) Standing, Familiarization, Long Contractions 7) Walking, Stimulus-Response, Long Contractions 8) Walking, Stimulus-Response, Mixed Long and Short Contractions (low tone – long contractions, high tone – short)

Results The online recognition rates for the four walking tasks were: Generic: 96% Short: 97% Long: 94% Mixed: 87%

Conclusion An EMG based wearable input device can be used for subtle and intimate interaction. The system presented can recognize motionless gesture without training or calibration. EMG gestures can be utilized as a socially acceptable alternative for mobile device interaction

Future Work Expand gesture alphabet Test in more “real world” scenarios, like when lifting something.

References Lumsden, J., Brewster, S. A paradign shift: alternative interaction techniques for use with mobile & wearable devices. Proc. Of the 13 th Annual IBM Centers for Advanced Studies Conference CASCON’2003. Costanza, E., Inverso, S., Allen, R. Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller.