The Implementation of a Glove-Based User Interface Chris Carey

Abstract  Multi-touch interfaces offer task simplification through more natural commands  A glove-based interface provides the utility of a multi-touch interface without the proximity restriction  Glove commands that merely match the complexity of mouse commands are less efficient  Instead efficiency is found by simplifying tasks since gestures provide more degrees of input

Background  Why now?  Accessibility of Technology  Increased Application Sophistication  Usage in Restriction Environments Minority Report (2002) - DreamWorksAyo Technology (2007) – Aftermath Entertainment

Past and Current Glove Systems Glove Systems Glove Systems –Haptic Gloves and VR Systems –Full Motion Capture Glove Systems –Basic Wiimote Glove Systems Non-Glove Systems Non-Glove Systems –Neural Network Hand Gesture Recognition –3D Model Reconstruction Gesture Recognition

Project Goals  Task Simplification  Improved User Experience  Overcoming Command Inaccuracy  Creative Applications for Usage

Hardware Implementation Logitech Webcam Logitech Webcam –IR-blocking filter removed –Visible-light blocking filter added IR LED Glove IR LED Glove –3 IR LEDs –3 1.5V AAA batteries

Software Implementation Java and Java Media Framework Java and Java Media Framework Custom LED Detection Custom LED Detection LED Tracking LED Tracking Gesture Recognition Gesture Recognition Command Execution Command Execution (Glove Interface)

Software Implementation  Manipulation of photos with both mouse and glove interfaces (drag, rescale, rotate)  Pre-defined tasks in which photos must be manipulated to reach a final state  Space and time data collection during task  Data export to CSV files (Photo Manipulation Application)

Software Implementation (Photo Manipulation Application)

LED Detection Binary Rasterization Binary Rasterization Brightness Threshold Determination Brightness Threshold Determination

LED Detection  Blob Detection  Finding centers of two overlapping LEDs Equal DistributionUnequal Distribution

LED Tracking  Initial Classification Required  Identifies left/right pointer/clicker/aux LEDs  Logic-Based Reclassification of new LEDs

Gesture Recognition  Application-Specific Gestures  Photo drag, rescale, and rotate  Current status:  Single pointer and clicker that matches mouse commands DragRescaleRotate

Gesture Recognition  Two-Handed Gestures: Rescale/Rotate  Executed when both hands pinch while their respective cursors are holding an image  Rescale: moving cursors closer/farther  Rotate: rotating cursors about midpoint  Still in development/evaluation phase RescaleRotate

Experiment  Three single-handed gesture commands were executed with both a mouse interface and the glove interface  Relevant space and time collected for comparison

Mouse Interface Time dragging: s Glove Interface Time dragging: s 42.6% (0.785 s) more time Task: Drag Photo 500 pixels to the Right

Mouse Interface Time dragging: s Glove Interface Time dragging: s 12.7% (0.282 s) more time Task: Rescale Photo to from 25% to 100%

Mouse Interface Time dragging: s Glove Interface Time dragging: s 72.3% (2.283 s) more time Task: Rotate Photo 2.0 Radians Clockwise

Analysis  Glove interface consistently spent more dragging time than mouse interface  Evaluation of glove interface:  Smooth during movement  Corrections during placement  Indicates lesser accuracy, especially with non-movement

Conclusion  The glove interface gestures evaluated only matched the mouse interface commands  Glove interface has no advantage in input of the same complexity  It is expected that the glove interface will only have an advantage over the mouse interface when it can simplify the command due to its versatile input capabilities