The Implementation of a Glove-Based User Interface Chris Carey

Abstract Multi-touch interfaces offer task simplification through more natural commands A glove-based interface provides the utility of a multi-touch interface without the proximity restriction Glove commands are effective when they provide a simplified or more natural alternative to the mouse command Glove commands are not effective for single input tasks or tasks requiring accuracy

Background Why now? Accessibility of Technology Increased Application Sophistication Usage in Restriction Environments Ayo Technology (2007) – Aftermath Entertainment Minority Report (2002) - DreamWorks

Past and Current Glove Systems Haptic Gloves and VR Systems Full Motion Capture Glove Systems Basic Wiimote Glove Systems Non-Glove Systems Neural Network Hand Gesture Recognition 3D Model Reconstruction Gesture Recognition

Project Goals Gestural Command Input Task Simplification Command Naturalization Overall Effectiveness in Speed and Accuracy

Hardware Implementation Logitech Webcam IR-blocking filter removed Visible-light blocking filter added IR LED Glove 3 950 nm IR LEDs 3 1.5V AAA batteries

Software Implementation (Glove Interface) Java and Java Media Framework Custom LED Detection LED Tracking Gesture Recognition Command Execution

Software Implementation (Photo Manipulation Application) Manipulation of photos with both mouse and glove interfaces (drag, rescale, rotate) Manipulation of viewpoint (pan, zoom) Tasks in which photos and/or viewpoint must be manipulated to reach a final state Space and time data collection during task Data export to CSV files

Software Implementation (Photo Manipulation Application)

LED Detection Binary Rasterization Brightness Threshold Determination

LED Detection Blob Detection Finding centers of two overlapping LEDs Circle Hough Transform – finding centers of multiple overlapping LEDs

LED Tracking Initial Classification Required Identifies left/right pointer/clicker/aux LEDs Logic-Based Reclassification of new LEDs

LED Tracking Standard Gestural Command Parameters

Gesture Recognition and Command Execution Photo Commands: Photo drag, rescale, and rotate Single Two-Finger Pinch Commands: Rescale/Rotate requires dragging on corner

Gesture Recognition and Command Execution Double Two-Finger Pinch Commands: Executed when both hands pinch while their respective cursors are holding an image Rescale: moving cursors closer/farther Rotate: rotating cursors about midpoint

Gesture Recognition and Command Execution Viewpoint Commands: Viewpoint pan and zoom Single Three-Finger Grab Commands: Double Three-Finger

Experiment 3 photo tasks performed: Mouse and Glove (Single) Task #1: Drag photo 500 pixels (right  left) Mouse and Glove (Single and Double) Task #2: Rescale photo (50%  150%) Task #3: Rotate photo (3.0 radians clockwise)

Experiment 7 viewpoint tasks performed: Mouse and Glove (Single) Task #4: Pan left (1000 pixels) Task #5: Pan right (1000 pixels) Task #6: Pan up (600 pixels) Task #7: Pan down (600 pixels) Task #8: Pan up (600 pixels) and left (1000 pixels) Task #9: Pan down (600 pixels) and right (1000 pixels) Mouse and Glove (Double) Task #10: Zoom in (75%  150%)

Analysis Glove interface vs. Mouse Interface: Slower with single-handed commands Faster with double-handed commands Faster with compositional panning Focus on natural and simplified commands improves glove interface performance time

Analysis Advantages Performs wirelessly at a distance Requires fewer on-screen controls Allows for the combination and simplification of multi-step commands Allows for more naturally-defined commands Disadvantages Less sensitivity and accuracy control Longer physical command execution time when command complexity matches Time delay between gesture performance and command execution Arm fatigue

Conclusion Glove interface performance: Provides the utility of multi-touch at a distance Effective with natural and simplified commands Not effective with commands requiring precision