HandTalk Design & Architecture Feb 15 th, 2008

Status Update Found sweet new module for the glove – does exactly what we need But now have to work with less than ideal number of I/O pins! Going to talk to Motorola contact to see if we can get sweeter modules, sensors or phone Sensors need power – almost solved that problem

Glove Bluetooth Data Acquisition Module Nokia 95 Flex SensorsA2D Bluetooth Module Gesture Recognition (Java2ME)‏ Text  Speech Engine Quantized data AnalogDigital Translated Gesture USERUSER AUDIENCEAUDIENCE Gesture Speech Architecture

Use Cases Power On  User activates application on cellphone which sends wake command over Bluetooth to glove module  Glove module LED turns green indicating device is active  Cellphone application sends command for glove to stream data and processes it as it arrives Recognize Gesture (Fist/Open Palm for now)‏  User makes gesture, causing a combination of readings to be sent  If measurement pattern matches a known gesture, the appropriate word is voiced through Text-to-Speech engine Sleep Mode  If user turns off application, sleep command is sent to glove  LED on glove module turns off indicating it is in sleep mode

Risks & Mitigation - Glove module has ability to power off sensors  solution works if we power sensors through module - Relay activated by general purpose I/O pin on module Putting powered sensors to sleep might require separate switch - Use 2 flex sensors for each of 3 fingers and 1 each for the other 2 - Use 1 sensor for each finger and 3D accelerometer  Requires rethinking of objective Too few I/O pins (8 as opposed to ideally 10+)‏ - Define acceptable thresholds - Expand voltage range of sensors using more complicated setup - Machine learning Difficult to match sensor readings to known patterns Glove module specifically does what we needBluetooth complicated to implement MitigationRisks