High Six The Sign Language Glove Group 6

Group Members Kirk Chan – CpE Brian Troili – EE Ali Mizan – CpE Laura Rubio-Perez – EE & CpE

Project Introduction Glove diagram contrast. Re-do please.

Motivation Fresh idea to the UCF community This project has the potential to help the speech impaired Based on the research, technologies necessary were interesting Android development Bluetooth communication Personal taste

Goals We want the following key factors: HARDWARE SOFTWARE Low-power Android Lightweight Accurate & fast translations Wireless Support text & audio translations Low power relative to Bluetooth Classic

American 26 letters, 10 numbers Specifications Component Design Specification Hand gesture translation Less than 1s Glove weight Less than 1.5lbs Battery life (from one charge) Approximately 20hr Application startup Within 5s Bluetooth connection range Up to 50m Default gesture library American 26 letters, 10 numbers

Design Approach Draw squares in final copy

Design Overview

Hardware Components Our hardware block diagram is divided into 4 subsystems: Analog sensors IMU Power supply Communication

Hardware Components Flex Sensors Able to detect changes in bend/flex Changes its resistance at several points along the device When a current is applied, it creates a voltage divider Due to the very little variety of flex sensor available in the market .. long-short We selected the 2.2” flex sensor. This decision was made because sensors get ruin when bend more than 90 degree. This was important to consider since most of the hand gestures in the American sign language blend the fingers at a degree greater than 90. There is a total of 11 flex sensors in our glove, there are 2 sensors in each finger and one across the palm

Hardware Components Pressure Sensors Acts as a force sensing resistor When the sensor is unloaded, its resistance is very high When pressure is applied, its resistance decreases For the hand gestures ,such as the letter S, that have specific points of contact we decided to use pressure sensors in order to identify them easily. We also selected the pressure sensors because there are cheaper and reliable than the case of adding flex sensors in between fingers

Hardware Components Analog/Digital Converter (ADC) Serial communication preferred. Large number of input channels. Avoid serial address conflict. - ADS7828 I2C compatible 8 Channel ADC variable I2C address

Hardware Components Analog/Digital Converter (ADC) Serial communication preferred. Large number of input channels. Avoid serial address conflict. - ADS7828 I2C compatible 8 Channel ADC variable I2C address

Hardware Components Accelerometer and Gyroscope Inertial Measurement Unit (IMU) Speed demand allow for serial buses. - ITG3200/ADXL345 combo board 3.3V input I2C compatible 3 axis each calibrate to 2, 4, 8, and 16g

Hardware Components ADXL345 3-axis accelerometer Low power Low current use: 40μA in measuring mode and 0.1μA in stand by Suited for mobile device applications

Hardware Components ITG3200 3-axis digital gyroscope Low-cost motion sensor Features 16-bit analog-to-digital converters Supply voltage range: 2.1V - 3.6V Current consumption of 6.5mA

Hardware Components Wireless Communication Wifi Bluetooth NFC Secure connection Portable connection Low power Approx. 100m range Approx. 7-8in

Hardware Components Wireless Communication Wifi Bluetooth NFC Secure connection Portable connection Low power Approx. 100m range Approx. 7-8in

Hardware Components HC-06 Bluetooth Module Bluetooth v3.0 Operating voltage: 3.6V – 6V Working current of 40mA Approximately $6 Range: 30ft

Hardware Components Microcontroller MSP430g2553 ATmega328p ATmega32u4 Operating Voltage 3.3V 5V 5.5V Digital I/O Pins 12 14 Analog Input Pins 6 Clock Speed 16MHz Architecture RISC Flash Memory 16KB 32KB

Hardware Components Microcontroller MSP430g2553 ATmega328p ATmega32u4 Operating Voltage 3.3V 5V 5.5V Digital I/O Pins 12 14 Analog Input Pins 6 Clock Speed 16MHz Architecture RISC Flash Memory 16KB 32KB

Hardware Components Development Environment Arduino IDE Code Composer Free Open source (Unlimited support ) Limited support Extensive libraries Direct access to control registers avr-gcc compiler C/C++ compilers and assembly

Hardware Components Development Environment Arduino IDE Code Composer Free Open source (Unlimited support ) Limited support Extensive libraries Direct access to control registers avr-gcc compiler C/C++ compilers and assembly

Hardware components Battery Nickel Cadmium Nickel Metal Hydride Polymer Lithium Ion Rechargeable Low energy density High energy density Highest energy density Good performance in low temperature Can operate in low temperature Resistant to overcharge Not environmental friendly Limited service life Require circuit protection High-self discharge Degradation in performance Age quickly

Hardware components Battery Nickel Cadmium Nickel Metal Hydride Polymer Lithium Ion Rechargeable Low energy density High energy density Highest energy density Good performance in low temperature Can operate in low temperature Resistant to overcharge Not environmental friendly Limited service life Require circuit protection High-self discharge Degradation in performance Age quickly

Hardware components LP-063048 Polymer Lithium Ion Battery Extremely light weight Outputs a 3.7V at 1000 mAh. Features 2C continuous discharge Robust power source under extreme conditions Long-term self-discharge rates Approximately $9 Bought from: Sparkfun As charge/discharge cycles are repeated, the battery capacity gradually declines

Hardware components Power Cell-LiPo Charger/Booster Single cell boost converter to 3.3V and 5V and Micro-USB charger all in one Boost converter is based on the TPS61200 from Texas Instrument Low input voltage synchronous boost converter Operating input voltage range from 0.3V to 5.5V Fixed and adjustable output voltage from 1.8V to 5.5V lm2576

Hardware components Power Cell-LiPo Charger/Booster micro-USB charger uses the MCP73831 It charges 3.7V LiPo cells at 100mA. Limits the charge current based on the die temperature during high power It utilizes a constant- current/ constant-voltage configuration The constant-voltage regulation has four options: 4.20V, 4.35V, 4.40V, and 4.50V lm2576

Hardware components LP2985 Regulator Low-dropout: 280 mV at 150-mA load current 7 mV at 1-mA load Low-noise operation with a typical output noise of 30 μVRMS Consumption of only 0.01 μA when the ON/OFF pin is pulled low. Overcurrent and thermal protection

PCB

PCB

PCB

PCB

Software Components

Android vs iPhone Android iPhone Cross platform capable integrated development environment Familiarity with the Java language The most popular mobile platform iPhone Can only be developed in Mac Apps written in objective C Apple development software only works with other apple development software

Android IDEs IntelliJ (free version) Eclipse Advantages: Less buggy More intuitive Faster Better GUI Disadvantages: Java, Groovy, or Scala are only 3 languages supported in free version Eclipse Advantages: More plug-ins available More commonly used Disadvantages: Has bugs and crashes a lot Talking about mobile application translation

Software Components Two main components: Android Application Is the interface between the user and the classification algorithm Takes in raw data from glove Displays letter on screen Translator Translates letters just by checking flex sensors and accelerometer to see if they fit certain boundaries The application has some default boundaries set, but can be customized by user

Traditional Bluetooth Setup

Our Bluetooth Setup

Our Android Setup

Successful Recognition of a letter Threshold – Based Algorithm How to recognize gestures? Compare sensor input from glove to high and low thresholds. Boundaries can be adjusted Default boundaries we created come with application New boundaries stored in local memory and can be reset back to defaults Successful Recognition of a letter Sensor High Threshold Sensor data Low Threshold Within boundaries? S1 18 15 13 yes S2 11 9 S3 25 21 17 From mobile development environment to deeper look into the layout of Android app

Threshold – Based Algorithm How to segment data? Recognition algorithm only called when hand is still for about .5 seconds Which sensors need to be read? The flex and pressure sensors alone can differentiate between 32 of the 36 character For the 4 hand gestures that have same shape, orientation is used to differentiate them From mobile development environment to deeper look into the layout of Android app

Final Financing DESCRIPTION QTY PRICE Flex sensors 2.2” 11 $79.50 IMU Digital Combo Board (3-Axis Gyro/Accelerometer) 1 $51.96 Flexiforce Pressure Sensors 4 $79.80 Bluetooth module (Free) $0.00 Microcontroller (Atmega 328P) $37.97 PCB 2 $584.78 Miscellaneous Surface Mounting - $85.00 TOTAL $919.00 Boeing sponsorship - $518.00 Total Available from Sponsorship Over budget $401.00 Total per person $100.25 here is what we have so far thank you boeing early stages of our development half way done through our budget underbudget-pcb

Progress

Issues No such thing as universal boundaries Hard to define boundaries Boundaries differ from user to user Hard to define boundaries Even with just one user, it may be difficult to define boundaries that work well most of the time BLE and Hidden Markov Model TPS61200 reference errors May lack libraries for Arduino IDE. BLE lacks online support since it’s new. We’ve determined that the hidden markov model is the appropriate algorithm that we need for our project, but the exact implementation is still under research. It’s a complex algorithm and it’s going to take time to implement it as a software module and test it against various hand sizes and gestures.

Approaching the issues No Universal Boundaries allow the user to train boundaries for their hand Boundaries hard to define To make letters easier to classify, add a tolerance on top of the boundaries Ie: if tolerance = 5%, high and low boundaries both multiplied by 1.05 BLE and Hidden Markov Model Switched to classic Bluetooth (3.0) Switched to deterministic algorithm

Questions?

DEMO