SP13 ECE 445: Senior Design Sign Language Teaching Glove Project #29: Reebbhaa Mehta, Daniel Fong, Mayapati Tiwari TA: Igor Fedorov

Introduction Motivation - No available portable devices to teach sign language Objective - Sensing unit to detect gestures accurately - Kalman Filter to reduce noise from data - Program to check each gesture - Provide feedback to user

American Sign Language

Overview

System Components Three Main Units - Sensing unit (MPU 6050 & Flex Sensors) - Software Component - Feedback unit (LED’s)

System Components Hardware - Power Supply – 9V - Microcontroller Unit – Arduino Uno - 5 flex Sensors & 5 MPU Bluetooth - 5 Sensors - LED Driver Software - Arduino Programming Environment - Kalman Filter - Perceptron Learning Algorithm

Power Supply – Actual Choice First Choice – Lithium Backpack - Supplies 5V to power Arduino Uno - Supplies 3.3V to power all other components - Matches Arduino size and fits on the back - Rechargeable via USB - Shorted !!

Power Supply – Alternate Alternate – Energizer 9V - Supplies 9V - Arduino on-board regulator generate 5V to power Arduino Uno - Arduino on-board regulator generate 3.3V to power all other components - Not very useful for effective space utilization - Not rechargeable

Sensing Unit Consists of: - 5 accelerometers and gyroscopes (MPU 6050) - 5 flex sensors (FLX-03) - I 2 C Multiplexer (TCA9548A)

Accelerometer & Gyroscope MPU Accelerometer & gyroscope in one chip - Helps to detect gestures - Accelerometers detect tilt - Gyroscopes detect angular velocity - Better space utilization - Placed near fingertips – PCB’s need to be small

MPU-6050

MPU-6050 Data Orientation from gravity Gyroscope for change in orientation

MPU 6050 Schematic

MPU 6050 PCB Pic

Flex Sensors Uni-directional flex sensors (FLX- 03) used Help to provide more accurate data Range: 10kΩ to 40kΩ - For unflexed hand: 10kΩ - For completely flexed hand: 40kΩ

Flex Sensor Circuit

Flex Sensor – Test & Data R (Flex Sensor)Vout 9.48 kΩ1.78 V 15.3 kΩ1.69 V 17.0 kΩ1.59 V 21.2 kΩ1.36 V 22.7 kΩ1.32 V

Flex Sensor - Problems Problem - Broke due to heat - Used copper tape to fix the problem without success - Only one working flex sensor Consequences - Less data from sensing unit - Reduced accuracy to differentiate between gestures - Failed requirement

I 2 C Multiplexer (TCA9548A) 8 bi-directional translating switches I 2 C bus compatible Channel selection via I 2 C bus 8-channel I 2 C switch communicate with up to 8 I 2 C devices which have the same address

Feedback Circuit Controls 10 LEDs using 8 bit shift register 3 arduino output pins.

Microcontroller Arduino Uno - Serial Communication - Easily programmable - Attaches to Lithium Backpack (better space utilization) - Voltage regulators to provide 3.3V and 5V - Works well with external battery - I 2 C protocol MPU 6050 communication library

Bluetooth Bluetooth Shield - Arduino Uno compatible - UART communication - Up to 10m communication - Fits the back of Arduino Uno Requirement Failed: No connection with computer. Reason: Broken antenna or chip not programmable

Kalman Filtering F = state u = gyroscope data Var(w) = Q Var(v) = R H = [1;0] B = delta (T)

Kalman Filtering Estimation:

Kalman Filtering Update:

Kalman Filtering Test A Sensor 1 – Little Finger

Kalman Filtering

Perceptron Learning Algorithm On-line, mistake driven algorithm. Linear classifier that updates the weight vector incrementally when mistakes are made. Decision rule checks whether the dot product of the weight vector with an input vector is greater that some threshold.

Perceptron Learning Algorithm

Failed Verification ProblemReason BluetoothInitialization code stopped working after a while Defective piece (maybe) Flex SensorsNot within the specified range Broke due to heat – resistive strip came off PerceptronWanted – 98% accuracy Achieved – 75% No data from flex sensor. Adding more variables. Feedback UnitNot integratedNo real time data processing

Accomplishments Optimum data from MPU-6050 Kalman Filter Perceptron to check gestures Effectively differentiate between 5 gestures (A, B, L, V and Y) LED for feedback Soldering very small components like MPU-6050

Future Steps Replacing broken flex sensors to get more accurate data for each gesture Increase the accuracy of perceptron by adding more features Real time implementation Better feedback unit with haptic feedback and LED’s With more accuracy and better detection words can be added to the library and thus progress can be made

Acknowledgement Prof. Scott Carney Igor Fedorov Mark Smart Skot Wiedmann Waltham Smith Daniel Mast Aadhar Jain Joseph Shim

Questions?