1. Translation of Sign Language to Speech Sensor Glove Development Name:Garry Rank Student Number:12046339 Supervisor:Mr Iain Murray

2. Design Requirements Fast Accurate and sensitive. Robust Low cost Low profile – the glove should not impede signing. Be able to detect everything necessary for translation. These are all problems that make most virtual reality gloves unsuitable for the task!

3. Required Data Amount of bend of base and first knuckle of each digit. Pitch of the wrist. Detection of finger splaying and fingers crossed. General area contacts. Top of handSide and palm of hand General hand locations important to ASL:

4. Bend Detection Capacitors lend a natural solution.  Sliding motion to change surface area.  Capacitance is changed.  Detectable by a micro-controller’s analogue to digital converter (ADC).

5. Contact Sensors Primary and Secondary contact locations. Primary  Finger splaying.  Touching the palm.  Fingers crossed. Secondary  Fingers touching elsewhere.  Chest, head, arms or elsewhere on the body. Different types of contact sensors for primary and secondary locations.

6. Primary Contact Improve speed of sign recognition. Amount of finger bend is redundant data if it is known a finger is touching the palm. Anode on palm and tops of fingers. Cathode on finger tips. When the fingers touch the palm a circuit is closed. Same reasoning can be used for finger splaying.

7. Primary Contact Locations Note that finger splaying contacts are not shown here. Palm of HandTop of Hand Cathode Anode

8. Secondary Contact Used to supplement overall sign recognition of the system. A simple button used to detect when a finger is touching something not detectable by the glove. Not as sensitive to soft contact  Some signs could be missed

9. Secondary Contact Locations Top of HandPalm of Hand

10. Testing Bend Sensors m16c development board linked to a PC via serial cable. 8bit ADC resolution mode used for its increased speed. Results showed 10° accuracy possible, but interference may produce erroneous values. 10bit resolution on the ADC offered a dramatically increased reliability and accuracy.

11. Software Flowchart Wait for capacitor charge Set output pin low (0V) Wait some time T Use ADC for voltage reading Transmit value to PC Set output pin high (5V) Start Here

12. Timing A real time system – at least 10 full glove readings per second. Including one for the wrist, three for the thumb and two for each finger, 24 bend sensors may be required. Conversion timing is important! Approximate 6us conversion time. Worst case 40us additional wait time. With up to 24 bend sensors, this is means a 1104us bend sensor clock time. Well within acceptable limits!!!

13. ADC Input Multiplexing m16c micro-controller doesn’t have enough ADC inputs for all the required bend-sensors! Inputs must be multiplexed. Diodes can be used to prevent all capacitors multiplexed from being charged.

14. Multiplexed Input If only A, only B or only C is turned on, then only one capacitor will affect the value as read from the voltage probe (ADC input).

15. Two Gloves The sensors so far assume only one glove. Two handed interaction should also be detected. More contact sensors seems like a good idea, or is it? Pins on microcontrollers are I/O.  They can be input or output.  By controlling this, much more data can be obtained from the same number of sensors.  Incurs a time penalty.

16. Problems The glove needs to be specially made.  Material moves too much with the hand.  Capacitors sewn onto the glove move with the material. Bending of the wrist causes the most problems here.  Wrist bending is calculated and so these errors can be algorithmically reduced Primary and secondary contacts become more sensitive with small, hard locations to attach to.  Thin plastic can be used as an exoskeleton on which to mount sensors.  Straps to keep the glove fixed on the hand. Electrical interference can induce currents that produce errors in results. These are all manufacturing problems, not theoretical ones!

17. Thanks for Listening! Feel free to ask some questions.