Department of Electrical and Computer Engineering He Zhou Hui Zheng William Mai Xiang Guo Advisor: Professor Patrick Kelly ASLLENGE

Department of Electrical and Computer Engineering Outline  Background  System Requirements  Design Challenges  System Overview  Hardware and Software  Team Roles  Timeline  Q/A 2

Department of Electrical and Computer Engineering Background  Leverage Kinect technology to recognize and offer feedback on American Sign Language  ASL is the one of most commonly used language in US(.5 to 2 million population use ASL as their primary form of communication)  Educational learning tool for ASL 3

Department of Electrical and Computer Engineering Motivation  Helping hearing impaired people to learn ASL.  Facilitate communication between ASL speaker and non-speaker 4

Department of Electrical and Computer Engineering System Requirements  Create a user-friendly interface that recognize sign language and offer feedback Software must be easy to use  Display translated word or phase being demonstrated by user Provide quality feedback to user concerning the accuracy of ASL  Cost should be within the budget 5

Department of Electrical and Computer Engineering Design Challenges  Human Factors  Background Interferences  Finger Gestures  Hardware limitation Kinect only has 320x240 resolution and 16 bit depth 6

Department of Electrical and Computer Engineering Kinect  Has a depth and regular Camera  Associated with software development kit Able to track a human being. Use computer generated algorithms to form human skeleton data points  Track upper body movements 7

Department of Electrical and Computer Engineering System Overview 8

Department of Electrical and Computer Engineering Video demonstration(Hand Rotation) 9

Department of Electrical and Computer Engineering Video demonstration(Hand Crossing) 10

Department of Electrical and Computer Engineering Video demonstration(Hand behind Head) 11

Department of Electrical and Computer Engineering Video demonstration(Hand behind Back) 12

Department of Electrical and Computer Engineering Hardware Implementation 1  Increasing the flex sensor output voltage with bend  Voltage output goes into an AD converter board  AD converter digitizes the output signal  PC captures the signal via USB or Bluetooth 13

Department of Electrical and Computer Engineering Hardware Implementation 2  Kinect Camera captures the raw image and the depth image data  The data send to PC to be processed into skeleton tracker  Combining the signal from flex sensor and search in database for matching motions 14

Department of Electrical and Computer Engineering Software Design Flow 15

Department of Electrical and Computer Engineering Development Model 16

Department of Electrical and Computer Engineering Cost of Major Components 17

Department of Electrical and Computer Engineering Team Roles  He - test and integrate software prototype  Hui- image processing and programming  William - build microcontroller to detect finger and palm movements  Xiang- website, overall testing and building ASL database 18

Department of Electrical and Computer Engineering MDR deliverables The goal for our MDR is to be able to detect and track arm and hand positions, possibly be able to translate simple hand gestures 19

Department of Electrical and Computer Engineering Conclusion Our project is good for SDP: Contains both hardware and software Expandable: database, audio recognition, braile Our project is a good product: Aid the disabled people Personal/commercial use Relatively inexpensive 20

Department of Electrical and Computer Engineering Timeline 21

Department of Electrical and Computer Engineering Q&A 22