1. Craig Newman Curtin University of Technology Translating Australian Sign Language into Speech

2. Introduction To Auslan Elements Of Auslan Signs Hardware Used Hardware Control Algorithm Elements Of Sign Recognition Algorithm Future Intentions Presentation Overview

3. British and Irish origins More than 4000 commonly used signs 0.1% of Australians use Auslan Introduction to Auslan

4. Two-handed fingerspelling alphabet

5. Handshapes Location Orientation Movement Expression Elements of Auslan signs

6. Handshapes

7. First Attribute: Degree to which fingers are bent

8. Handshapes Second Attribute: Combination of fingers touching

9. Handshapes Third Attribute: Pitch of the hand

10. Neutral Space Primary Locations Secondary Locations Location

13. Orientation

14. Roll required to differentiate Yaw required to differentiate

15. Large Scale Straight Line Series of Straight Lines Arcs Circles Small Scale Changes in Orientation Changes in Fingers New Handshape Movement

16. Head Eyebrows Eyes Mouth Cheeks Expression

17. P5 glove from Essential Reality

18. Known through the P5 Degree of Finger Bend Position Primary Locations, Movement Orientation Required Contacting Secondary Locations What was Needed?

19. Acquiring Secondary Locations

20. M16C/62 Single-chip Microcontroller

21. The Prototype

22. Collect Data Every 17ms 60 Hertz Synchronisation with P5 Test Pins Not Logged Write Pin High, Read Remaining Pins Log Connected Pins Debounce: Test For Consistency Encapsulate for Transmission Transmit When Required Contact Sensor Algorithm

23. Working Example 1

24. Working Example 2 INSERT CLIP: IM

25. 3 Samples Stored in stop_buffer Position Orientation Finger Bend Contacts Check Last 2 Samples within Range When was the hand considered Stopped?

26. Wait Until Hands Stop Take First Sample From stop_buffer Filter Through if/else Structure to find Correct Combination Return Handshape Identifying Handshapes

27. Working Example 1 INSERT CLIP: GUN

28. Working Example 2 INSERT CLIP: MOTHER

29. Record Position of Head Shoulder Bicep Forearm Extrapolate Primary Locations Else is Neutral Space Problem Requires Dynamic Position Tracking Identifying Primary Locations

30. Wait Until “not stopped” While “not stopped” copy stop_buffer Allow for Slight Pauses Filter Data By Testing For Inherent Geometric Structure Output Movement Type Identifying Movement Type

31. Identify Initial Handshape, Location, Orientation Filter Through if/else Structure If Sign Recognised Output Sign Else Identify Movement Identify Final Handshape, Location, Orientation Filter through if/else Structure Output Sign Putting it all together: Recognizing Signs

32. Working Example INSERT CLIP: WRAPUP2

33. Design Composite Glove Dynamic Tracking Of Primary Locations Introduce Learning Algorithms Identifying Handshapes Identifying Movement Translate Signing Context Into English Text Integrate Text To Voice Synthesizer Future Intentions

34. Questions & Answers