1. An Open-Source Based Speech Recognition Android Application for Helping Handicapped Students Writing Programs
Tong Lai Yu, Santhrushna Gande School of CSE, California State University,
San Bernardino, California
Ronald Yu
Department of Computer Science, University of Southern California,
Los Angeles, California

2. Overview
Application allows students with difficulty typing because of physical disabilities to write Java code using voice recognition
Application consists of a server and a client
Student speaks into an android phone running the client
Client sends data over to server via Wi-Fi
Server processes data and displays the Java code on a workstation

3. Thread Synchronization on Android Client
Main activity thread interacts with user, calls voice recognition engine, and puts voice rec results in a shared queue
Communication thread reads data from the shared queue and sends it to the server
Producer-Consumer paradigm used to make sure threads don’t interfere with each other

4. Workstation Server Threads
Read thread reads data from network sent over by the android client and puts string in a shared buffer
Processing thread reads data from shared buffer and processes it
Synchronize threads with condition variables so threads wait for and signal to each other
Simpler to implement than producer-consumer paradigm used for android client thread synchronization
Only one thread can access the shared queue at a time

5. Keywords and Commands Voice Recognition engine commonly makes mistakes
Server only processes keywords that occur frequently in Java (import, public, int, etc.) and program commands (compile, execute, etc.)
Use a hashtable to map similar-sounding words to a keyword or command

6. Hashing Keywords
Similar-sounding words are mapped to keywords in a hash table
O(1) lookup
Hash table stored in a text file which is read every time the program is started

7. Hashing Keywords (contd.)
Hashing is not reversible
Two different texts may map to the same hash index, leading to collision
Very unlikely to happen
Preprocessing to ensure collision-free
If collision occurs between two texts, one text needs to be deleted or modified
Ex. If Chava and Impact collide...

8. Hashing Commands Commands are stored in a similar hashing scheme
Examples of commands:
capital- Capitalizes the next word
variable- Creates a new keyword to be stored in the hashing table
space- Toggles whether a space is inserted after a word
number- Displays line numbers
undo- Undo the last keyword
execute- Executes the program

9. Results Tested in a standard classroom at CSUSB
Works well if only one or two students use the application in a class session

10. Conclusions
Have developed an Android application using open-source tools
Helps handicapped students write programs in Java using voice recognition
Producer-consumer paradigm and condition variables are used to synchronize events
Hashing is used to map ambiguous texts to specific keywords or commands
Hash collisions are handled by preprocessing
Can be extended to other languages such as C/C++

11. Demo