1. Vocal Joystick A New Dimension in Human-Machine Interaction ET 2 Presentation Group 3 Jeremy Moody, Carrie Chudy

2. Introduction A new technology is emerging in the software area of information technology. This “emerging technology” is called Vocal Joystick and will bring new light to the way we operate computers.

3. What Is It? Vocal Joystick is software that will enable individuals with motor impairments to control objects on a computer screen and ultimately electro-mechanical instruments simply by using vocal parameters. www.sciencentral.com/articles/view.php3?type =article&article_id=218393030

4. Who Uses It? Vocal Joystick’s primary user is anyone with a physical impairment that limits the use of their arms and hands. “There are many people who have perfect use of their voice who don’t have use of their hands and arms.”-Jeffrey Bilmes, Creator However, Vocal Joystick software will be increasingly useful for everyday computer operators.

5. How Does It Work? Vocal Joystick utilizes three main components Acoustic signal processing Pattern recognition Motion control Translates verbal cues into directional movements.

6. Acoustic Signal Processing The signal processing module extracts short-term acoustic features – Energy – Autocorrelation coefficients – Linear prediction coefficients – Mel frequency cepstral coefficients (MFCC) Signal conditioning and analysis techniques are needed for accurate estimation of these features

7. Pattern Recognition The features are then directed into the pattern recognition module where energy smoothing, pitch and formant tracking, vowel classification and discrete sound recognition take place. This stage involves statistical learning techniques such as neural networks and dynamic Bayesian networks.

8. Motion Control Energy, pitch, vowel quality and discrete sound become acoustic parameters to be transformed into direction, speed and other motion related parameters. These motion control parameters are then used to launch corresponding actions.

9. Engine Diagram

10. Vowel Corpus The Vocal Joystick vowel corpus represents the sounds used to control the movements of the “mouse”. Vocalic signals are used with manipulations of pitch and intensity for variation.

11. Vocal Corpus (cont’d) Detects sounds 100 times a second Users are able to transition smoothly from one vowel to another unlike in speech recognition. Louder sounds make the cursor move faster.

12. Applications Browsing the Web Drawing on a screen Controlling a cursor Operating a robotic arm In the future, Vocal Joystick is optimistic about its use to control an electronic wheelchair.

13. Conclusion Vocal Joystick is an amazingly beneficial “Emerging Technology” that will have a major impact on the Information Technology world. With simply a microphone, a computer with a standard sound card and a user who can vocalize, anyone can utilize Vocal Joystick!

14. References http://www.sciencentral.com/articles/view.ph p3?type=article&article_id=218393030 http://www.sciencentral.com/articles/view.ph p3?type=article&article_id=218393030 http://seattletimes.nwsource.com/html/busin esstechnology/2008231288_btjoystick06.html http://seattletimes.nwsource.com/html/busin esstechnology/2008231288_btjoystick06.html http://ssli.ee.washington.edu/vj/

15. References (cont’d) http://ssli.ee.washington.edu/vj/engine.htm http://ssli.ee.washington.edu/vj/corpus.html http://www.yubanet.com/artman/publish/arti cle_67481.shtml http://www.yubanet.com/artman/publish/arti cle_67481.shtml