1. Natural Language Systems
ITCS 6010
Natural Language Systems

2. Overview
Welcome to ITCS 6010
Syllabus
Introduction

3. Syllabus
Syllabus

4. Good Design (our goal!)
“Every designer wants to build a high-quality interactive system that is admired by colleagues, celebrated by users, circulated widely, and imitated frequently.” (Shneiderman, 1992, p.7)
…and anything goes!…

5. What is an interface?
An interface refers to the part of technology that people interact with
Interactions include information transfer:
From user to computer
From computer to user

6. Interaction Components
Interaction hardware include:
Keyboard, mouse, stylus, keypad, microphone
Interaction software include:
Window, page, sound, talking voice

7. What is a ‘well-designed’ interface?
Depends on your perspective……
Examples:
For a programmer – works within technical constraints of project
For a usability engineer – designed with particular user group in mind
For a user – works the way expected

8. Types of Interfaces Character-based user interface (CHUI)
Graphical user interface (GUI)
Web user interface (WUI)
Speech user interfaces (SUI)
Auditory user interface (AUI)
Graphical user interface with speech (S/GUI)
Voice user interface (VUI)

9. Speech User Interface A software interface that employs speech
Human speech
Simulated human speech

10. Auditory User Interfaces
An Auditory user interface (AUI) is an interface which relies primarily or exclusively on audio for interaction, including speech and sound. (Weinschenk & Barker 2000)
Examples:
Hands-free automobile navigational system
Interactive voice response system (IVR) like automated payment center
Products for visually impaired

11. Auditory User Interfaces
Natural Language/Speech User Interfaces
Conversation is natural
Multimodal User Interfaces
Combines voice, text, graphics, gestures, keypad, stylus, etc. into one interface

12. Graphical User Interface with Speech (S/GUI)
Multimodal interface that involves speech and a GUI
Examples:
Voice activated calling on cell phone
Dictation software that allows text entered via text, speech or both

13. Graphical User Interface with Non-Speech Audio
Interface that includes non-verbal audio
Earcons – auditory icons/sounds that communicate information
Examples:
System beeps when user makes an error
System knocks when someone wants to chat

14. Multimodal User Interfaces
Simultaneous Multimodality
Multiple modes at the same time, voice-visual
Sequential Multimodality
Uses multiple modes sequentially and seamlessly

15. Voice User Interface
A voice user interface (or VUI) is what a person interacts with when communicating with a spoken language application. (Cohen et al, 2004)

16. Why a VUI? Characteristics that favor VUI: Hands-busy situation
No keyboard, mouse, stylus available
Disablilties
Context-specific, command driven application

17. But What Makes a Good VUI?
Functionality
Speed & efficiency
Reliability, security, data integrity
Standardization, consistency
USABILITY !

18. Closer to Fine: A Philosophy
…The human user of any system is the focus of the design process. Planning and implementation is done with the user in mind, and the system is made to fit the user, not the other way around….
Bruce Walker
Georgia Institute of Technology

19. How Do You Know It’s Good?!
Usability Test and Evaluation

20. Human Factors in Speech

21. Human Factors in Speech
High Error Rates
Speech recognition
Background noise, intonation, pitch, volume
Grammars (missing words, size limitations)
“When speech recognition becomes genuinely reliable, this will cause another big change in operating systems.” (Bill Gates, The Road Ahead 1995)

22. Human Factors in Speech
Unpredictable Errors
Grammars
Sound alike words Austin-Boston
Missing words
Grammar size limitations
Note: We do not like using unpredictable machines.

23. Human Factors in Speech
User Expectations
Novice users have high expectations of computers and speech
Natural language
Novices expect to say “anything” to the machine
i.e. Star Trek
Spoken language differs from written language.
i.e. ums or uhs appear in spoken language

24. Human Factors in Speech
Memory
Speech only systems can be taxing on human memory, i.e. large telephone menu systems.
Miller - 7 plus or minus 2

25. Definitions and Terms

26. List all the U-N-C Charlotte orders.
Speech Recognition
List all the U-N-C Charlotte orders.
Refers to the technologies that enable computing devices to identify the sound of human voice.

27. Speech Recognition Continuous Recognition Discrete Recognition
Allows a user to speak to the system in an everyday manner without using specific, learned commands.
Discrete Recognition
Recognizes a limited vocabulary of individual words and phrases spoken by a person.

28. Speech Recognition Word Spotting
Recognizes predefined words or phrases.
Used by discrete recognition applications.
“Computer I want to surf the Web”
“Hey, I would like to surf the Web”

29. Speech Recognition Voice Verification or Speaker Identification
Voice verification is the science of verifying a person's identity on the basis of their voice characteristics.
Unique features of a person's voice are digitized and compared with the individual's pre-recorded "voiceprint" sample stored in the database for identity verification.
It is different from speech recognition because the technology does not recognize the spoken word itself.

30. James, here are the U-N-C Charlotte orders.
Speech Synthesis
Refers to the technologies that enable computing devices to output simulated human speech.
James, here are the U-N-C Charlotte orders.

31. Speech Synthesis Formant Synthesis
Uses a set of phonological rules to control an audio waveform that simulates human speech.
Sounds like a robot, very synthetic, but getting better.

32. Speech Synthesis Concatenated Synthesis
Uses computer assembly of recorded voice sounds to create meaningful speech output.
Sounds very human, most people can’t tell the difference.

33. Uses of Speech Technologies
Interactive Voice Response Systems
Call centers
Medical, Legal, Business, Commercial, Warehouse
Handheld Devices
Toys and Education
Automobile Industry
Universal Access (visual/physical impaired)

34. Requirements Analysis
A E I O U (Y)
Artifacts
Environment
Interaction
Outcome
User
WhY

35. A Artifacts
What are the artifacts?
Things
Places

36. E Environments What are the environments? Physical Places of Operation
Operating Environments/Systems

37. I Interactions What are the interactions? Between humans
Between machines
Between humans and machines

38. O Outcomes What are the outcomes? Tangible outcomes
Intangible outcomes

39. U Users
Who are the users?
Customers
Clients
Developers
Users

40. Y Why?
Why are you doing this?
Motivation
Client
User
You