1. Interaction Design Session 12 LBSC 790 / INFM 718B Building the Human-Computer Interface

2. Agenda Questions Interaction design Some examples Project presentations

3. Interaction Design Play to the strengths of machine and human Place the locus of control with the user Make it easy to do the right thing Support multiple interaction styles

4. Strengths Machine –Speed –Storage –Repeatability Human –Initiative –Flexibility –Recognition

5. Putting the User in Control Familiar metaphor Visible objects –Non-modal design Predictable behavior Feedback on progress Explicit user models –Basic and expert modes Optional “wizards”

6. Making it Easy Visible context Understandable icons and messages –Tool tips and drill-down Atomic actions Obvious results Previews and reversability

7. Multiple Interaction Styles Point-and-click Keyboard shortcuts Command line Spoken dialog

8. Interactive Voice Response Systems Operate without graphical interfaces –Hands-free operation (e.g., driving) –Telephone access Built on three technologies –Speech recognition (input) –Text-to-speech (output) –Dialog management (control) Example: TellMe (1-800-555-TELL)

9. Speech Recognition Isolated words recognition is easy –Specialized dictation and telephone applications Continuous speech is slow and error prone –Hands-free tasks, dictation, speech retrieval Performs best when trained for one speaker Limited vocabulary and language coverage Does not work well in noisy environments

10. Speech Recognition Phoneme Detection Word Construction Word Selection Phoneme String Phoneme Lattice Word String Pronunciation dictionary Word n-gram language model One-best phoneme transcription N-best phoneme sequences One-best word transcript

11. Speech Recognition Lattice

12. Dialogue Management Turn-taking –User initiative –System initiative (allows smaller vocabulary) –Mixed initiative (e.g., barge in) Interaction style –Direct answers Achieving conversational goals

13. System Initiative Finite state control automates scripts –Restaurant, airline reservation, … A “state” encodes everything you know –What prompt to offer –What to do for each possible answer Loops allow for compact representations

14. Finite State Control Example Where are you departing from? Where do you want to go? What day do you want to travel? VerificationGoodbye Wrong Confirmed Baltimore National Dulles San Francisco Oakland San Jose Anywhere else Day when there are flights Not a day Sorry Another day Anywhere else

15. Cooperative Responses I want to fly to Tysons Corner on Friday –Completion All of the flights are sold out –Correction There is no airport in Tyson’s Corner –Suggestion Dulles is the closest airport –Conditional answer The only flight is on Tuesday –Summary answer I have flights on US carriers or KLM

16. The CSLU Toolkit IVRS development environment Graphical finite state dialog editor Text-to-speech, plus an animated face Isolated-word speech recognition Available at http://cslu.cse.ogi.edu/toolkit/

17. Supporting Information Access Source Selection Search Query Selection Ranked List Examination Recording Delivery Recording Query Formulation Search System Query Reformulation and Relevance Feedback Source Reselection

20. NPR Online

22. SpeechBot

24. Thesaurus-Based Search

26. Design Critique Select any 3 GUI’s you know and can use here –e.g., Windows XP, Google, USMAI catalog Work in in groups of 3 to critique each –Using IBM design guidelines http://www-3.ibm.com/ibm/easy/eou_ext.nsf/publish/6 –What are the 3 best features of each? –What are the 3 principal weaknesses of each?

27. An Example http://www.philipglass.com/