1. ITM 734 Introduction to Human Factors in Information Systems Cindy Corritore cindycc@gmail.com This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Dialog Design 1ITM 734

2. 2 Dialog Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/Natural language 5. Gesture, pen

3. ITM 7343 Dialog Design - Humor How does a user interact with the interface?

4. ITM 7344 General Issues in Choosing Dialogue Style Who is in control - user or computer Initial training required Learning time to become proficient Speed of use Generality/flexibility/power Special skills - typing Gulf of evaluation / gulf of execution Screen space required Computational resources required

5. ITM 7345 Command Languages Earliest interaction style  Well, at least if you forget about console switches and punched paper tape and cards and plug boards :) Examples  MS-DOS shell  UNIX shell  dBase

6. ITM 7346 CL Characteristics Little or nothing is visible so…  Work primarily by recall, not recognition  Heavy memory load Poor choice for novices but...

7. ITM 7347 CL Advantages for Expert Users Speed, conciseness  % ls (hard to beat for speed) Can express actions beyond a limited set  Flags, piping one command to another Repetition, extensibility  Scripting, macros Power  Abstraction, wild cards Little run-time cost  Irrelevant except in special situations

8. ITM 7348 CL Advantages for Implementers Easier implementation? Lots of well-developed tools for implementing  Yacc, Lex But Visual Basic is a well-developed tool for GUIs Implementation cost/time is usually a red herring

9. ITM 7349 Some CL Disadvantages Error-prone Harder to learn Requires typing With added power, comes added responsibility and danger  UNIX –% rm -r f* –Deletes every file that you have, provides no feedback, and you can’t get them back

10. ITM 73410 Unix Shell CL Disadvantages Learning takes a long time Hard to remember command names Some command names don’t make sense, so have to memorize No in-progress feedback - how much longer? System state is invisible, and have to know which commands to use to get which information Hard to make sense of outputs, such as with ls - no headings, no code interpretations No “look” No warning if bad things are going to happen No universal Undo; to reverse a command, have to know the inverse command (create directory, delete directory) Have to use man command to find desired command How to get help? Because commands are short, typos can lead to incorrect command Inconsistent flag meanings Inconsistent parameter orders Have to type a lot - touch typing needed

11. ITM 73411 CL Reflection Command languages are often maligned (for good reason) But increased functionality can win out over bad UI (e.g., UNIX)  Try to get both  Avoid excess functionality (comes at cost)

12. ITM 73412 CL Design Goals/Guidelines Consistency Good naming and abbreviations Doing your homework in design can help alleviate some of the negatives

13. ITM 73413 CL: Consistency Provide a consistent syntax  In general: Have options and arguments expressed the same way everywhere  UNIX fails! –Commands were developed by lots of different people at different organizations –No guidelines / style guide If commands are long, have simple and consistent abbreviations

14. ITM 73414 CL: Syntax – Be Consistent! Simple command list  e.g, vi, minimize keystrokes Commands plus arguments  realistic, can provide keyword parameters  % cp from=foo to=bar Commands plus options plus arguments  what you usually see Abbreviations make sense

15. ITM 73415 CL: Abbreviations Matter...

16. ITM 73416 CL: Order English: S-V-O (Subject-Verb-Object)  “you” is assumed as the subject - imperative! CL: S assumed (you)  Is V-O or O-V better?  % delete file or % file delete Keep ordering consistent  VO (Verb Object) seems to be the most natural  Typically need to pick where options go

17. ITM 73417 Dialog Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/Natural language 5. Gesture, pen

18. ITM 73418 WIMP Menus, Buttons, Forms, Icons Predominant desktop interface paradigm (along with some direct manipulation)

19. ITM 73419 Window Pros Facilitate multi-tasking, which many people do. Maps well onto overlapping sheets of paper on our desks, so is a familiar concept. Makes computer usage easier for most of us.

20. ITM 73420 Window Cons Can make concentrating on a single task hard (that incoming mail….) An extension of the cluttered desk :) May be unnecessary for dedicated-use environments that run a single application

21. ITM 73421 Menus - Many Different Types Pop-up Pull-down Radio buttons Pie buttons Hierarchical/cascading

22. ITM 73422 Pie Menus From Second Life

23. ITM 73423 Pop-up Hierarchical

24. ITM 73424 Menu Pros and Cons What is good about menus? What is bad about menus?

25. ITM 73425 Menu Pros One keystroke or mouse operation vs. many - no typing! No memorization of commands  Recognition, not recall Limited input set

26. ITM 73426 Menu Cons Less direct user control - have to find correct menu / menu item Not so readily extensible Slower than keyboarding for experienced users, at least without accelerators

27. ITM 73427 Menu Items Create groups of logically similar items Cover all possibilities Ensure that items are non-overlapping Keep wording concise, understandable

28. ITM 73428 Menu Item Ordering What rules do Mac, Netscape, Windows, etc, use?

29. ITM 73429 Menu Item Order - Some Rules One possible methodology (priority order) 1.Natural order (if exists) 2.Time – e.g. Breakfast, Lunch, Dinner 3.Numeric ordering – e.g. Point sizes for font 4.Categorical - with visual separation 5.Frequency of use 6.Order of use 7.Alphabetical - esp for long lists Don’t change dynamically!

30. A Good Menu Example Logical grouping Visual separation of groups Disabled items “grayed out” Shortcuts shown … indicates leads to dialogue Go forth and find some bad examples! 30ITM 734

31. 31 Dialog Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/Natural language 5. Gesture, pen ITM 734

32. 32 DM What is direct manipulation? That is, how can we characterize DM?  How do we know it when we see it? ITM 734

33. 33 DM Characteristics Continuous visibility of the objects and actions of interest Rapid, reversible, incremental actions whose effect is immediately noticeable Replacement of command language syntax by direct manipulation of object of interest (physical actions, buttons, etc.) Shneiderman ‘82 ITM 734

34. 34 DM Examples Examples  WYSIWYG text and graphics editors  VISICALC - 1 st electronic spreadsheet  CAD  Desktop metaphor  Video games  3D immersive Virtual Reality systems ITM 734

35. 35 DM Essence Representation of reality that can be manipulated The user is able to apply intellect directly to the task The tool itself seems to disappear ITM 734

36. 36 Some DM Advantages Easier to learn & remember, particularly for novices Direct WYSIWYG Flexible, easily reversible actions helps reduce anxiety in users User in control ITM 734

37. 37 DM Advantages Provides context & instant visual feedback so user can tell if objectives are being achieved Easier to remember how to use if are infrequent user Exploits human use of visual spatial cues Limits types of errors that can be made Reduces need for error messages ITM 734

38. 38 Some DM Disadvantages Screen space intensive (info not all that dense) Need to learn meaning of components of visual representation Visual representation may be misleading Mouse ops may be slower than typing Subtle aspects may not be self- explanatory ITM 734

39. 39 DM Disadvantages Not good at  Repetition  History keeping (harder)  Certain tasks (Change all italics to bold) ITM 734

40. 40 What systems use DM? Word? Powerpoint? Unix? Cell Phones? ITM 734

41. 41 Any Way to Combine DM and CL? Show commands that DM is using Show DM effects of commands  Word 2007 font changes … ITM 734

42. 42 Ultimately... In end, must characterize direct manipulation by feeling of directness and illusion of manipulating objects at hand ITM 734

43. 43 Dialog Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/natural language 5. Gesture & pen

44. ITM 73444 When to Use Speech Hands busy Mobility required Eyes occupied Conditions preclude use of keyboard Visual impairment Physical limitation

45. ITM 73445 Speech What is speech?  Vibrations of vocal cords creates sound “ahh”  Mouth, throat, tongue, lips shape sound English speech  40 phonemes; 24 consonants, 16 vowels Sounds transmit “language”

46. ITM 73446 Waveform & Spectrogram Speech does not equal written language

47. ITM 73447 Parsing Sentences "I told him to go back where he came from, but he wouldn't listen."

48. ITM 73448 Speech Input Speaker recognition Speech recognition Natural language understanding

49. ITM 73449 Speaker Recognition Tell which person it is (voice print) Could also be important for monitoring meetings, determining speaker

50. ITM 73450 Speech Recognition Primarily identifying words Improving all the time Commercial systems:  IBM ViaVoice, Dragon Dictate,...

51. ITM 73451 Demonstrated Effectiveness of Command Input via Speech Using MacDraw to re-create drawings Baseline - use mouse to menu-select commands and give positions on drawing Speed improvement using keyboard accelerators - 14.5% Speed improvement using voice - 21.2% Pausch, R. and Leatherby, J. H. "A Study Comparing Mouse-Only Input vs. Mouse-Plus-Voice Input for a Graphical Editor," Proceedings of the AVIOS `90 Voice I/O Systems Applications Conference, September 1990, pages 227 - 231, and http://www- 2.cs.cmu.edu/~stage3/publications/91/conferences/avios/paper.html

52. ITM 73452 Natural Language Understanding Putting meaning to the words Input might be spoken or typed Holy grail of Artificial Intelligence problems

53. ITM 73453 Eliza - a “Faked” Understanding Person : Men are all alike. ELIZA : In what way? Person : They're always bugging us about something or other. ELIZA : Can you think of a specific example? Person : Well, my boyfriend made me come here. ELIZA : Your boyfriend made you come here? Person : He says I'm depressed much of the time. ELIZA : I am sorry to hear that you are depressed. Person : It's true. I am unhappy. ELIZA : Do you think coming here will help you not to be unhappy? Person : I need some help, that much seems certain. ELIZA : What would it mean to you if you got some help? Person : Perhaps I could learn to get along with my mother. ELIZA : Tell me more about your family http://www-ai.ijs.si/eliza/eliza.html Weizenbaum, J., "ELIZA -- A computer program for the study of natural language communication between man and machine", Communications of the ACM 9(1):36-45, 1966

54. ITM 73454 SR/NLU Advantages Easy to learn and remember Powerful Fast, efficient (not always) Little screen real estate

55. ITM 73455 SR/NLU Disadvantages Doesn’t work good enough yet Assumes knowledge of problem domain  Not prompted, like menus Requires typing skill (if keyboard) Enhancements are invisible Expensive to implement

56. ITM 73456 Recall A natural language interface need not be speech A speech interface need not use natural language (might be more command language-like)

57. ITM 73457 Designing the Interaction Constrain vocabulary  Limit valid commands  Structure questions wisely (Yes/No)  Manage the interaction Slow speech rate, but concise phrases Design for failsafe error recovery Process preview & progress indicator

58. ITM 73458 Dialog Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/natural language 5. Gesture & pen

59. ITM 73459 PDAs Becoming more common and widely used Smaller display (160x160), (320x240) Few buttons, interact through pen Recent Improvements  Wireless, color, more memory, better CPU, better OS, BlueTooth Traditional PDAs are caught between  Increasingly capable cell phones  Increasingly portable laptops

60. ITM 73460 Input Pen is dominant form Main techniques  Free-form ink  Soft keyboard  Numeric keyboard => text  Stroke recognition - strokes not in the shape of characters  Hand printing / writing recognition Sometimes can connect keyboard

61. ITM 73461 Free-form Ink Ink is the data, take as is Human is responsible for understanding and interpretation Like a sketch pad

62. ITM 73462 Soft Keyboards Common on PDAs and mobile devices

63. ITM 73463 Numeric Keypad -T9 Tegic Communications developed You press out letters of your word, it matches the most likely word, then gives optional choices Faster than multiple presses per key Used in mobile phones Demo: http://www.nuance.com/t9/textinputhttp://www.nuance.com/t9/textinput/#

64. ITM 73464 Stroke Recognition - Graffiti Graffiti - Unistroke alphabet on Palm PDA  What are your experiences with Graffiti?

65. ITM 73465 Stroke Recognition - Cirrin Developed by Jen Mankoff (GT -> Berkeley CS Faculty) Word-level unistroke technique UIST ‘98 paper Use stylus to go from one letter to the next ->

66. ITM 73466 Stroke Recogntion - Quikwriting Developed by Ken Perlin Demo http://mrl.nyu.edu/projects/ http://mrl.nyu.edu/projects/ quikwriting/Quikwrite2.html

67. ITM 73467 Quikwriting Example pl http://mrl.nyu.edu/projects/quikwriting/ e Said to be as fast as graffiti, but have to learn more

68. ITM 73468 Pen Gesture Commands -Might mean delete -Insert -Paragraph Define a series of (hopefully) simple drawing gestures that mean different commands in a system Denim - http://dub.washington.edu/projects/denim/download/windows/ http://dub.washington.edu/projects/denim/download/windows/ download_alone.shtml