1. Interaction modalities Command languages, direct manipulation, and WIMP

2. Midterm – date change? Monday, Oct. 6

3. What’s wrong with this? 0.Borrow a book from the library 1.go to the library 2. log in to library catalogue 2.1 access the search screen 2.2 enter search criteria 2.3 identify required book 2.4 note location 3.go to correct shelf and retrieve book 3.1 use library map to find shelf 4.take book to checkout counter

4. Now what’s wrong? 0.In order to borrow a book from the library 1.go to the library 2.if you don’t know where the book is, find the required book 2.1 access library catalogue 2.2 access the search screen 2.3 enter search criteria 2.4 identify required book 2.5 note location 3.go to correct shelf and retrieve book 4.check out book

5. Don’t forget your plans! 0.In order to borrow a book from the library 1.go to the library 2.find the required book 2.1 access library catalogue 2.2 access the search screen 2.3 enter search criteria 2.4 identify required book 2.5 note location 3.go to correct shelf and retrieve book 4.check out book Plan 0: do 1, 2, 3; repeat 2, 3, if book not found; then do 4 Plan 2: do 2.1, 2.2; repeat 2.3 until found; 2.4, 2.5

6. Interface types 1980s interfaces Command line WIMP/GUI 1990s interfaces Advanced graphical (multimedia, virtual reality, information visualization) Web Speech (voice) Pen, gesture, and touch Appliance 2000s interfaces Mobile Multimodal Shareable Tangible Augmented and mixed reality Wearable Robotic

7. Interaction Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/Natural language 5. Gesture & pen

8. General Issues in Interaction Style Who is in control - user or computer Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

9. Command Languages Earliest UI interaction paradigm Examples: MS-DOS shell, UNIX, Linux Little or nothing is visible so… – Work primarily by recall, not recognition – Heavy memory load Poor choice for novices but...

10. CL Advantages Advantages for experts – Speed, conciseness % ls (hard to beat) – Can express actions beyond a limited set Flags, piping one command to another – Repetition, extensibility Scripting, macros – Easier implementation, less overhead – Power Abstraction, wild cards

11. CL Dangers With added power, comes added responsibility and danger – UNIX % rm -r * Deletes every file that you have, and you can’t get them back

12. CL Design Goals Consistency – Have options and arguments expressed the same way everywhere Good naming and abbreviations UNIX fails here because commands were developed by lots of different people at different organizations No guidelines provided

13. Names and Abbreviations Specificity versus Generality – General words More familiar, easier to accept – Specific (typically better) More descriptive, meaningful, distinctive – (Nonsense does surprisingly well in small set) Abbrevs. allow for faster actions – Expert performance begins to be dominated by motor times such as # of keystrokes – Not good idea for novices – (Allow but don’t require)

14. General Issues - CL Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

15. Interaction Styles 1. Command language 2. Direct manipulation 3. WIMP 4. Speech/Natural language 5. Gesture, pen, VR

16. Direct Manipulation Essence Representation of reality that can be manipulated The user is able to apply intellect directly to the task The tool itself seems to disappear

17. Direct Manipulation Definition: 1) Continuous visibility of the objects and actions of interest 2) Rapid, reversible, incremental actions whose effect is immediately noticeable 3) Replacement of command language syntax by direct manipulation of object of interest (physical actions, buttons, etc.) Shneiderman ‘82

18. Direct Manipulation Examples – WYSIWYG editors and word processors – VISICALC - 1 st electronic spreadsheet – CAD – Desktop metaphor – Video games

19. Example: maps

20. More Psychological View What is directness? (not always done well) Related to two things: – Distance – Engagement Unobtrusive and responsive Hutchins, Hollan, Norman ‘86 Goals System Execution Evaluation

21. Example: Homefinder

22. DM Advantages Easier to learn & remember, particularly for novices Flexible, easily reversible actions helps reduce anxiety in users Provides context & instant visual feedback so user can tell if objectives are being achieved Exploits human use of visual spatial cues Limits types of errors that can be made

23. DM Problems Screen space intensive (info not all that dense) Need to learn meaning of components of visual representation Visual representation may be misleading Mouse ops may be slower than typing Not self-explanatory (no prompts) Not everything can be represented visually

24. DM Problems Not good at – Repetition – History keeping (harder) – Certain tasks (Change all italics to bold) – Abstract elements (variables) – Macros harder

25. General Issues - DM Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

26. Dialog Design 1. Command language 2. Direct manipulation 3. WIMP 4. Speech/Natural language 5. Gesture, pen, VR

27. WIMP Windows, Icons, Menus, Pointers – Focus: Menus, Buttons, Forms Predominant interface paradigm now (with some direct manipulation added) Advantages: – ?

28. Menus Advantages: – 1 keystroke or mouse operation vs. many – No memorization of commands – Limited input set Disadvantages: – Less direct user control - have to find correct menu / menu item – Not so readily extensible – Slower than keyboarding for experienced users, at least without accelerators

29. Menu Items Various types: – Flat – Cascading – Pop-up (contextual) Organization strategies – Create groups of logically similar items – Cover all possibilities – Ensure that items are non-overlapping – Keep wording concise, understandable

30. A Good Example Logical grouping Visual separation of groups Disabled items “grayed out” Shortcuts shown … indicates leads to dialogue

31. Presentation Sequence Forms, dialogue boxes, menus Use natural if available – Time e.g. Breakfast, Lunch, Dinner – Numeric ordering e.g. Point sizes for font Other possibilities: – Alphabetical – Group related items – Frequently used first – Most important first

32. Pointers Allows users to specify location parameter of a command – Provides for some of that direct manipulation – Put this file HERE Also displays system state info: – Tracking – Busy – Hints – Modes (text cursor vs. selection pointer)

33. Other WIMP issues Windows management – How to locate, move, find – Transfer information between – Switch attention between Icons – Need graphic design attention Toolbars

34. Questions How many windows do you generally have open at once? How do you arrange them? How do you switch between them?

35. Newer icons

36. Simple icons

38. General Issues - WIMP Who is in control - user or computer Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

39. Dialog Design: next time 1. Command language 2. Direct manipulation 3. WIMP 4. Speech/Natural language 5. Gesture, pen, VR