Interaction modalities Command languages, direct manipulation, and WIMP

What’s wrong with this? 0.Borrow a book from the library 1.go to the library 2. log in to library catalogue 2.1 access the search screen 2.2 enter search criteria 2.3 identify required book 2.4 note location 3.go to correct shelf and retrieve book 3.1 use library map to find shelf 4.take book to checkout counter

Now what’s wrong? 0.In order to borrow a book from the library 1.go to the library 1.1 enter library 1.2 go to computer 1.3 log into catalogue 2.find book 2.1 access library catalogue 2.2 access the search screen 2.3 enter search criteria 2.4 identify required book 2.5 note location 3.go to correct shelf and retrieve book 4.check out book

Don’t forget your plans! 0.In order to borrow a book from the library 1.go to the library 2.find the required book 2.1 access library catalogue 2.2 access the search screen 2.3 enter search criteria 2.4 identify required book 2.5 note location 3.go to correct shelf and retrieve book 4.check out book Plan 0: do 1, 2, 3; repeat 2, 3, if book not found; then do 4 Plan 2: do 2.1, 2.2; repeat 2.3 until found; 2.4, 2.5

Interface types 1980s interfaces Command line WIMP/GUI 1990s interfaces Advanced graphical (multimedia, virtual reality, information visualization) Web Speech (voice) Pen, gesture, and touch Appliance 2000s interfaces Mobile Multimodal Shareable Tangible Augmented and mixed reality Wearable Robotic

Interaction Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/Natural language 5. Pen & Touch

General Issues in Interaction Style Who is in control - user or computer Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Command Languages Earliest UI interaction paradigm Examples: MS-DOS shell, UNIX, Linux Little or nothing is visible so… – Work primarily by recall, not recognition – Heavy memory load Poor choice for novices but...

CL Advantages Advantages for experts – Speed, conciseness % ls (hard to beat) – Can express actions beyond a limited set Flags, piping one command to another – Repetition, extensibility Scripting, macros – Easier implementation, less overhead – Power Abstraction, wild cards

CL Dangers With added power, comes added responsibility and danger – UNIX % rm -r * Deletes every file that you have, and you can’t get them back

CL Design Goals Consistency – Have options and arguments expressed the same way everywhere Good naming and abbreviations UNIX fails here because commands were developed by lots of different people at different organizations No guidelines provided

Names and Abbreviations Specificity versus Generality – General words More familiar, easier to accept – Specific (typically better) More descriptive, meaningful, distinctive – (Nonsense does surprisingly well in small set) Abbrevs. allow for faster actions – Expert performance begins to be dominated by motor times such as # of keystrokes – Not good idea for novices – (Allow but don’t require)

General Issues - CL Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Interaction Styles 1. Command language 2. WIMP 3. Direct Manipulation 4. Speech/Natural language 5. Gesture, pen, VR

WIMP/GUI interfaces Xerox Star first WIMP -> rise to GUIs Windows – could be scrolled, stretched, overlapped, opened, closed, and moved around the screen using the mouse Icons – represented applications, objects, commands, and tools that were opened when clicked on Menus – offering lists of options that could be scrolled through and selected Pointing device – a mouse controlling the cursor as a point of entry to the windows, menus, and icons on the screen

Windows Windows were invented to overcome physical constraints of a computer display – Overlapping windows make more efficient use of screen real estate – Can allow dragging and dropping between them – But can become cluttered or overwhelming Multiple windows can make it difficult to find desired one, so techniques used – Listing, iconising, shrinking

Questions How many windows do you generally have open at once? How do you arrange them? How do you switch between them?

Menus Advantages: – 1 keystroke or mouse operation vs. many – No memorization of commands – Limited input set Disadvantages: – Less direct user control - have to find correct menu / menu item – Not so readily extensible – Slower than keyboarding for experienced users, at least without accelerators

Menu Items Various types: – Flat – Cascading – Pop-up (contextual) Organization strategies – Create groups of logically similar items – Cover all possibilities – Ensure that items are non-overlapping – Keep wording concise, understandable

A Good Example Logical grouping Visual separation of groups Disabled items “grayed out” Shortcuts shown … indicates leads to dialogue

Icons Icons are assumed to be easier to learn and remember than commands Can be designed to be compact and variably positioned on a screen Issues: – Consistency – Legibility – Convey meaning

Simple icons

Newer icons

Pointers Choosing commands from menus and icons Allows users to specify location parameter of a command – Edit HERE, Put this file HERE Also displays system state info: – Tracking – Busy – Hints – Modes (text cursor vs. selection pointer)

General Issues - WIMP Who is in control - user or computer Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Direct Manipulation Essence Representation of reality that can be manipulated The user is able to apply intellect directly to the task The tool itself seems to disappear

Direct Manipulation Definition: 1) Continuous visibility of the objects and actions of interest 2) Rapid, reversible, incremental actions whose effect is immediately noticeable 3) Replacement of command language syntax by direct manipulation of object of interest (physical actions, buttons, etc.) Shneiderman ‘82

Direct Manipulation Examples – WYSIWYG editors and word processors – VISICALC - 1 st electronic spreadsheet – CAD – Desktop metaphor – Video games

Example: maps

More Psychological View What is directness? (not always done well) Related to two things: – Distance – Engagement Unobtrusive and responsive Hutchins, Hollan, Norman ‘86 Goals System Execution Evaluation

Example: Homefinder

DM Advantages Easier to learn & remember, particularly for novices Flexible, easily reversible actions helps reduce anxiety in users Provides context & instant visual feedback so user can tell if objectives are being achieved Exploits human use of visual spatial cues Limits types of errors that can be made

DM Problems Screen space intensive (info not all that dense) Need to learn meaning of components of visual representation Visual representation may be misleading Mouse ops may be slower than typing Not self-explanatory (no prompts) Not everything can be represented visually

DM Problems Not good at – Repetition – History keeping (harder) – Certain tasks (Change all italics to bold) – Abstract elements (variables) – Macros harder

General Issues - DM Initial training required Learning time to become proficient Special skills Speed of use Generality/flexibility Power Screen space required Computational resources required Match to the user’s activity

Interaction styles: next time 1. Command language 2. WIMP 3. Direct Manipulation 4. Speech/Natural language 5. Touch and Pen