Direct Manipulation and Virtual Environment Chapter 5 Direct Manipulation and Virtual Environment Copyright © 2005, Pearson Education, Inc.

Introduction Positive feelings associated with good user interfaces: Mastery of the interface (user in control) Competence in performing tasks Ease in learning the system originally and in assimilating advanced features Enjoyment in using the system Eagerness to show the system off to novices Desire to explore more powerful aspects of the system Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (1) Command line vs. display editors and word processors Training times with display editors are much less than line editors Line editors are generally more flexible and powerful By the early 1990s, the display editors, described as what you see is what you get (WYSIWYG), had become standard for word processors. The advances of WYSIWYG word processors: Display a full page of text showing 20 to 60 lines of text simultaneously gives the reader a clearer sense of context for each sentence, while permitting easier reading and scanning of the document. By contrast, working with the one-line view offered by line editors is like seeing the world through a narrow card board tube. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems The advances of WYSIWYG word processors (cont.): Display of the document in the form that it will appear when the final printing is done Table, lists, page breaks, skipped lines, section headings, centered text and figures can be viewed in their final form. Show cursor action Seeing an arrow, underscore, or blinking box on the screen gives the operator a clear sense of where to focus attention and to apply action. Control cursor motion through physically obvious and intuitively natural means Arrow keys or cursor-motion devices – such as a mouse, or trackpad – provide natural physical mechanisms for moving the cursor. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems The advances of WYSIWYG word processors (cont.): Use of labeled icon for actions Most word processors have labeled icons in a toolbar for frequent actions. These buttons act as a permanent menu-selection display to remind users of the features and to enable rapid selection. Display of the results of an action immediately When users press a button to move the cursor or center text, the results are shown immediately on the screen. Deletions are apparent immediately: the character, word, or line is erased, and the remaining text is rearranged. In contrast, with line editors, user must issue print or display commands to see the results of changes. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems The advances of WYSIWYG word processors (cont.): Provide rapid response and display Most display editors operate at high speed; a full page of text appears in a fraction of a second. This high display rate, coupled with short response time, produces a satisfying sense of power and speed. Offer easily reversible actions An alternative offered by many display editors is a simple undo action to return the text to the state that is was in before the previous action. Easy reversibility reduces user anxiety about making a mistake or destroying the file. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) Technologies that derive from the word processor: Integration of graphics, spreadsheets, animations, photographs, and so on is done in the body of a document. Desktop publication software produces sophisticated printed formats with multiple columns and allows output to high-resolution printers. Multiple fonts and color permit preparation of high-quality documents, newsletters, reports, newspapers, or books. Examples include Adobe PageMaker and QuarkXPress. Slide-presentation software produces color text and graphic layouts for use directly from the computer with a large-screen projector to allow animations. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) Technologies that derive from the word processor (cont.): Hypermedia environments and the World Wide Web allow users to jump from one page or article to another with selectable buttons or embedded hot links. Improved macro facilities enable users to construct, save, and edit sequences of frequently used actions. Spell checker and thesaurus Grammar checkers Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) (2) The VisiCalc spreadsheet (the first electronic spreadsheet) and its descendants VisiCalc users delighted in watching the program propagate changes across the screen. “Instantly calculating electronic worksheet” that permitted computation and immediate display of results across 254 rows and 63 columns Competitors to VisiCalc emerged quickly; they made attractive improvements to the user interface and expanded the tasks that were supported. Lotus 1-2-3 dominated the market in the 1980s, but the current leader is Microsoft’s Excel, which has numerous features and specialized additions. Excel and other modern spreadsheet programs offer numerous graphics displays, multiple windows, statistical routines, and database access. The huge numbers of features are invoked with menus or toolbars, and extensibility is provided by powerful macro facilities. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) (3) Video games From PONG to Nintendo GameCube, Sony PlayStation 2, and Microsoft Xbox Field of action is visual and compelling Commands are physical actions whose results are immediately shown on the screen No syntax to remember Most games continuously display a score Educational game - Direct manipulation in SimCity (learn about urban planning) Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) (4) Computer-aided design Most computer-aided design (CAD) systems for automobiles, electronic circuitry, aircraft, or mechanical engineering use principles of direct manipulation. Building and home architects now have powerful tools such as AutoCad with components to handle structural engineering, floorplans, interiors, landscaping, plumbing, electrical installation, and much more. With programs like this, the operator may see a circuit schematic on the screen and, with mouse clicks be able to move components into or out of the proposed circuit. Similarly, newspaper-layout artists or automobilebody designers can easily try multiple designs in minutes (generate alternatives easily). The basic strategy for this design is to eliminate the need for complex commands that the operator might only need to recall during a once-a-year emergency. Copyright © 2005, Pearson Education, Inc.

Examples of Direct-Manipulation Systems (cont.) (5) Office automation Xerox Star was a pioneer with sophisticated formatting Users could move (but not drag) a document icon to a printer icon to generate a hardcopy printout. Rapid and continuous graphical interaction Microsoft Windows is a descendant The Microsoft Windows design is still a close relative of the Macintosh design, and both are candidates for substantial improvements in window management, with simplifications for novices and increased power for sophisticated users. Copyright © 2005, Pearson Education, Inc.

Discussion of Direct Manipulation Problems with direct manipulation Spatial or visual representations can be too spread out Visual representations are not necessarily an improvement over text, especially for blind or vision-impaired users who will need special software. Graphical user interfaces were a setback for vision-impaired users, who appreciated the simplicity of linear command languages. Designs may force valuable information off of the screen, requiring scrolling or multiple actions. For experienced users, a tabular textual display of 50 document names may be more appropriate than only 10 graphic document icons with the names abbreviated to fit the icon size. Users must learn the graphical representations A graphic icon may be meaningful to the designer, but for users may requires as much learning time as a word, or more. The visual representation may be misleading Typing commands with the keyboard may be faster For experienced typists, taking a hand off the keyboard to move a mouse or point with a finger may take time than typing the relevant command. Copyright © 2005, Pearson Education, Inc.

Discussion of Direct Manipulation (cont.) The attraction of direct manipulation is apparent in the enthusiasm of the users. Three principles of direct manipulation: Continuous representation of the objects and actions of interest Physical actions or presses of labeled buttons instead of complex syntax Rapid incremental reversible operations whose effect on the object of interest is immediately visible Copyright © 2005, Pearson Education, Inc.

Discussion of Direct Manipulation (cont.) Using these three principles, it is possible to design systems that have these beneficial attributes: Novices learn quickly Experts work rapidly Intermittent users can retain concepts Error messages are rarely needed Users see if their actions are furthering their goals Users experience less anxiety (action can be reversed easily) Users gain confidence and mastery (they are the initiators of action, they feel in control) Copyright © 2005, Pearson Education, Inc.

Discussion of Direct Manipulation (cont.) Direct manipulation is a human-computer interaction style that was defined by Ben Shneiderman and which involves continuous representation of objects of interest, and rapid, reversible, incremental actions and feedback. Having real-world metaphors for objects and actions can make it easier for a user to learn and use an interface (some might say that the interface is more natural or intuitive), and rapid, incremental feedback allows a user to make fewer errors and complete tasks in less time, because they can see the results of an action before completing the action. An example of direct-manipulation is resizing a graphical shape, such as a rectangle, by dragging its corners or edges with a mouse. Copyright © 2005, Pearson Education, Inc.

Visual Thinking and Icons An icon is an image, picture, or symbol representing a concept Icon-specific guidelines Represent the object or action in a familiar manner Limit the number of different icons Make icons stand out from the background Consider three-dimensional icons (they are eye-catching but also can be distracting) Ensure a selected icon is visible from unselected icons (distinctive) Design the movement animation (when dragging an icon, the user might move the whole icon) Copyright © 2005, Pearson Education, Inc.

Visual Thinking and Icons (cont.) Icon-specific guidelines (cont.) Add detailed information, such as shading to show the size of a file (larger shadow indicates larger file), Color to show the age of a document (older might be yellower or grayer) Explore combinations of icons to create new objects or actions, for example, dragging a document icon to a folder, trash can, outbox, or printer icon has great utility. Can a user set security levels by dragging a document or folder to a guard dog or police car icon? Copyright © 2005, Pearson Education, Inc.

3D Interfaces Some designers dream about building interfaces that approach the richness of three-dimensional (3D) reality. They believe that the closer the interfaces are to the real world, the easier usage will be. “Enhanced” interfaces, better than reality, can help reduce the limitations of the real-world, e.g., providing simultaneous views. User can magically change colors or shapes, duplicate objects, shrink/expand objects, group/ungroup components. Among the many innovations, there have been questionable 3D prototypes, such as Digital libraries: showing books on shelves may be nice for browsing, but it inhibits search and linking Copyright © 2005, Pearson Education, Inc.

3D Interfaces (cont.) Copyright © 2005, Pearson Education, Inc.

3D Interfaces (cont.) Features for effective 3D (checklist for designers): Minimize the number of navigation steps for users to accomplish their tasks. Keep text readable. Avoid unnecessary visual clutter, distraction, contrast shifts, and reflections. Simplify user movement. (avoid surprises like going through walls) Prevent errors. Simplify object movement Organize groups of items in aligned structures to allow rapid visual search. Copyright © 2005, Pearson Education, Inc.

3D Interfaces (cont.) Guidelines for inclusion of enhanced 3D features: Provide overviews so users can see the big picture Offer X-ray vision so users can see into or beyond objects. Provide history keeping (recording, undoing, replaying, editing) Permit rich user actions on objects (save, copy, annotate, share, send) Enable remote collaboration (synchronous, asynchronous) Give users control over explanatory text and let users select for details on demand. (pop-up, screen tips) Copyright © 2005, Pearson Education, Inc.

3D Interfaces (cont.) Guidelines for inclusion of enhanced 3D features (cont.): Offer tools to select, mark, and measure. Implement dynamic queries to rapidly filter out unneeded items. Support semantic zooming and movement (simple action brings object front and center) Allow multiple coordinated views (users can see data in more than one arrangement at a time) Develop novel 3D icons to represent concepts that are more recognizable and memorable. Copyright © 2005, Pearson Education, Inc.

Virtual and Augmented Reality Virtual reality or virtual environment breaks the physical limitations of space and allow users to act as though they were somewhere else E.g. flight-simulator designers work hard to create the most realistic experience for fighter or airline pilots. 5DT (Fifth Dimension Technologies) is a high technology company specializing in Virtual Reality (VR), see http://www.5dt.com/ Augmented reality shows the real world with an overlay of additional overlay; for example, While users are looking at the walls of a building, their semitransparent eyeglasses show where the electrical wires or plumbing are located. Medical applications, such as allowing surgeons to look at a patient while they see an overlay of an x-ray to help locate a cancer tumor. Copyright © 2005, Pearson Education, Inc.

Virtual and Augmented Reality (cont.) Successful virtual environments depend on the smooth integration of: Visual Display The head-mounted displays block other images, so the effect is more dramatic, and head motion produces new images, so the users can get the impression of 360-degree coverage. Head position sensing Head-mounted displays can provide differing views depending on head position. Look to the right, and you see a forest; look to the left, and the forest gives way to a city. Copyright © 2005, Pearson Education, Inc.

Virtual and Augmented Reality (cont.) The Head-mounted Display (HMD) offers the user affordable quality, high resolution (SVGA), a crisp image and superior sound quality packaged in a sleek, comfortable and extremely light headmount design. Copyright © 2005, Pearson Education, Inc.

Virtual and Augmented Reality (cont.) Successful virtual environments depend on the smooth integration of (cont.): Hand-position sensing The DataGlove is an innovative invention that continues to be refined and improved beyond its current low resolution. Sensors for other body parts, such as knees, arms, or legs, are finding uses in sports training and movement capture. The DataGlove is designed to satisfy the stringent requirements of modern Motion Capture and Animation Professionals. Copyright © 2005, Pearson Education, Inc.

Virtual and Augmented Reality (cont.) Successful virtual environments depend on the smooth integration of (cont.): Force feedback Hand-operated remote-control devices for performing experiments in chemistry laboratories or for handling nuclear materials provide force feedback that gives users a good sense of when they grasp and object or bump into one. Sound input and output Making convincing sounds at the correct moment with full three-dimensional effect is possible, but it too is hard work. The digital sound hardware is adequate, but the software tools are still inadequate. Copyright © 2005, Pearson Education, Inc.