Predictive Assessment of Usability Laura Marie Leventhal

Evaluating the Usability of an Interaction Design As you are designing your interaction solution, wouldn't it be nice to predict, in advance, that one of your possible designs would be more learnable, usable… than others? In other words, wouldn't it be nice if you could predict quantitatively something about the quality of your design from a usability perspective (in Chapter 11, we discuss design quality from a software engineering perspective.) A number of researchers have considered this to be a worthy goal as well.

Predictive Assessment Models - GOMS GOMS stands for goals, operators, models and selection. The idea behind GOMS was that a user's knowledge of an interface task could be broken down into small mental and physical actions. Each of these actions has a characteristic time to think or do. Based on these characteristic times, one could make a prediction as to how long the interface task would take with a specific interface. Given interface designs, one could then compute the time for some standardized tasks or could compare the predicted times between two interfaces and make a comparison.

Different Flavors of GOMS The keystroke-level model is probably the simplest. In the keystroke-level model, there are a limited set of operator categories and each category has a characteristic time. The focus of the KLM method is on keystroke and mouse level operators and does not focus on goals, methods or selection. KLM also includes a number of heuristics to that describe the use of the M operator. Show example.

Other flavors of GOMS CMN GOMS for Card-Moran-Newell GOMS. This variation of GOMS was also suggested in Card, Moran and Newell, In CMN GOMS, the analyst develops a detailed goal hierarchy. The analyst then generates methods to accomplish the subgoals, using a psuedo-code like method description. Methods are described by other methods, operators and conditionals. NGOMSL (Natural GOMS language) was developed by Kieras (1988) and is described in John and Kieras (1996a and 1996b). NGOMSL shows a GOMS model with a structured language and is tied to a more detailed theoretical model of the mental activities of the user than either CMN or KLM. CPM-GOMS (Cognitive-Perceptual-Motor), described in John (1990) and John and Kieras (1996a and 1996b) supports parallel sequences of operators, if this is appropriate to the task.

Usefulness of Predictive Assessment GOMS has been used successfully as a predictive tool in a number of instances. For example, Gray, John, and Atwood (1993) did a CPM GOMS analysis on a proposed workstation replacement for telephone operators at the NYNEX company. As a result of their analysis they found that operators were actually more effective with the existing workstations than the proposed one. The project was scrapped and saved NYNEX about two million dollars per year!

Challenges of Predictive Assessment and Assessment Models GOMS assumes idealized or expert performance on the part of the user. The GOMS hierarchy is somewhat subjective - two different designers may develop two different GOMS models for the same interface task and come up with different predictions. Rosson and Carroll (2002) note that models such as GOMS models do not include parameters to model learning or social and organizational relationships that potentially impact usability.