Cognitive Modeling for HCI * 07/16/96 Cognitive Modeling for HCI David Crow david.crow@acm.org www.cs.cmu.edu/~dccrow/ 11/12/2018 *

HF vs. HCI Human Factors Human-Computer Interaction * 07/16/96 HF vs. HCI Human Factors discovers and applies information about human behavior, abilities, limitations and other characteristics to the design of tools, machines, systems, tasks, jobs and environments for save comfortable and effective human use (Sanders & McCormick,1987, pp..5) Human-Computer Interaction is concerned with such issues as user’s capabilities and preferences and the degree to which interfaces support their activities. I used to think there was a difference between human factors and human-computer interaction. But I have changed my mine after spending time working and learning in both communities. I have come to realize that we are really talking about the same things, just we have approached them differently in the past. But things are changing and Human Factors is becoming more open to new methodologies, new techniques, new people (those other than cognitive psychologists and industrial engineers). Human-Computer Interaction has seen interfaces move beyond the desktop. They have begun to use methodologies and whole-system approaches previous used in human factors. But both fields address the issues of human behavior and limitations through empirical investigation, theory and modeling. 11/12/2018 *

Cognitive Modeling Is a form of theory Uses empirical results * 07/16/96 Cognitive Modeling Is a form of theory Uses empirical results Develops analytical models that make quantitative predictions about a user’s performance Cognitive modeling is a form of theory. It uses empirical results to develop analytical models that make quantitative predictions about a users’ performance. Applying theory and models from cognitive psychology to the design of human-computer interfaces and dialogs. The idea is that this theory or model will be very useful because with the many design decisions on a project there are too many alternatives to build and test them all. A cognitive model can predict performance and inform design decisions that will reduce the development cycle and lead to a better product. 11/12/2018 *

* 07/16/96 History Information processing concepts borrowed from computer science and engineering During WWII, complex military displays analyzed to improve their design Led to mathematical models of human behavior (Fitts Law, Hick-Hyman Law) Why talk about the history of a science? George Santayana (1863-1952) “Those who cannot remember the past are condemned to repeat it” During the late 1940s and early 1950s psychologists borrowed concepts of information processing from computer scientists and engineers. They used a computer metaphor to describe a human being as an information processing machine. This work was different from the behaviorists of the early 1900s, modern psychology was influenced by 3 different factors: The research on human performance during WWII The development of information-processing approach particularly artificial intelligence which tries to get computers to behave intelligently Research by Noam Chomsky in linguistics analyzing the structure of language The combination of these influences and the large data sets collected to improve the design of complex military equipment including visual displays lead to the development of mathematical models of human behavior and new applications of statistical analyses. The results of these new statistical approaches can be found in the “psychological laws” discovered. 11/12/2018 *

* 07/16/96 History (continued) Cognitive Psychology first described by Neisser in 1967 Appearance of the journal of Cognitive Psychology in 1970 The role of the environment The growth of cognitive psychology continued and was give legitimacy with the publication of Ulric Neisser’s Cognitive Psychology in 1967. The book consisted of 6 chapters on attention and perception and 4 chapters on language, memory, and thought. In 1970, the journal of Cognitive Psychology first appeared and has made great strides in giving definition to the field. There has been a debate about the role the environment plays in cognition. J. J. Gibson argued that we need not to understand the internal structure to explain cognition but to look at cognition as a response to the structure in the environment. This is the ecological approach. There are arguments for both sides in this cognitive debate. But many researchers (Marr, Simon, Shepard, & Anderson) (including two whom I have studied with) have choose to argue that both the internal structure and the external structure complement each other in terms of coming to an understanding of cognition. It is here where the field of applied cognitive psychology has arose. By studying the external structure, i.e., the characteristics of technologies that lead to behaviors, and the internal structure, I.e., how people think and represent information to design technologies. 11/12/2018 *

Applied Cognitive Psychology * 07/16/96 Applied Cognitive Psychology Carnegie Mellon University in the early to mid-1970s Moved to XeroxPARC Culminated in Card, Moran & Newell’s (1983) The Psychology of Human-Computer Interaction 11/12/2018 *

* 07/16/96 Psychology of HCI “Theory-based, in the sense of articulating the mechanisms underlying the observed phenomena” Hard Science Engineering theories of the user Task analysis, calculation and approximation Card, Moran and Newell were insistent that the psychology of human-computer interaction be “theory-based, in the sense of articulating the mechanisms underlying the observed phenomena”. This ensured that their work would be a HARD SCIENCE. They began to develop engineering theories of the user through task analysis, calculation, and approximation. This work was a significant advance from the kind of cognitive modeling offered at the time. This work pioneered the work on cognitive engineering models and the explicit analyses of the knowledge people need to perform a procedure. It provides us the ability to predict how users will interact with proposed designs and validated this work in a psychological and engineering framework. 11/12/2018 *

Cognitive Models Constrain the design space Estimate total task time * 07/16/96 Cognitive Models Constrain the design space Estimate total task time Estimate training time and documentation Identify complex, error prone stages of the task Cognitive models are most useful in: Initially constraining the design space, so that one does not build an interface, for example, that requires more items to be kept in memory than will fit in working memory (WM) Answering specific design questions, so that one can decide, for example, between a dialogue that requires few keystrokes but difficult retrieval from memory or one that involves more keystrokes but is easier to remember Estimating the total time for task performance with sufficient accuracy to make decisions about how many people are need to staff the performance of a repetitive operational task on a computer Providing the base from which both to calculate training time and to guide training documentation to help the user determine in which situations which method is most efficient Knowing which stages of activity take the longest time or produce the most errors, in directing research toward the aspects of human-computer interaction that will have strong performance implications The framework has two key components: The model human processor and the GOMS model. 11/12/2018 *

* 07/16/96 Model Human Processor An idealized information processing model of human perceptual, motor and cognitive systems Describes behavior using 3 processors, 4 memories, 19 parameters and 10 principles of use The Model Human Processor is a general characterization of the human-information processing system in terms of both a system architecture and quantitative parameters of component performance. It is a summary of a large body of research in cognitive psychology. 11/12/2018 *

The GOMS Model a formal representation of routine cognitive skill * 07/16/96 The GOMS Model a formal representation of routine cognitive skill a detailed description of knowledge required by an expert user to perform a specific task without error a usefully approximation of the processes underlying human behavior The GOMS model is actually a family of models that describe the knowledge necessary and the four cognitive components of skilled performance. GOMS models are a way of describing what the user needs to know in order to perform computer-based tasks. 11/12/2018 *

GOMS Goals Operators Methods Selection rules * 07/16/96 GOMS Goals what the user wants to accomplish Operators the actions a user is able to take Methods well-learned subgoals and operators Selection rules guidelines for deciding between multiple methods The GOMS framework provides the basis for predicting the methods and operators users would follow in carrying out a particular well-known task (the goal) and , given a method, how long that task would take. GOMS strength is its ability to predict the time it takes a skilled user to execute a task based on the composite of actions of retrieving plans from long-term memory, choosing among alternative available methods depending of the features of the task at hand, keeping track of what has been done and what needs to be done, and executing the motor movements necessary for the keyboard and mouse. GOMS models are both serial (CMN-GOMS, NGOMSL, KLM) and parallel (CPM-GOMS, SOAR) in nature. And can be used to make useful approximations and predictions about human behavior. 11/12/2018 *

Why use cognitive models? * 07/16/96 Why use cognitive models? “It is useful to analyze the knowledge of how to do a task in terms of goals, operators, methods and selection rules.” (John & Kieras, 1996) Validated engineering models of human performance GOMS assumes that routine cognitive skills can be described as a serial (or parallel in the case of CPM-GOMS) sequence of cognitive operators and motor activities. The ability to predict how users will interact with proposed designs is a useful tool for the system designer. It is also assumed that “It is useful to analyze the knowledge of how to do a task in terms of goals, operators, methods and selection rules”. And by doing so we can predict performance based on a validated model of human performance. We can also use the model around which to base other discussion about the nature of the task and users goals (and mental models). 11/12/2018 *

Why use cognitive models? (continued) * 07/16/96 Why use cognitive models? (continued) Provide a functional model of the task Specific information includes: Tricks of the trade Perceptual skills Big picture Cognitive models can provide an analysis of the functions required for a task. This is more than the steps required to complete the task but an analysis of the underlying functionality in the system. It can provide answers to the following questions: What is it that really needs to be done? What are merely surface procedures which can and perhaps should be changed? Cognitive models can include the rules for decided how to accomplish a task based on context. That is they can capture the strategies that experts use. These strategies and skills can be used in training new users and in the redesign of specific parts of the system. 11/12/2018 *

Where have cognitive models been used? * 07/16/96 Where have cognitive models been used? Production systems, GLEAN (John & Kieras, 1996), JIAA (Crow, 1998) Text editing (Card et al., 1983; John, 1996) Phone operators (Atwood, Gray, & John, 1995) Video games (John, Vera, & Newell, 1993) GOMS and other cognitive models have been used for a large number of domains and an even larger number of tasks. The applications range from telephone operators to Super Mario Brothers, from map digitization to cockpit design analysis. 11/12/2018 *

How have cognitive models helped? * 07/16/96 How have cognitive models helped? Identification of potential errors Purchase of new workstations Prediction of learning time Lerch used GOMS models of spreadsheet data entry to analyze the contenets of working memory to identify potential places and sources of errors. By tracking the contents of working memory during a task Lerch compared the Lotus 1-2-3 spreadsheet to the Interactive Financial Planning System. The GOMS models showed the Working Memory load for the two different interfaces. Lerch was able to predict where errors were most likely to occur when working memory items in the model were greater than 8 items in working memory. GOMS models can be very useful in the analysis of large repetitive tasks. Project Ernestine that was completed on a telephone operators workstation for NYNEX by Atwood, Gray and John. Help in the decision of the purchase of new workstations that would have cost the company over 8 million dollars per year. Kieras and Polson provide and extension to the GOMS framework that focused on both the timje to learn new procedures and the transfer of training between various procedures. This extension called Cognitive Complexity Theory provides the basis for making quajtitative predictions about the time to learn each new piece of a task and the amount of transfer. To determine the learning time first the steps needed to be encoded in NGOMSL. They found that each step took about 30 seconds to learn. 11/12/2018 *

Where does GOMS fit? Product Design is to develop a product that has: * 07/16/96 Where does GOMS fit? Product Design is to develop a product that has: Utility Usability Acceptance Salability Functional Level, Documentation, Keystroke Analyses Well so what? GOMS does all of these great things (predicting time to task completion, working memory errors, and learning time) but where does it fit into the consumer products development cycle. 11/12/2018 *

What GOMS is not! Usability Testing Interviewing technique * 07/16/96 What GOMS is not! Usability Testing Interviewing technique Naturalistic observation Heuristic evaluation and other checklist Cognitive walkthroughs A design technique 11/12/2018 *

Extensions New Input Devices Effects of fatigue and stress * 07/16/96 Extensions New Input Devices Effects of fatigue and stress Individual differences Collaborative cognitive models 11/12/2018 *