1. Usability and Human Factors
This is the second part of the introductory lecture on usability and human factors.
People and Technology, Studies of Technology
Unit 1b
This material (Comp15_Unit1b) was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC

2. Lecture Objectives Patient safety issues
Demonstrate concept knowledge of principles of user-centered design, methods of cognitive research, and sources of usability evidence
Describe the seven stages of User Activity in Norman’s Theory of Action
Describe the role of human factors and human computer interaction concerning patient safety in the healthcare setting
Demonstrate concept knowledge of principles of user-centered design and sources of usability evidence
To continue the discussion from the previous lecture in this section we will focus on the following:
Describe patient safety issues, particularly in relation to health technologies
Demonstrate concept knowledge of principles of user-centered design, methods of cognitive research, and sources of usability evidence.
Describe the seven stages of User Activity in Norman’s Theory of Action
Describe the role of human factors and human computer interaction concerning patient safety in the healthcare setting.
Demonstrate concept knowledge of principles of user-centered design and sources of usability evidence
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

3. Freedom from accidental injury
Patient Safety
Freedom from accidental injury
Institute of Medicine Report:
98,000 preventable deaths due to human error
Eighth leading cause of death
“Given the complexity of health care and the formidable obstacles it presents to change, to overcome these barriers and create a safe culture does indeed seem to be the ultimate challenge for those who specialize in human factors” (Lucian Leape, 1994)
The subject of patient safety has been an issue of tremendous concern dating back at least a decade. There was a report by the Institute of Medicine that really rocked the field and the results that emanated from that were quite shocking. 98,000 preventable deaths every single year are due to human error. In fact, human error is the 8th leading cause of death. Now, we have to recognize that error is inevitable in any discipline. However, in medicine error is especially costly. Lucian Leape is a thought leader in this community. Leape said, “Given the complexity of healthcare and the formidable obstacle it presents to change, to overcome these barriers and create a safe culture seems to be the ultimate challenge for those who specialize in human factors.” There are many complexities involved in creating a safe culture. Part of it is realizing that the things we do, the way that we collect data, the way that we scrutinize the way people work and use technology is an essential ingredient of creating that same culture. But that by itself is not sufficient. For example you would need to have buy-in from authoritative figures in order to make the changes needed to create a safe culture. If errors are not reported, which is too often the case, then it is much harder to learn from them.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

4. Annual Death Rates in US
This chart serves to highlight the annual death rates in the U.S. attributable to medical error. As mentioned previously, a total of 98,000 deaths occur due to medical error. This exceeds those resulting from motor vehicle accidents, breast cancer, AIDS, and so forth, making this quite an extraordinary figure.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

5. Medical Device Errors
FDA recognizes most user errors with medical devices are:
Influenced by device design and device labeling
Not "inevitable human error"
The Food and Drug Administration is the government body that oversees and approves medical devices. This body, the FDA, recognizes that most user errors with medical devices are influenced by device design and device labeling. They're not inevitable human error. This may seem obvious but it's an important point of recognition. Usability is viewed as increasingly important by the FDA, but in many respects their standards are not quite up to what is necessary to achieve a reliable product in terms of usability. They're much more focused on, for example, the accuracy and safety of the device. These issues are important, but certainly one would not want to ignore issues of usability.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

6. FDA Device Reports
There's an extraordinary range of devices and many of them present formidable challenges to users. In a given year there are over 25,000 FDA device reports related to user errors and many of them have been implicated in patient/user deaths.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

7. Glucose Monitors
Glucose meters are a widely used device by the growing number of patients with diabetes. They can be used reliably and effectively. But studies of various glucose monitors, especially those with more advanced functions, have significant documented usability problems. So they are an example of a device that needs to be scrutinized more carefully. This is especially true, given the context that glucose devices are used by the growing number of elderly patients who constitute a relatively sizable portion of patients with diabetes.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

8. Digital Divide
Socioeconomic and demographic divisions between computer users and nonusers
Elderly adults over age 65
Earn less than $25,000
Hispanics and African Americans
Rural populations
Education less than high school
A Nation Online (2002)
Glucose meters are an example of a patient-centered technology. As technologies become increasingly commonplace for patients and health consumers it's really important to understand that we don't yet have a level playing field. The digital divide refers to a phenomenon involving socioeconomic and demographic divisions between computer users and non-users. In particular, older adults over the age of 65, individuals who earn less than 25,000 a year, minorities including Hispanics and African-Americans, individuals living in rural population, and those who have not completed high school are more likely to be a part of digital divide. The good news is that this divide has been closing over the course of the last 5 to 10 years, but it still represents a substantial division. One of the ironies of consumer health is that we're trying to develop tools for patients, particularly those most in need. However, if only the more affluent and the more able can use these technologies productively, then potentially we're increasing health disparities rather than reducing them. Clearly the intent is to reduce them but the danger is that we could be exacerbating an existing problem if health technologies are not usable for those most in need. Closing or reducing the digital divide is a very important societal goal.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

9. Community-Related Initiatives
The good news is that there are an increasing number of community-related initiatives to bridge the digital divide. This picture illustrates a community trainer training members of the community about accessing health information online. This program is for Head Start parents and they are gaining exposure to a website known as KidsHealth. Another important area of focus reflects efforts to reach out to seniors who by and large have less access to computers, and experience considerable difficulties using them.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

10. Community-Related Initiatives (cont.)
This slide illustrates older adults learning to access health information online. These community classes have the potential to close some of the digital divide gaps. The goal is to give these individuals the potential to develop their skills in a familiar context and at an appropriate learning pace.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

11. Telemedicine Device
There's also a growth of new and different kinds of technology. This is a particular telemedicine device designed for patients with diabetes. It could be used to communicate with a nurse or a physician, typically a nurse at particular periods of time. This particular device was especially tailored to older adults.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

12. Touch Screen Device
The telemedicine device shown in the last slide improved over several iterations and we went from a conventional mouse and Windows based system to one that used touch screens, in an effort to increase its usefulness and usability to large segments of the population who have difficulty using a mouse. In general, the mouse is not a device that's well suited to older adults, particularly those who have never used computers before. It presents significant challenges.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

13. Design A plan/scheme conceived in mind & intended for execution
Tradeoffs balancing conflicting requirements
Generating alternatives
Use representations
Natural language, diagrams, prototypes
Interaction design
Developing plan informed by product’s intended use, target domain & relevant practical considerations
Of course all systems are conceived through a design process. Design is a plan or scheme conceived in mind and intended for execution resulting in a particular product. Invariably, it involves a set of tradeoffs, balancing conflicting requirements. For example, a given system might want to target high performing experts in a particular domain or alternatively might want to support relatively novice users. Naturally, there is going to be a certain degree of conflict because it's very hard to do both well. The design process involves generating alternatives and using various kinds of representations that lead to the creation of design concepts. They're expressed in different ways, such as in natural language, mockups in the form of diagrams, and eventually instantiated in the form of prototypes. Interaction design is a subfield devoted to studying particular facets of design in relation to user experience. It involves developing a plan and form by the product's intended use based on an understanding of a target domain. Healthcare is a rather unique kind of target domain, which comes with a range of practical considerations. And as important as it is to get expert input, one also needs to solicit input from users and from the communities that will be using the technology.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

14. Iterative Design Process
Needs analysis
Develop alternatives
Prototype
Implement
Evaluate
Increasingly we think of design as not merely a linear process leading to a particular product, but rather an iterative process that might start with a needs analysis of the particular population. This involves developing various kinds of alternatives that might lead to a succession of prototypes, further leading to a beta version, and ultimately to a mature system that can be implemented. The implementation process itself in the context of healthcare is a particularly challenging one. Evaluation can play a role at different points of time either prior to or after the implementation, which should contribute to an iterative process leading to an improvement in design and a more usable product. Needs requirement and requirement engineering is discussed in the next set of lectures. Other facets of the design process will be dealt with later on in the course.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

15. User-Centered Design
More and more, designers recognize that different populations have different kinds of needs and that it's incumbent on them to solicit views. This is an interesting case in part of user-centered design. This is a product developed by Cornelia Ruland and colleagues in Norway. Her goal was to develop a system for children with cancer to better express themselves about their illness and to make help make choices that could potentially affect their treatment. Now, of course, you can't present medical facts to young children. It's rather difficult to even present text to children.
She created a really remarkable system and it involved a user-centered design process that engaged the participation of various children in creating a prototype, creating mockups, creating design concepts. Now, anytime you're engaged in a user-centered design process, you must realize that it will not in itself constitute the product 'cause of course lay people are not designers and don't fully understand design. However, they seed valuable concepts and it's an extremely useful and important part of the process.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

16. Norman's Seven Stages of User Activity
Now we are going to introduce a little bit of theory in view to understand the nature of interaction. The diagram on this slide depicts Norman’s seven stages of user activity based on his theory of action. It is a very influential theory and it is deceptively simple in character. We can think of it as characterizing any interaction with any system from opening a water faucet to executing a complex command sequence to query a database. It all begins with a particular goal (for example, finding information on strep throat), specifying an action (such as needing to do a Google search) and executing an action, thereby coming into contact with the physical system (typing strep throat into the search box). The system then responds (and returns nearly 2 million results). One then needs to interpret the output. Does it meet with my expectations? Should I do a different search? Should I start over again?
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

17. Bridging Gulfs Gulf of Execution Gulf of Evaluation
Gap between user’s goals & state of system
How do I… ?
Gulf of Evaluation
Gap between observed & expected system state
What happened?
There are two classes of problems that arise from any interaction with a system. The first class of problems, the gulf of execution, refers to any problem associated with performing the action. For example, you may click on the search button thinking that it will take you to a specific page. The second class of problems is known as the gulf of evaluation. Once you’ve executed your search, you may have difficulties interpreting the results. For example, you may not know to differentiate the sponsored links from the other results on the page. It's a property of many things, of course the experience of the user, but most importantly the kind of feedback that our system provides. Systems and applications are variably informative, and this is a very important part of the usability evaluation process is determining whether system feedback is adequate to guide these.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

18. Sources of Usability Evidence
Basic Observation
Expert review/usability inspection
Testing situation
End-user observation
Video capture & review
Qualification & quantification
Formal coding
There are many different ways to evaluate the usability of a system and that will be the subject of a forthcoming lecture. You can even learn something about usability by watching a friend or colleague using a system. Often that will reveal things that are not evident when you use a system yourself. One may also choose to use an expert review method such as heuristic evaluation or the cognitive walkthrough. These are performed by trained analysts and rely on their expert judgment as a source of usability evaluation. Usability testing involving representative users reflects the gold standard for usability assessment, although it is also the most expensive and time consuming. Most usability testing today involves some sort of video capture method and may or may not involve formal coding of the video. We will be discussing these methods in much greater detail in the forthcoming lectures.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

19. Summary
Patient safety has been an issue of tremendous concern over the last 10 or so years.
Medical devices are potential sources of error that can compromise patient safety
Digital divide explains the unequal access to computer resources and their consequence
The class introduces the concept of patient safety which has been an issue of concern over the last 10 or so years. Medical devices are potential sources of error that can compromise patient safety. Another societal problem pertaining to health information technology is the digital divide which explains the unequal access to computer resources, the internet and their consequences
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

20. Summary (cont.)
Iterative design is integral to developing sounds design practices that can improve patient safety
User-centered design is an exemplary method for engaging users into the design process.
Norman’s seven stages of user interactivity
Sources of usability evidence
In the next class, we will discuss various examples of good and poor design
The class introduces the concept of patient safety which has been an issue of concern over the last 10 or so years. Medical devices are potential sources of error that can compromise patient safety. Another societal problem pertaining to health information technology is the digital divide which explains the unequal access to computer resources, the internet and their consequences.
Iterative design is integral to developing sounds design practices that can improve patient safety. User-centered design is an exemplary method for engaging users into the design process. The class used the SISOM project developed by Cornelia Ruland and colleagues as an example of user-centered design.
Norman’s seven stages of user interactivity has been a highly influential model that has informed work in usability analysis. The class also introduced an important distinction to explain user problems, namely the gulf of execution and the gulf of evaluation. In the next class, we will discuss various examples of good and poor design
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b

21. References
Kaufman, D.R. & Starren, J. B. (2006). A methodological framework for evaluating mobile health devices. In the Proceedings of the American Medical Informatics Annual Fall Symposium. Philadelphia: Hanley & Belfus. 978
Kaufman, D.R., Pevzner, J, Hilliman, C., Weinstock, R.S., Teresi, J. Shea, S. & Starren, J. (2006). Re-designing a telehealth diabetes management program for a digital divide seniors population. Home, Healthcare, Management & Practice. 18:
Kaufman, D.R. & Rockoff, M.L. (2006). Promoting online health information-seeking in seniors: a community-based organizations approach. Generations 30(2):
Kohn, L. T., Corrigan, J., & Donaldson, M. S. (2000). To err is human: building a safer health system: National Academy Press.
Leape, L. L., Error in Medicine. (1994).  JAMA,  272:
Norman, D. A. (1986). Cognitive engineering. In D. A. Norman & S. W. Draper (Eds.), User centered system design: New perspectives on human-computer interaction (pp ). Hillsdale, NJ: Lawrence Erlbaum Associates.
Ruland, C. M., Starren, J., & Vatne, T. M. (2008). Participatory design with children in the development of a support system for patient-centered care in pediatric oncology. J Biomed Inform, 41(4), doi: S (07) [pii] /j.jbi
U.S. Department of Commerce. (2002). A nation online: How Americans are expanding their use of the Internet. Washington, DC: U.S. Government Printing Office.
No audio.
Health IT Workforce Curriculum Version 3.0/Spring 2012
Usability and Human Factors People, Technology, Studies of Technology Unit 1b