1. Process Improvement, System Design, and Usability Evaluation:
Health Informatics in Low- and Middle-Income Countries
Short Course for Health Information System Professionals
Process Improvement, System Design, and Usability Evaluation:
People and Technology, Studies of Technology
Lecture B

2. People and Technology, Studies of Technology
Learning Objectives—Lecture B
Patient safety issues
Demonstrate concept knowledge of principles of user-centered design, methods of cognitive research, and sources of usability evidence
Describe the seven stages of User Activity in Norman’s Theory of Action
Demonstrate concept knowledge of principles of user-centered design and sources of usability evidence
To continue the discussion from the previous lecture in this section we will focus on the following:
Describe patient safety issues, particularly in relation to health technologies
Demonstrate concept knowledge of principles of user-centered design, methods of cognitive research, and sources of usability evidence.
Describe the seven stages of User Activity in Norman’s Theory of Action
Describe the role of human factors and human computer interaction concerning patient safety in the healthcare setting.
Demonstrate concept knowledge of principles of user-centered design and sources of usability evidence

3. Deaths Due to Medical Errors: As if a Jet Crashed Every Day
Blue Moon Voyager, 2010, CC BY-NC-ND 2.0
Medical errors kill more people every year than car wrecks, breast cancer, and AIDS combined. According to the Institute of Medicine, 98,000 people every year die due to medical errors. This will lead us into the rest of the material in this lecture, because it sets a context or a frame for one of the big driving factors in healthcare now, which is quality, safety and medical errors.
Health IT Workforce Curriculum Version 4.0

4. Patient Safety Freedom from accidental injury
Institute of Medicine Report:
98,000 preventable deaths due to human error
Eighth leading cause of death
“Given the complexity of health care and the formidable obstacles it presents to change, to overcome these barriers and create a safe culture does indeed seem to be the ultimate challenge for those who specialize in human factors” (Lucian Leape, 1994).
The subject of patient safety has been an issue of tremendous concern for many years. The report “To Err is Human” by the Institute of Medicine in 2001 really rocked the field, and the results that emanated from that report were quite shocking: 98,000 preventable deaths every single year are due to human error. In fact, human error is the 8th leading cause of death. Now, we have to recognize that error is inevitable in any discipline. However, in medicine error is especially costly. Lucian Leape said, “Given the complexity of healthcare and the formidable obstacle it presents to change, to overcome these barriers and create a safe culture seems to be the ultimate challenge for those who specialize in human factors.” There are many complexities involved in creating a safe culture. Part of it is realizing that the things we do, the way that we collect data, the way that we scrutinize the way people work and use technology is an essential ingredient of creating that same culture. But that by itself is not sufficient. For example, you would need to have buy-in from authoritative figures in order to make the changes needed to create a safe culture. If errors are not reported, which is too often the case, then it is much harder to learn from those errors.
Health IT Workforce Curriculum Version 4.0

5. Annual Death Rates in the United States
This chart shows the annual death rates in the United States attributable to medical error. As mentioned previously, almost 100,000 people die every year due to medical error. This exceeds those resulting from motor vehicle accidents, breast cancer, AIDS, and so forth, making this quite an extraordinary figure.
(Kohn, Corrigan, & Donaldson, 2000)

6. Medical Device Errors
The U.S. Food and Drug Administration recognizes that most user errors with medical devices are:
Influenced by device design and device labeling
Not “inevitable human error”
The Food and Drug Administration is the government body that oversees and approves medical devices. This body, the FDA, recognizes that most user errors with medical devices are influenced by device design and device labeling. They're not inevitable human error. This may seem obvious but it's an important point. Usability is viewed as increasingly important by the FDA, but in many respects their standards are not quite up to what is necessary to achieve a reliable product in terms of usability. They're much more focused on, for example, the accuracy and safety of the device. These issues are important, but certainly one would not want to ignore issues of usability.

7. Digital Divide
Socioeconomic and demographic divisions between computer users and nonusers
Elderly adults
Less affluent
Rural populations
Less educated
Gender
(Council of Economic Advisors, 2015)
Glucose meters are an example of a patient-centered technology. As technologies become increasingly commonplace for patients and health consumers it's really important to understand that we don't yet have a level playing field. The digital divide refers to a phenomenon involving socioeconomic and demographic divisions between computer users and non-users. In particular, older adults, lower income individuals, minorities including Hispanics and African-Americans, individuals living in rural areas, and those who have not completed high school are more likely to be a part of the digital divide.
The good news is that this divide has been closing over the last several years, but it still represents a substantial division. One of the ironies of consumer health is that we're trying to develop tools for patients, particularly those most in need. However, if only the more affluent and the more able can use these technologies productively, then potentially we're increasing health disparities rather than reducing them. Clearly the intent is to reduce them, but the danger is that we could be exacerbating an existing problem if health technologies are not usable for those most in need. Closing or reducing the digital divide is a very important social goal.

8. Touch Screen Device (Kaufman, Pevzner, Hilliman, et al., 2006)
The telemedicine device shown in the last slide improved over several iterations and it went from a conventional mouse and Windows based system to one that used touch screens, in an effort to increase its usefulness and usability to large segments of the population who have difficulty using a mouse. In general, the mouse is not a device that's well suited to older adults, particularly those who have never used computers before. It presents significant challenges.
(Kaufman, Pevzner, Hilliman, et al., 2006)

9. Design A plan or scheme conceived in mind and intended for execution
Tradeoffs balancing conflicting requirements
Generating alternatives
Use representations
Natural language, diagrams, prototypes
Interaction design
Developing plan informed by product’s intended use, target domain, and relevant practical considerations
Of course all systems are conceived through a design process. Design is a plan or scheme conceived in mind and intended for execution resulting in a particular product. Invariably, it involves a set of tradeoffs, balancing conflicting requirements. For example, a given system might want to target high performing experts in a particular domain or alternatively might want to support relatively novice users. Naturally, there is going to be a certain degree of conflict because it's very hard to do both well. The design process involves generating alternatives and using various kinds of representations that lead to the creation of design concepts. They're expressed in different ways, such as in natural language, mockups in the form of diagrams, and eventually instantiated in the form of prototypes. Interaction design is a subfield devoted to studying particular facets of design in relation to user experience. It involves developing a plan and form by the product's intended use based on an understanding of a target domain. Healthcare is a rather unique kind of target domain, which comes with a range of practical considerations. And as important as it is to get expert input, one also needs to solicit input from users and from the communities that will be using the technology.
Health IT Workforce Curriculum Version 4.0

10. Iterative Design Process
Increasingly we think of design as not merely a linear process leading to a particular product, but rather an iterative process that might start with a needs analysis of the particular population. This involves developing various kinds of alternatives that might lead to a succession of prototypes, further leading to a beta version, and ultimately to a mature system that can be implemented. The implementation process itself in the context of healthcare is a particularly challenging one. Evaluation can play a role at different points of time either prior to or after the implementation, which should contribute to an iterative process leading to an improvement in design and a more usable product. Needs requirement and requirement engineering is discussed in the next set of lectures. Other facets of the design process will be dealt with later on in the course.
Health IT Workforce Curriculum Version 4.0

11. User-Centered Design (Ruland, Starren, & Vatne, 2008)
More and more, designers recognize that different populations have different kinds of needs and that it is incumbent on them to solicit views. This is an interesting case in part of user-centered design. This is a product developed by Cornelia Ruland and colleagues in Norway. Her goal was to develop a system for children with cancer to better express themselves about their illness and to make help make choices that could potentially affect their treatment. Now, of course, you can't present medical facts to young children. It's rather difficult to even present text to children.
She created a really remarkable system and it involved a user-centered design process that engaged the participation of various children in creating a prototype, creating mockups, creating design concepts. Now, anytime you're engaged in a user-centered design process, you must realize that it will not in itself constitute the product because of course lay people are not designers and don't fully understand design. However, they provide valuable concepts and this is an extremely useful and important part of the process.
(Ruland, Starren, & Vatne, 2008)

12. Norman’s Seven Stages of User Activity
Now we are going to introduce a little bit of theory in view to understand the nature of interaction. The diagram on this slide depicts Norman’s seven stages of user activity based on his theory of action. It is a very influential theory and it is deceptively simple in character. We can think of it as characterizing any interaction with any system from opening a water faucet to executing a complex command sequence to querying a database. It all begins with a particular goal (for example, finding information on strep throat), specifying an action (such as needing to do a Google search) and executing an action, thereby coming into contact with the physical system (typing strep throat into the search box). The system then responds (and returns nearly 2 million results). One then needs to interpret the output. Does it meet with my expectations? Should I do a different search? Should I start over again?
(Norman, 1986)

13. Bridging Gulfs Gulf of execution
Gap between user’s goals and state of system
How do I… ?
Gulf of evaluation
Gap between observed and expected system state
What happened?
There are two classes of problems that arise from any interaction with a system. The first class of problems, the Gulf of Execution, refers to any problem associated with performing the action. For example, you may click on the search button thinking that it will take you to a specific page. The second class of problems is known as the Gulf of Evaluation. Once you’ve executed your search, you may have difficulties interpreting the results. For example, you may not know to differentiate the sponsored links from the other results on the page. It's a property of many things, of course the experience of the user, but most importantly the kind of feedback that our system provides. Systems and applications are variably informative, and this is a very important part of the usability evaluation process is determining whether system feedback is adequate to guide these.
Health IT Workforce Curriculum Version 4.0

14. Sources of Usability Evidence
Basic observation
Expert review and usability inspection
End-user observation
Testing situation
Video capture and review
Formal coding
Qualification and quantification
There are many different ways to evaluate the usability of a system and that will be the subject of a forthcoming lecture. You can even learn something about usability by watching a friend or colleague using a system. Often that will reveal things that are not evident when you use a system yourself. One may also choose to use an expert review method such as heuristic evaluation or the cognitive walkthrough. These are performed by trained analysts and rely on their expert judgment as a source of usability evaluation. Usability testing involving representative users reflects the gold standard for usability assessment, although it is also the most expensive and time consuming. Most usability testing today involves some sort of video capture method and may or may not involve formal coding of the video. We will be discussing these methods in much greater detail in the forthcoming lectures.
Health IT Workforce Curriculum Version 4.0

15. People and Technology, Studies of Technology
Summary—Lecture B
Patient safety is an issue of tremendous concern.
Medical devices are potential sources of error that can compromise patient safety.
The digital divide explains the unequal access to computer resources and their consequence.
The class introduces the concept of patient safety which is an issue of tremendous concern. Medical devices are potential sources of error that can compromise patient safety. Another societal problem pertaining to health information technology is the digital divide which explains the unequal access to computer resources, the internet and their consequences.

16. People and Technology, Studies of Technology
Summary—Lecture B (cont.)
Iterative design is integral to developing sounds design practices that can improve patient safety
User-centered design is an exemplary method for engaging users in the design process
Norman’s seven stages of user interactivity
Sources of usability evidence
This concludes Lecture B of Usability and Human Factors, People and Technology, Studies of Technology.
Iterative design is integral to developing sounds design practices that can improve patient safety. User-centered design is an exemplary method for engaging users into the design process. The class used the SISOM project developed by Cornelia Ruland and colleagues as an example of user-centered design.
Norman’s seven stages of user interactivity has been a highly influential model that has informed work in usability analysis. The class also introduced an important distinction to explain user problems, namely the gulf of execution and the gulf of evaluation. In the next class, we will discuss various examples of good and poor design.

17. People and Technology, Studies of Technology
References—Lecture B
References
Kaufman, D. R., & Starren, J. B. (2006). A methodological framework for evaluating mobile health devices. In The Proceedings of the American Medical Informatics Annual Fall Symposium (p. 978). Philadelphia, PA: Hanley & Belfus.
Kaufman, D. R., Pevzner, J., Hilliman, C., Weinstock, R. S., Teresi, J. Shea, S., & Starren, J. (2006). Re- designing a telehealth diabetes management program for a digital divide seniors population. Home, Healthcare, Management & Practice, 18,
Kaufman, D. R., & Rockoff, M.L. (2006). Promoting online health information-seeking in seniors: A community-based organizations approach. Generations, 30(2):
Kohn, L. T., Corrigan, J., & Donaldson, M. S. (2000). To err is human: Building a safer health system. Washington, DC: National Academy Press.
Leape, L. L. (1994). Error in medicine. JAMA, 272,
Council of Economic Advisors. (2015). Mapping the digital divide. Issue brief.
Norman, D. A. (1986). Cognitive engineering. In Norman, D. A., & Draper, S. W. (Eds.), User centered system design: New perspectives on human-computer interaction (pp ). Hillsdale, NJ: Lawrence Erlbaum Associates.
Ruland, C. M., Starren, J., & Vatne, T. M. (2008). Participatory design with children in the development of a support system for patient-centered care in pediatric oncology. J Biomed Inform, 41(4),
U.S. Department of Commerce. (2002). A nation online: How Americans are expanding their use of the Internet. Washington, DC: U.S. Government Printing Office.
No Audio.

18. People and Technology, Studies of Technology
References—Lecture B
Chart
Kohn, L. T., Corrigan, J., & Donaldson, M. S. (2000). To err is human: Building a safer health system. Washington, DC: National Academy Press.
Images
QANTAS Jet [online image]. (n.d.). Retrieved June 21, 2016, from
Kaufman, D. R., Pevzner, J., Hilliman, C., Weinstock, R. S., Teresi, J., Shea, S., & Starren, J. (2006). Re- designing a telehealth diabetes management program for a digital divide seniors population. Home, Healthcare, Management & Practice, 18,
No Audio.

19. People and Technology, Studies of Technology
References—Lecture B
Ruland, C. M., Starren, J., & Vatne, T. M. (2008). Participatory design with children in the development of a support system for patient-centered care in pediatric oncology. J Biomed Inform, 41(4),
Norman, D. A. (1986). Cognitive engineering. In Norman, D.A., & Draper, S.W. (Eds.), User centered system design: New perspectives on human-computer interaction (pp ). Hillsdale, NJ: Lawrence Erlbaum Associates.
No Audio.

20. This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC This material was updated by The University of Texas Health Center at Houston under Award Number 90WT0006.
This presentation was produced with the support of the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. Views expressed are not necessarily those of USAID or the United States government.