University of South Florida Needs Assessment The secret to successful system is to design something that meets the information needs of the end user. But for many reasons this has not been easy to do. Farrokh Alemi, Ph.D. University of South Florida

University of South Florida Needs Assessment Let us start with some ways that I do not think you should do and that is commonly done. Farrokh Alemi, Ph.D. University of South Florida

University of South Florida Analyze Don’t Do What We Do Analyze organizational tasks and see how information is used in these tasks. Then plan an information system around these needs. This approach often fails because what we do in 6 months or a year from now could be very different. What we will do is not the same as what we are currently doing. University of South Florida

University of South Florida Copy Copy Don’t Do Copy You can also not do any soul searching and just do as other health care providers are doing. Copy others, after all what could be so different in an electronic health record. This approach also is not recommended because it fails to take strategic and competitive advantage from the investment in electronic health records. It relegates the electronic health record to a commodity that cannot be used to distinguish your practice. University of South Florida

University of South Florida Ask Don’t Do Users You can also ask users. But users typically have a limited view, do not know the technology that is emerging, cannot anticipate their own needs, may not be there when the system is ready and ask for far more things than they need. They give long lists of what they want but use very little of the information. University of South Florida

Key Reason for Failure of IT Investment “I can’t get the report I need?” “What is this for?” “This is not what I do.” Why are so many information systems unused? Current approaches of information needs assessment have been failing us. You often hear about information systems that do not meet the needs of the clients. The common complaint seems to be that the right information is presented too late or the data collected is not related to the decision that needs to be made. So the system goes unused. University of South Florida

University of South Florida THIS IS NOT WHAT I NEED Systems that do not meet our needs present the wrong information or present it at the wrong time. Don’t you want to reach out in the past and shake up the person who designed this system and tell him or her how frustrating it is to use it. That person is you. As a manager, you are responsible to make sure that the system you have purchased fits the needs of the organization. University of South Florida

University of South Florida Why is it so hard to design systems that meet our needs? No one sets out to fail. Clearly the person who designed the current system wanted to do something that helps but why did he fail? University of South Florida

University of South Florida Perhaps it was not for planned to be used by you. Systems are organized for organizations and not individuals. Individuals change. Systems have to continue to work. Not for you. University of South Florida

University of South Florida Systems are typically designed many years ago, when technology and organizations mission were different. Not for now. University of South Florida

University of South Florida Data collection and analysis is not cheap. The data that is collected quickly becomes what the organization excels in. The data not collected is often ignored. The cost of collecting all necessary data is prohibitive not only in the data collection time but also in what happens to the organization when it has no focus. Choices have to be made to reduce cost and to help organization focus. Not free. University of South Florida

University of South Florida Individual needs change over time. What the physician was planning to use is often not they actually use. People say they want things but fail to use it. We buy things on a whim and are stuck with them. You changed. University of South Florida

University of South Florida The solution is to move away from what people say they want today and focus on what they will need tomorrow. Here is a way to accomplish this. University of South Florida

University of South Florida 1 List Future Decisions The first step is to focus on the future and anticipate the decisions that would be made after the electronic health record is implemented. Some of these decisions are repetitive and similar to what the clinic decides about right now. Other decisions are unique and emerging. Whether repetitive or emerging, the focus should be on decisions and not tasks. Tasks are repetitive operations we do, decisions are points in the task where we have alternative choices. This is what you have to focus on. What are the clinical and managerial choices that we will face in a year or so. University of South Florida

University of South Florida 2 Information for Decision For each decision then you list the information that is needed. What will make us choose one alternative over another. Will a particular lab test added to standing orders really make a different in the treatment decision or should this be ordered separately. The key is that we are not saying that the information is wanted but it is needed for the choice among two alternative course of actions. University of South Florida

University of South Florida 3 Set Priority Collate the information needed across various decisions and set priorities for their collection. University of South Florida

University of South Florida 4 Focus on Persistent Needs Focus on information judged to be important in different decisions. University of South Florida

Mental Health Information System As an example let us look at how we designed a system for mental health commissioners. As you know, mental health commissioners need information systems that can help them decide and convince others about their new policies. The Federal government asked us to design a system that could help state commissioners. University of South Florida

University of South Florida Future Decisions Paraprofessional’s reimbursement Role of primary care Quality of care in nursing homes A Delphi process is used to identify future upcoming decisions. This process is a form of crowd sourcing where a large group of experts identify what might occur in one to two years. In this process we identified a large number of decisions including: How to reimburse the growing number of paraprofessionals who treat mental health clients? What educational programs or policies should the state adopt to upgrade knowledge of mental illness among primary care physicians who deliver many services to rural, mentally ill clients? Many chronically mentally ill patients live in nursing homes that cannot provide adequate care. What policies should the state adopt to improve care for these patients? University of South Florida

University of South Florida 2 out of 69 possible needs Here are 2 out of 69 items that were identified. University of South Florida

Step 3: Rate the information in context of specific decisions For the issue "xxx," assign a score reflecting the need for each of the 69 information items. For each item we asked the commissioners and an expert panel to rate the importance of different pieces of information. University of South Florida

Data Collection Plans and Analysis of Ratings We then calculated average and range and based on the values of the average and range assigned the information item to different categories. Note that something that is persistently rated as important will have a high average score and a low range. University of South Florida

Examples of Essential Information How expensive is it to provide services that fully meet clients’ needs? Here is a piece of information judged to be consistently important across various decisions. University of South Florida

University of South Florida Examples of Rapid Data Here are items that were judged to need rapid data collection. University of South Florida

Examples of periodic information Who gets what priority for different types of care How different providers serve the undeserved areas Constraints specifically limiting substitutability of providers The existing provider certification and education systems Here are some items that were judged to need periodic data collection and did not need to be part of the electronic record. University of South Florida

Examples of periodic and ignored information sets Episodes of illness: frequency and responsiveness to treatment The role of the family in caring for client What knowledge is needed to correct the problem How existing laws and regulations duplicate each other Here are examples of information items that the group decided to ignore, at least until more funding was available. University of South Florida

Plan for Future Decisions Take Home Lesson The take home lesson should be simple. For needs assessment, focus on the future decisions and then see what information are needed in these decisions. University of South Florida