Context of Decision Making Session 3 In this session, we are going to discuss the importance of using data to inform decisions and basic concepts, tools, and approaches to facilitate the use of data and information.

Session Objectives Explain the context of decision making Define the concept of stakeholders Explain the importance of involving stakeholders throughout the data use in decision-making cycle Introduce the Stakeholder Analysis and Engagement tools By the end of this session, the learner will be able to: NOTE to facilitator: Read slide.

Session Overview Context of decision making Stakeholders Decisions Data Small Group Activity 3: Stakeholder Analysis and Engagement Specifically, we will discuss the context of decision making, which includes stakeholders, the decisions they make, and the data they rely on. We also will discuss how to assess the many stakeholders that affect decision making and determine how best to engage them.

Group Participation How can we ensure that information is being used to make diagnoses and inform decisions? Now that we have discussed: What it means to use data and information to inform decision making and know ... That we need to identify and overcome potential barriers to using data and information ... Let’s consider the context of decision making.

Context of Decision Making Data Stake-holders Decisions On this slide, you see a graphic depicting the context of decision making. To make a decision, three elements are critical: Data Decisions The involvement of stakeholders The point of this graphic is to show that ALL THREE elements are equally important. Without all of these components, you will either fail to make a decision or fail to make an evidence-based decision.

Context of Decision Making Data Stake-holders Decisions Let’s first discuss stakeholders’ roles in the context of decision making.

What Is a Stakeholder? Any person, group, or organization with a particular interest in a policy or program Government agencies Beneficiaries Policymakers Funding agencies Providers / implementers Civil society Researchers M&E specialists What is a stakeholder? A stakeholder is anyone who has a “stake” or interest in your program. We often think of government agencies, policymakers, funding agencies, and even implementers or providers as stakeholders. However, we often do not think of the beneficiaries or health programs as stakeholders. The people that our programs and services strive to serve make decisions about seeking services and continuing to seek care. It is vital to consider these stakeholders when designing and implementing any program or service. As we begin to talk about stakeholders, it is important to note that they often fall into two groups: data users and data producers. We typically think only of data users as our stakeholders but, as you can see here, data producers play a key role in the data-informed decision-making process.

Stakeholders Nongovernmental organizations Professional associations Religious leaders Journalists/media Private sector/business The list of stakeholders can be lengthy. Beyond those that we have just mentioned, what other groups may be considered stakeholders? NOTE to facilitator: Ask the group to contribute their ideas. After two to three individuals have spoken, click to reveal some possible answers. Read slide.

Data Producers vs. Data Users Data producers think that decision makers: Value “political” considerations over evidence Are unprepared to measure or evaluate the consequences of their decisions Decision makers or data users think that health researchers and M&E specialists: Lack responsiveness to priorities Favor numbers / jargon over transparent communication Prefer written reports to face-to-face conversation First, let’s consider some fundamental differences between data producers (researchers / M&E specialists) and data users (decision makers). Generally speaking, the former are educated to be objective, analytical, and detail oriented. Decision makers need to be responsive, action oriented, decisive. These frequently opposing approaches contribute to the breakdown in the decision-making cycle. For example: NOTE to facilitator: Read slide.

Importance of Knowing Your Stakeholders View activities from different perspectives Have different degrees of understanding Need / want different information Need information at different levels of complexity Have different intensities of interest Have different roles in the decision-making process It is important to recognize that different stakeholders will affect the data-informed decision-making process in different ways. Different stakeholders …. NOTE to facilitator: Read slide. Remind participants to think about the different needs of data users and data producers.

Results of Involving Stakeholders in Data Use Process Relevance of data Ownership of data Appropriate dissemination of data Use of data By ensuring stakeholder involvement in the data use process, you can tailor data collection and use efforts to the specific needs of the stakeholders, thus increasing the relevance of the data-use activity to local needs. Ownership of data is built so that when data-informed decisions are made, the necessary buy-in exists to move the decision forward. Stakeholder involvement strengthens the information cycle and highlights the value of data to program improvement. When stakeholders understand the data they are using, it increases the relevance and ownership of the data, as well as the appropriate dissemination, which in turn increases the use of data.

Stakeholder Analysis Matrix & Engagement Plan Clarify who has interest in a program and what that interest is Identify who can help a program and how, and who can hurt it Help you use this information for the success of the planning effort Now that we have discussed the importance of involving stakeholders, let’s discuss how to ensure appropriate stakeholder involvement in the data-use activity. MEASURE Evaluation has developed a tool, which is similar in nature to tools developed by other organizations – the Stakeholder Analysis and Engagement tool. It consists of a Stakeholder Analysis Matrix and a Stakeholder Engagement Plan. This tool helps us to systematically and formally assess all of our stakeholders. It clarifies who has an interest in the activity; what that interest is; who can help the activity, and how; who can hurt it; and how this information can be leveraged to ensure success. 14 14 8 12 9

Stakeholder Analysis Matrix The Stakeholder Analysis Matrix is a framework and process for: Identifying stakeholders Defining their roles and resources Identifying dynamics among stakeholders Setting the optimum stakeholder group Let’s first look at the Stakeholder Analysis Matrix. This tool helps to identify individuals and groups that are stakeholders in an M&E or data-use activity, either as contributors, influencers, or beneficiaries. The tool provides a structured way to define the roles that stakeholders play in the activity and assess the resources they could bring to bear. It also provides a framework for assessing the interests, knowledge, positions, alliances, resources, power, and importance of various stakeholders. Who will resist the initiative? Who will support it? What are their reasons? The tool helps to assess which stakeholders to include in the process by determining the relative priority of stakeholders. Which stakeholders have the highest priority?

Stakeholder Analysis Matrix Name of Stakeholder organization, group or individual Stakeholder description Primary purpose, affiliation, funding Potential role in the issue or activity Level of knowledge of the issue Level of commitment Support or oppose the activity, to what extent, and why? Available resources Staff, money, technology, information, influence NOTE TO FACILITATOR: click on each column to show the red circle highlighting each column. Here is an example of the Matrix. At the end of this session, your groups will have the opportunity to use the tool to select a set of stakeholders specific to a professional decision you need to make. For now, let’s look at the information required in each column. In the first column, you list your stakeholder and whether this stakeholder is a person, group, or organization. In the second column, you describe the stakeholder, including job title, organizational purpose, funding sources, etc. In the third column, you include a brief explanation of why this stakeholder is relevant to your activity. In the fourth column, you list the stakeholder’s level of knowledge about your issue. This is important because sometimes you will choose a stakeholder for his/her knowledge level, and sometimes you will choose one in spite of his/her knowledge level because of other resources he/she can bring to the activity. In the fifth column, you list each stakeholder’s level of commitment to the activity. Finally, in the last column, you list the resources that each stakeholder brings to the activity. We will show you a completed example of the Stakeholder Analysis Matrix later in this session.

Stakeholder Analysis Matrix Program issue Develop plan (inc. M&E plan) to scale up PMTCT programs throughout system. Proposed activity Convene stakeholders to identify priorities based on available data and develop action plan. Date November 2006 Name of stakeholder organization, group, or individual Stakeholder description Primary purpose, affiliation, funding Potential role in the issue or activity Level of knowledge of the issue Level of commitment Support or oppose the activity, to what extent, and why? Available resources Staff, money, technology, information, influence National AIDS Control Committee (NACC) Involved in planning, implementation, M&E of all HIV/AIDS programs in the country; approves donor and NGO-funded HIV/AIDS programs Facilitates the stakeholder meeting, prepares for meeting by identifying data sources and preparing an agenda High – receives reports on PMTCT activities from MCH division at MOH; Medium level of knowledge of int’l guidelines and studies Strongly supports activity but hesitant to use international data sources. NACC opposes use of the DHS and most recent international estimates, as it considers these sources to overestimate HIV prevalence Staff available to facilitate; room and computers available for meetings at NACC headquarters Here you see an example of a Stakeholder Analysis Matrix. As you can see, this matrix was created during the development of a plan to scale up PMTCT programs throughout the system. Let’s walk through each column. NOTE to facilitator: Read each column from the example and ask the participants if they have any clarifying questions.

How to Involve Stakeholders Quarterly program management meetings Quarterly meetings to interpret RHIS data Involvement of facility staff to interpret program data M&E system improvement Indicator planning and/or harmonization Data quality review meetings Now that we’ve talked about who your stakeholders are, it is important to think about how to engage them in your activity. Remember to plan to engage stakeholders throughout the activity, not just at the beginning or end. On the program side at either the national or subnational levels, one can engage users and producers in many ways. Examples include opportunities at quarterly meetings, either for interpretation of program or RHIS data. In M&E system improvement, the involvement is usually at the national level but still involves both users and producers. Often opportunities center around national indicators or data systems. Here we’ve listed a few ideas, but can you think of others? NOTE to facilitator: Ask participants to brainstorm other ways of engaging stakeholders. For those interested in research, here are some other examples: Dissemination meetings, design, implementation, application – involve beyond dissemination of results – formulate questions, use data/research, disseminate data for program improvement.

Stakeholder Engagement Plan Program issue Proposed activity Date Stakeholder organization, group, or individual Potential role in the activity Engagement strategy How will you engage this stakeholder in the activity? Follow-up strategy Plans for feedback or continued involvement Once you analyze each of the stakeholders, it is helpful to create a Stakeholder Engagement Plan to ensure that stakeholders are involved throughout the activity. The first column lists the stakeholder, while the second lists the potential role of that stakeholder. The third column shows how you plan to involve the stakeholder, and the final column lists who is responsible for ensuring involvement. NOTE to facilitator: Cick on each column to show the red circle highlighting each column.

Stakeholder Engagement Plan Program issue Develop plan (inc. M&E plan) to scale up PMTCT programs throughout system. Proposed activity Convene stakeholders to identify priorities based on available data and develop action plan. Date November 2006 Stakeholder organization, group, or individual Potential role in the activity Engagement strategy How will you engage this stakeholder in the activity? Follow-up strategy Plans for feedback or continued involvement National AIDS Control Committee (NACC) Facilitates the stakeholder meeting, prepares for meeting by identifying data sources and preparing an agenda that allows for the sources to be discussed The NACC is the lead in this activity. It will be important for the NACC to involve more specifically the PMTCT coordinator, clinical care coordinator, and National AIDS Program Coordinator The NACC is responsible for following up with the prioritized stakeholders Let’s look at an example. You will see that we took the first two columns (the stakeholder and its potential role in the activity) from the Stakeholder Analysis Matrix and inserted that information here. NOTE to facilitator: read through this example.

Context of Decision Making Data Stake-holders Decisions Now let’s discuss decisions and the types that should be data informed.

Decision Areas Program design and evaluation Program management and improvement Strategic planning Advocacy and policy development There are four main types of decision areas in the health sector. They pertain to: Program design and evaluation Program management and improvement Strategic planning Advocacy and policy development NOTE to facilitator: Ask participants to brainstorm some types of decisions that fit into these categories.

Program Design and Evaluation Select messages for prevention campaigns Evaluation Determine if new program approaches are needed to ensure that health impact objectives are met Decisions related to program design and evaluation could include: Selecting key messages for prevention campaigns Identifying and choosing new strategies to increase the impact of specific services NOTE to facilitator: Also recall some of the responses from the plenary brainstorm from the previous slide that are relevant to this category.

Program Management and Improvement Determine if the program is meeting its process objectives Improvement Develop new strategies to increase coverage An example of a decision that is relevant to program management could include: Deciding if a program is meeting its process objectives – for example, training the stated number of providers Deciding what to do to increase the coverage of program services NOTE to facilitator: Also recall some of the responses from the plenary brainstorm from slide 20 that are relevant to this category.

Strategic Planning Identify geographic areas of highest need Determine human resource allocation Determine which of offered services is making the greatest impact Examples of decision making in strategic planning could include: NOTE to facilitator: Read and recall some of the responses from the plenary brainstorm from slide 20 that are relevant to this category.

Advocacy and Policy Development Identifying and quantifying underserved populations Identifying focus areas for new policies Last, decisions around advocacy and policy formulation could include: Selecting priorities for a new strategic plan Identifying focus areas or populations for new policies NOTE to facilitator: Also recall some of the responses from the plenary brainstorm from slide 20 that are relevant to this category.

Context of Decision Making Data Stake-holders Decisions Now let’s discuss the last element in the context of decision making – Data.

Data and Information Census Vital events data Surveillance data Household surveys Facilities-level service statistics Financial and management information Modeling, estimates, and projections Health research As you know, there are many sources of data and information that we can use in decision making, from the national level to the facility level, and even as consumers or beneficiaries of health services. We have listed some of the more common sources on this slide. They include: NOTE to facilitator: Read slide.

Factors Other than Data that Influence Decisions Arbitrariness Political Ideology Data Competing Priorities Public Opinion Decisions Stake-holders Now let’s look back at the entire context of decision making and the three necessary elements that we just discussed – stakeholders, decisions, and data. We all know that these three elements often are influenced by factors other than data. Some of these factors include: NOTE to facilitator: Click to reveal each additional influence on decision making: Political ideology and favoritism – Decision makers may make health decisions based on what their political parties support, rather than data. Public opinion – Decision makers may decide to listen to public opinion rather than data because elections could be coming up. Power relationships – A decision maker may decide on a course of action because a superior told him/her to do it. Competing priorities – Decision makers may decide to follow a course of action based on related or competing priorities, not on data. And last, a decision may be made because the decision maker feels in his or her ‘gut’ that it is the right thing to do. Power Relationships

Strengthening the Decision-making Process Stakeholders Involve new counterparts Decisions Understand service delivery realities In summary, we can strengthen the decision-making process by: Involving new data use counterparts and stakeholders. We need to involve potential users of the data from the outset to ensure that the information we are producing can be used. We also need to understand service delivery realities on the ground so as to understand the decisions being made routinely and how they can be influenced by evidence-based information. By understanding the intended audiences and what is important to them and their programs and ensuring that people are seeing the value in their reporting of data, we also will help to ensure on-time reporting as well as better-quality data. It’s important to note that we may require data beyond traditional baseline and reporting indicators. Often this information is insufficient to make specific program design, planning, management, and operations decisions. Data May require additional data

Key Messages Every decision is made in the context of available data and stakeholders Critically assessing and involving stakeholders will facilitate data-informed decision making Before we move on to the small group activity, let’s reveiw the key messages of this session. They include: NOTE to facilitator: Read slide and then ask if there are any clarifiying questions on the material covered in this session.

Small Group Activity 3: Instructions Break into small groups. Each group should select a decision that they make in their work settings and complete the Stakeholder Analysis Matrix around that decision. A minimum of 7 stakeholders should be identified. Complete the Matrix across the columns for 1–2 stakeholders. Select 1 stakeholder and complete the Stakeholder Engagement Plan for that stakeholder. Time for activity: 45 minutes In this session, we will practice using the Stakeholder Analysis Matrix and the Stakeholder Engagement Plan. Divide into small groups. NOTE to facilitator: Read slide. Hand out copies of the Stakeholder Analysis Matrix and Engagement Plan. Instruct the groups to select a decision they make in their work settings and complete the Stakeholder Analysis Matrix and Engagement Plan around that decision. Specify that because this exercise usually takes more than 45 minutes to complete, they should complete the matrix across (stakeholder description, potential role in activity, level of knowledge, level of commitment, resources) for only one to two stakeholders. Provide at least 45 minutes for this exercise. Remind participants to move on to filling out the Engagement Plan after 25 minutes. They need to complete the Engagement Plan for only one stakeholder.

Group Work Report Back Have note taker transfer your final Stakeholder Analysis Matrix onto flip chart paper Share the decision your group chose Share the priority stakeholders selected Choose 1 stakeholder and share the entire row from the Matrix for that stakeholder Share the Engagement Plan for the same stakeholder Time for report back: 10–15 min per group NOTE to facilitator: Read slide.

Visit us online at http://www.cpc.unc.edu/measure THANK YOU! MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide. Visit us online at http://www.cpc.unc.edu/measure