Data Demand & Use (DDU) Why we collect health-related data

Session Overview  Understanding the need for data  Importance of improving data informed decision making  Explain the context of decision-making  Role of monitoring and evaluation data in decision making  Highlight determinants of data use  List potential barriers to data use  Assessment of Barriers to Data Use Tool  Explain the relationship of stakeholders to the data use in decision making cycle  Stakeholder Assessment and Engagement Tools  Linking decisions and questions with potential data sources  Identify priority decisions and programmatic questions  Create a time-bound plan for using data in decision making  Framework for Linking Data with Action Tool

Understanding the need for data

Decisions Within Programs  Ensure program activities are executed as planned & services are delivered as intended  Are we doing the right things, doing them right, and reaching those we intended?  Decisions about:  employment & manpower  mobilization & allocation of resources  needed information and developing feedback channels

Factors other than evidence-based information influence decisions  Power relationships  Timing  Competing priorities  Public opinion  Political ideology  Arbitrariness  Local culture of decision-making  Other information sources

Understanding Decision-Making  What is the decision to be made?  Who makes it?  When and why is the decision made?  How is the decision made?  What information is needed?  What is my role in decision making?

Why Improve Data Informed Decision Making?  HIV epidemic  Resurgence of TB  Continued prevalence of malaria  Pockets of stalled fertility decline  Population burden  Shortage of health care workers

Context Pressing need to develop health policies, strategies and interventions

Why Improve Data Informed Decision Making?  Increased financial investments for service delivery  Increased accountability requirements  Improved national HMIS  Increased demand for evaluation and other research

Why address data demand & use?

Evidence-based Decision Making Process

Level of Dissatisfaction that Policy is Based on Scientific Evidence Percent dissatisfied Overseas Development Institute, Jones et al., 2008.

Challenges  Integrated HMIS still not fully functioning  Little or no communication between data producers and data users  Low capacity to collect, analyze & interpret data  Limited or no culture of data use  Data collection and use not a priority

The Response Monitoring & Evaluation Systems Better Health Outcomes Data Informed Decisions

Group Participation How do you and your organization use data and information?

We can use information to…  Inform health policies and plans  Raise additional resources  Strengthen programs and improve results  Ensure accountability and reporting  Improve quality of services provided  Contribute to global lessons learned

“Making Data Speak” in Thailand  Need: Strengthen commitment of policy makers to HIV Prevention  Data: Behavioral and epidemiological data  Response:  Analyzed data with Asian Epidemic Model and Goals model  Determined responses and resources needed  Communicated data to stakeholders  Decision/Action:  Successfully emphasized prevention agenda in national strategic plan and developed an operation plan to guide prevention programming

Key Messages  Decisions based on evidence lead to better health outcomes  We all have a role in M&E – partners in progress  High quality information is needed for decision- making at policy, planning and program levels  Purpose of M&E is not just to produce more information but to inform action

Determinants of DDU

What Determines Data Demand & Use? ORGANIZATIONAL TECHNICAL BEHAVIORAL * Based on PRISM analytical framework (LaFond, Fields et al The PRISM: An Analytical Framework for Understanding Performance of Health Information Systems in Developing Countries. MEASURE Evaluation).

Data are often underutilized because of… Technical constraints  Individual technical skills,  Availability of computers,  Data system design,  Definition of indicators,  Lack of data quality assurance protocols

Data are often underutilized because of… Organizational constraints  Structural – roads, telecommunication  Organizational – clarity of roles, support, flow of information  Political interference

Data are often underutilized because of… Individual constraints  Decision maker attitudes,  Staff motivation,  Lack of “data culture”

What Determines Data Demand & Use? ORGANIZATIONAL TECHNICAL BEHAVIORAL POLITICS CULTURE SOCIETY * Based on PRISM analytical Fields et al The PRISM: An Analytical Framework for Understanding Performance of Health Information Systems in Developing Countries. MEASURE Evaluation).

What barriers have you faced to using or getting others to use data and information? Discussion

Assessment of Data Use Constraints Tool  Purpose  To improve understanding of the demand for data and the constraints to data use  Description  Key informant interview guide designed to identify constraints  Identifies effective practices in data use  Two versions - Facility level assessment & national and sub-national assessment

Assessment of Data Use Constraints Tool Technical Constraints Technical constraints are related to the ability to generate high-quality data and analyses. RA8Have you ever had an experience while making a policy or program related decision when you were concerned about the quality of the information being used? RA9Are there multiple sources of information or statistics for issues of importance to you, and have you experienced any problems caused by having different estimates? RA10I am interested in knowing about technical capacity for collecting and using information. Does your agency have the technical capacity to produce reliable information without a lot of external technical assistance? RA11Does your agency have the technical capacity to ensure access to and availability of reliable data? RA12Has there been an occasion when data quality or local technical capacity made it difficult for you to use information in making a decision? RA13How would you have gone about preventing this situation?

Barrier: Steps involved Person Responsible Other stake- holders General timeline Addressing Barriers to Using Data and Information in Decision-making

Activity: Assessment of Data Use Constraints Tool Choose a note taker Discuss barriers to data use experienced in your work. Here are some questions to start your discussion:  Have you ever had an experience while making a policy or program related decision when you were concerned about the quality of the information being used?  Does your agency have the technical capacity to ensure access to and availability of reliable data?  What specific challenges have you experienced among your staff when it comes to using data?  How does your organization support having the necessary information to make decisions? Time for activity: 45 minues

Barrier: Lack of technical capacity in M&E Steps involved Person Responsible Other stake- holders General timeline Proposed Intervention: Train all program managers in X organization on basic monitoring and evaluation (indicators, developing M&E plan, documenting results, managing towards results, etc.) Identify funding DirectorDeputy, program mgr January 2008 Seek out trainers and curricula M&E Specialist Deputy director, Training coordinator February 2008 Addressing Barriers to Using Data and Information in Decision-making

Report Back on the Assessment of Data Use Constraints Tool  Share priority barriers  Discuss solutions crafted  Present action plan for two priority barriers  Report back: 2 groups, each 10 minutes

Context of decision-making

Discussion How can we ensure that information is being used to make diagnoses and inform decisions?

Stake- holders Decisions Context of Decision-making Data

Stake- holders Decisions Context of Decision-making Data

What is a Stakeholder? Any person, group or organization with a particular interest in a policy or program  Government agencies  Beneficiaries  Policymakers  Funding agencies  Providers / Implementers  Civil society  Researchers  M&E Specialists

Stakeholders  Non-governmental organizations  Professional associations  Religious leaders  Journalists/media  Private sector/business

Data Producers vs. Data Users Data producers think that decision-makers: Value “political” considerations over evidence Are unprepared to measure or evaluate the consequences of their decisions Decision makers or data users think that health researchers and M&E specialists:  Lack responsiveness to priorities  Favor numbers / jargon to transparent communication  Prefer written reports to face-to-face conversation

Importance of Knowing Your Stakeholders  View activities from different perspectives  Have different degrees of understanding  Need/want different information  Need information at different levels of complexity  Have different intensities of interest  Have different roles in the decision making process

Results of Involving Stakeholders in Data Use Process Relevance of data Ownership of data Appropriate dissemination of data Use of data

Stakeholder Analysis Matrix & Engagement Plan  Clarifies who has interest in a program and what that interest is  Identifies who can help a program and how, and who can hurt it  Helps you use this information for the success of the planning effort

Stakeholder Analysis Matrix The Stakeholder Analysis tool is a matrix framework and process for:  Identifying stakeholders  Defining their roles and resources  Identifying dynamics among stakeholders  Setting the optimum stakeholder group

Stakeholder Analysis Matrix Name of stakeholder organization, group or individual Stakeholder description Primary purpose, affiliation, funding Potential role in the issue or activity Level of knowledge of the issue Level of commitment Support or oppose the activity, to what extent, and why? Available resources Staff, money, technology, information, influence

Name of stakeholder organization, group or individual Stakeholder description Primary purpose, affiliation, funding Potential role in the issue or activity Level of knowledge of the issue Level of commitment Support or oppose the activity, to what extent, and why? Available resources Staff, money, technology, information, influence National AIDS Control Committee Involved in planning, implementation, M&E of all HIV/AIDS programs in the country; approves donor and NGO-funded HIV/AIDS programs Facilitate the stakeholder meeting, prepares for meeting by identifying data sources and preparing an agenda High – receives reports on PMTCT activities from MCH division at MOH; Medium level of knowledge of int’l guidelines and studies Strongly support the activity, but hesitant to use international data sources. NACC opposes use of the DHS and most recent international estimates as they consider these sources to overestimate HIV prevalence Staff available to facilitate; Room and computers available for meetings at NACC headquarters Stakeholder Analysis Matrix Program issueDevelop plan (inc. M&E plan) to scale-up PMTCT programs throughout system Proposed activityConvene stakeholders to identify priorities based on available data and develop action plan Date November 2006

How to Involve Stakeholder  Quarterly program management meetings  Quarterly meetings to interpret RHIS data  Involvement of facility staff to interpret program data  M&E System improvement  Indicator planning and/or harmonization  Data quality review meetings

Stakeholder Engagement Plan Stakeholder organization, group or individual Potential role in the activity Engagement strategy How will you engage this stakeholder in the activity? Follow-up strategy Plans for feedback or continued involvement Program issue Proposed activity Date

Stakeholder Engagement Plan Stakeholder organization, group or individual Potential role in the activity Engagement strategy How will you engage this stakeholder in the activity? Follow-up strategy Plans for feedback or continued involvement National AIDS Control Committee Facilitate the stakeholder meeting, prepares for meeting by identifying data sources and preparing an agenda that allows for the sources to be discussed The NACC is the lead in this activity. It will be important for the NACC to involve more specifically the PMTCT coordinator, clinical care coordinator and National AIDS Program Coordinator The NACC is responsible for following up with the stakeholders prioritized Program issueDevelop plan (inc. M&E plan) to scale-up PMTCT programs throughout system Proposed activityConvene stakeholders to identify priorities based on available data and develop action plan Date November 2006

Stake- holders Decisions Context of Decision-making Data

Decision Areas  Program design and evaluation  Program management and improvement  Strategic planning  Advocacy and policy development

Program Design & Evaluation  Design  Select messages for prevention campaigns  Evaluation  Determine if new program approaches are needed to ensure that health impact objectives are met

Program Management and Improvement  Management  Determine if the program is meeting its process objectives  Improvement  Develop new strategies to increase coverage

Strategic Planning  Identify geographic areas of highest need  Determine human resource allocation  Determine which of offered services is making the greatest impact

Advocacy and Policy Development  Identifying and quantifying underserved populations  Identifying focus areas for new policies

Stake- holders Decisions Context of decision-making Data

Data and Information  Census  Vital events data  Surveillance data  Household surveys  Facilities level service statistics  Financial and management information  Modeling, estimates and projections  Health research

Stake- holders Decisions Factors Other than Data that Influence Decisions Data Political Ideology Competing Priorities Power Relationships Public O pinion Arbitrariness

Stakeholders new counterparts Involve new counterparts Strengthening the Decision-Making Process Decisions realities Understand service delivery realities Data data May require additional data

Activity: Stakeholder matrix & engagement plan  Break into small groups  Each group should select a decision that they make in their work settings and complete the stakeholder analysis around that decision.  A minimum of 7 stakeholders should be identified  Complete the matrix across the columns for 1-2 stakeholders.  Select one stakeholder and complete the stakeholder engagement plan for that stakeholder  Time for activity : 45 minutes

Group Work Report Back  Have note taker transfer your final Stakeholder Analysis Matrix onto flip chart paper  Share the decision your group chose  Share the priority stakeholders selected  Choose 1 stakeholder and share the entire row from the matrix for that stakeholder  Share the Engagement Plan for the same stakeholder  2 groups report back: 10 minutes each

Linking decisions and questions with potential data sources

Building Data Use into Your Work  PLAN PLAN PLAN !  Regularly review your data – schedule time  Use the Framework for Linking Data With Action  Engage in dialogue with stakeholders to fully understand the  decisions they make  information they need  best way to present that information

Elements of the Framework  Decision makers and stakeholders with potential interest in your data  Decisions / Actions that the stakeholder makes (possible uses of data)  Questions to which the stakeholder requires answers  When the decision will be made

Elements of the Framework cont’d  Indicators and/or data of interest (to respond to stakeholder need)  Source of data  How will data be presented (what types of analyses, graphs, formats)?

Framework For Linking Data With Action Decision / Action Program/ Policy Question Decision Maker (DM) Other Stakehold- ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Channel

Framework For Linking Data With Action Decision/ Action Program /Policy Question Decision Maker (DM) Other Stakehold- ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel

What Are Decisions?  Choices that lead to action  All decisions are informed by questions  All questions should be based data

Decisions  Allocation of resources across countries/ states / districts/facilities  Revising OVC program approaches to emphasize fostering and adoption  Develop and institute workplace policies on HIV/AIDS in all institutions in state X  Hire and allocate staff to facilities

Programmatic Questions  What percentage of HIV+ pregnant women in care are actually delivering in health facilities?  What percentage of clients starting ART are lost to follow up?  Are the number of family planning clients decreasing?  What percentage of pregnant patients who are HIV+ are actually receiving ART?

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehol ders (OS) Indicator/ data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: Client?

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: Client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training

Framework For Linking Data With Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training 711 form indicator s K41, B73, B 91 Service statistics Logistics manage- ment system

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training 711 form indicator s K41, B73, B 91 Service statistics Logistics manage- ment system

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicato r/data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training 711 form indicator s K41, B73, B 91 Service statistics Logistics manage- ment system Dec. 2010, March 2011, June 2011, September 2011, December 2011

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training 711 form indicator s K41, B73, B 91 Service statistics Logistics manage- ment system Dec. 2010

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training 711 form indicator s K41, B73, B 91 Service statistics Logistics manage- ment system Dec. 2010Short summary presented to facility manager at weekly clinic meeting

Framework For Linking Data With Action Decision/ Action Program/ Policy Question Decision Maker (DM) Other Stakehold ers (OS) Indicator /data Data Source Timeline (Analysis) (Decision) Commu- nication Chanel DM OS Hire more PMTCT counsel- ors Are we reaching testing targets in PMTCT? Do we have sufficient test kits? What is nurse: client? DM – Head of Regional Health Committee OS – Other providers, Division of Clinical Training 711 form indicator s K41, B73, B 91 Service statistics Logistics manage- ment system Dec. 2010Short summary presented to facility manager at weekly clinic meeting

Framework for Linking Data with Action  Creates a time bound plan for information informed decision making  Encourages greater use of existing information  Monitors the use of information in decision making

Activity: Framework for Linking Data with Action  Select a note taker  On flip chart paper create the Framework table  Brainstorm 3 decisions or questions in columns 1 & 2  Complete the remaining columns  Time: 1 hour

Small Group Activity - Report Back  Remaining group will have 10 minutes to present their completed Framework  Group discussion - are there other data sources that might have been used in this decision? Were there other stakeholders that should have been considered? (10 minutes)

MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.