Using Information for Health Management; Part I - Health Information Systems Strengthening

Epidemiology is the study (or the science of the study) of the patterns, causes, and effects of health and disease conditions in defined populations. It is the cornerstone of public health, and informs policy decisions and evidence-based medicine by identifying risk factors for disease and targets for preventive medicine.

Learning objectives the information cycle; tools and processes for turning data into action the relationship between data use and data quality hierarchy of standards / essential data set common reasons for compromised data quality, and various counter measures different information products for communicating different meanings

Data, information, knowledge observations and measurements Information (processed data) facts extracted from a set of data (interpreted data), data brought together to demonstrate facts Meaningful and useful Knowledge contextualized information actionable

42 "Answer to the Ultimate Question of Life, the Universe, and Everything".

42 / 60 = 70% Fully Immunized Children < 1y Where 60 is the target population

42 / 60 = 70% Fully Immunized Children < 1y For month May 2010 Organization Unit: Whatever With additional contextual insight we may act upon this information Like we have seen with Johans examples the numerator could be wrong due to counting errors etc. The denominatro may be wrong due to census data (not collected with health org units in mind) data divided by org. Units. Collected for the nation or district and split into lower org units according to some algorithm – Algorithm can be improved by involving geography etc.

Spatial /organizational context Immunization Coverage May 2012 Org.Unit Immunization Coverage May 2012 Whatever 70 Notsogood 40 Verybad 15 Superduper 98

Immunization Coverage for Whatever Temporal context Month 2012 Immunization Coverage for Whatever Jan 83 Feb 80 Mar 70 Apr 52 May 64 Jun 60 Jul 54 Aug 43 Sep 37 Oct 39

Information cycle; from data to action Collection Processing Presentation Action The relationship between data and information is better seen by considering a diagram, that is called info cycle. So, information cycle is a diagrammatic way of looking at data and information. The Inf Cycle enables one to see the links between the different stages of information. What are these stages or phases? As you can see through this diagram, phases of information handling include the process of data capturing, through observation of real events (collecting), processing data (process means ensuring data quality by auditing data (3Cs), collating (aggregating) data… so that one can analyse it and turn it into information. The information is then presented, interpreted and used for decision-making.

Data collection Input: Using data sources and tools to collect quality data Common problems: Too much to collect Poor understanding of data collection tools Timeliness of reporting Low data quality Output: relevant data Collection Processing Presentation Action

Processing Input: Relevant data Processes: Quality checks Aggregation to relevant levels Calculation of indicators Analysis of data => information Common problems: Much irrelevant data Low data quality Limited knowledge of data needs and analysis Output: data converted to information Collection Processing Presentation Action

Presentation Input: Analysed information, to be presented to communicate meaning Common problems: Importance of information is lost through poor presentation Data can be misrepresented or misinterpreted Output: various information products Charts Tables Maps Collection Processing Presentation Action

Action Input: Information products Process: Interpretation of information, prioritizing action, making action plan, and execute action through budgets and plans with goals and targets Common problems: Incomplete information Lack of good targets No plan for evaluation Output: Decision-making for health management Collection Processing Presentation Action

Data collection – at the source of data creation (point of care) Service data collected by nurses and doctors in-between attending to patients Usually several (manual) steps before it is in any database/storage Tally sheets Tally sheet totals at end of month Monthly summary forms which are reported to the next level Often too much to collect for already overworked staff Collection Processing Presentation Action

What data elements should be collected? Collection Processing Presentation Action What data elements should be collected? Cannot be obtained elsewhere (e.g. survey) Are easy to collect (cost vs usefulness) Do not require much additional work or time Can be collected relatively accurately Is part of one or more indicators Because we are dealing with thousands of data elements, we have to select exactly the data we need, the data which will contribute for decision-making.

Comparability of collected data Collection Processing Presentation Action Comparability of collected data Stable standardised definitions To ensure spatial comparability between different facilities, districts, provinces and nations To ensure comparability over time Note: Revising poor indicators /data sets /data elements may not be advisable due to cost and loss of backward comparability

Balancing varying information needs Collection Processing Presentation Action Balancing varying information needs

Essential data sets (EDS) Collection Processing Presentation Action Essential data sets (EDS) As data flows from lower levels to higher levels, less data is required. Data collection normally begins at the community/facility level. It  should then be sent to the district office. From there it may go to a regional office and then to a provincial level before being sent to a national level. The national level will have international reporting requirements. Each level has different requirements. The facility might want to know how many swabs are used on a daily basis. This information would be irrelevant to the national level. However, information such as the infant mortality rate is of international concern and thus important for all levels of the health system. Hierarchy of standards

EDS Example: vaccination data Collection Processing Presentation Action EDS Example: vaccination data Input (community) Staff attendence, vaccines, to whom, when, where Process (district) Vaccination of children Output (province) Coverage of child immunization Outcome (national) Immunization rates going up Impact (international) Healthier children, less disease. Synergies

Where do we get data from? Collection Processing Presentation Action Where do we get data from? Routine data collection Routine health unit and community data Activity data about patients seen and programmes run, routine services and epidemiological surveillance Semi-permanent data about the population served, the facility itself and staff that run it Civil registration (vital events being integrated with health e.g. India) Non-routine data collection Surveys Population census (headcounts proportion/facility catchment’s area) Quantitative or qualitative rapid assessment methods Special registers (birth registers, TB registers, mental health registers)

Data Sources in the HMN data warehouse concept Collection Processing Presentation Action Data Sources in the HMN data warehouse concept

Client record cards Road to Health card Collection Processing Presentation Action Client record cards Road to Health card Record details of the client’s interaction with the health service, e.g.: Health facility record system (traditional) Associated with misfiling and loss vs Client-held record system (Road to Health Card, Child Health Booklet, Women’s Health Book, TB patient treatment card); Associated with efficiency of the individual concern, suitable for mobile community 24

Collection Processing Presentation Action Tally sheets Easy way of counting identical events that do not have to be followed-up (e.g. headcounts, children weighed)

Registers Collection Processing Presentation Action Record data that need follow-up over long periods such as ANC, immunisation, Family Planning, Tuberculosis (TB)

Collection Processing Presentation Action

Reports weekly, monthly, quarterly Collection Processing Presentation Action Reports weekly, monthly, quarterly

Processing; assuring data quality and calculate indicators Turning data into information How to assess data quality? What are indicators, and why do we need them? Collection Processing Presentation Action Review the situation of overloaded health workers who collect a range of stats without knowing why (% nursing time spent on stats collection) Comment on how health workers are often expected to collect stats and send them up without expectation of analysis, feedback & use of information Reinforce how the aim of a DHIS, to improve coverage & quality of local health services, is facilitated by only collecting data that can be analysed & used at the local level

Why checking data is vital? Collection Processing Presentation Action Why checking data is vital? Use of inaccurate data leads to Wrong priorities (focus on the wrong data) Wrong decisions (not applying the right actions) Garbage in = garbage out Producing data is expensive Waste of resources to collect poor data

Routine data should be.. Reliable: Correct, Complete, Consistent Collection Processing Presentation Action Routine data should be.. Reliable: Correct, Complete, Consistent Timely: fixed deadlines for reporting Actionable: no action = throw data away Comparable: same numerator and denominator definitions used by all data processers BUT striving for comparability can compromise local relevance

Complete data? Spatial: submission by all (most) reporting facilities Collection Processing Presentation Action Complete data? Spatial: submission by all (most) reporting facilities Temporal: can you do analysis over time? Does provided services cover the full population? Many indicators depend on population figures as denominators Many developing countries have low service utilization – many people do not use offical health serives or live out of reach

Correct data? Are we collecting the data we need? Collection Processing Presentation Action Correct data? Are we collecting the data we need? The data values seems sensible/plausible? The same definition applied uniformly? Are there any preferential end digits used? JAN FEB MARCH APRIL MAY JUNE JULY 250 230 245 225 240

Collection Processing Presentation Action Consistent data? Data in the similar range as this time last year or similar to other organization units No large gaps or missing data No multiplicity of data (same data from multiple sources –which one to trust?)

Timely data? Late reports weaken the potential for comparison, Collection Processing Presentation Action Timely data? Late reports weaken the potential for comparison, action can be too late, but still useful for documenting trends Sierra Leone

What are the causes of poor data quality? Collection Processing Presentation Action What are the causes of poor data quality? Too many forms to fill out that are not useful to health workers Data collection tools are poorly designed and hard to understand Too many steps of manual aggregation and transfer of figures Limited feedback on data quality to those who collect it Data is not used

Accuracy enhancing principles Collection Processing Presentation Action Accuracy enhancing principles Capacity building through training (90% of HISP activities) User-friendly collection/collation tools Feedback on data errors (but not only!) Availability of processed data Local Use of information Essential Data Set Clear definitions; standards Information sharing

Collection Processing Presentation Action

What can be done to improve data quality? Collection Processing Presentation Action What can be done to improve data quality? Systemic changes Essential dataset Feedback routines Information for action Promote information use at all levels Data validation mechanisms Min/Max rules in software Data validation rules, check for consistency in logic of data Completeness and timeliness reports

Indicators - measure service COVERAGE and QUALITY Collection Processing Presentation Action Indicators - measure service COVERAGE and QUALITY Calculated by combining two or more pieces of data, so that They can measure trends over time They can provide a yardstick whereby facilities / teams can compare themselves to others (spatial, organizational) monitor progress towards defined targets Good for measuring change To do this, indicators need to have a numerator and denominator

Indicator types Collection Processing Presentation Action Reinforce that indicators are measures of the frequency of events Unpack how indicators can be viewed as calculation types – count, proportion & ratio Explore the 3 types of calculations that can be carried out on indicators Explore how indicators can be used to measure events in terms of – count (raw numbers), proportions (aggregated) & ratios

numerator indicator = = % denominator X 100 Collection Processing Presentation Action numerator indicator = X 100 = % denominator Example: How is maternal mortality rate defined? “the number of maternal deaths per 1,000 women of reproductive age in the population (generally defined as 15–44 years of age).” What about maternal mortality ratio? “the number of maternal deaths per 100,000 live births in same time period.” Numerator: Number of deaths assigned to pregnancy-related causes during a given time interval Denominator: Number of live births during the same time interval Multiplier: 100,000 Millenium Development Goals have a set of proposed indicators [weblink] The maternal mortality rate, the number of maternal deaths per 1,000 women of reproductive age in the population (generally defined as 15-44 years of age) . The maternal mortality ratio, the number of maternal deaths per 100,000 live births in same time period.

a count of the event being measured Collection Processing Presentation Action Atop the line – numerators (activities / interventions / events / observations / people) a count of the event being measured How many occurrences are there: morbidity (health problem, disease) mortality (death) resources (manpower, funds, materials) Generally raw data (numbers) Unpack the meaning & use of numerators (illustrate with appropriate examples)

Under the line – denominators (population at risk) Collection Processing Presentation Action Under the line – denominators (population at risk) size of target population at risk of the event What group do they belong to: general population (total, catchment, target) gender population (male / female) age group population (<1, >18, 15-44) cases / events – per (live births, TB case) Unpack the meaning & use of denominators (illustrate with appropriate examples) Explore population groups used most frequently – age group cohorts

An ideal indicator RAVES !!! Collection Processing Presentation Action  An ideal indicator RAVES !!! 

Collection Processing Presentation Action Indicators should be RELIABLE gives the same result if used by different people APPROPRIATE fits with context, capacity, culture and the required decisions VALID truly measures what you want to measure EASY feasible to collect the data SENSITIVE immediately reflects changes in events being measured Unpack formulation of indicators – how to write them Explore how indicators are used in the management & planning of health services Clarify the use of terms: input, process, output & outcome, as they relate to aspects of service management

Essential indicators: determines the essential data set at each level Collection Processing Presentation Action Essential indicators: determines the essential data set at each level As data flows from lower levels to higher levels, less data is required. Data collection normally begins at the community/facility level. It  should then be sent to the district office. From there it may go to a regional office and then to a provincial level before being sent to a national level. The national level will have international reporting requirements. Each level has different requirements. The facility might want to know how many swabs are used on a daily basis. This information would be irrelevant to the national level. However, information such as the infant mortality rate is of international concern and thus important for all levels of the health system. Become familiar with the information flows within and outside your district. Hierarchy of standards

Indicator Operationalization Collection Processing Presentation Action Defining the sources of the data – both numerator & denominator (how is it to be collected?) Determining the frequency of collection and processing of the indicator (How often should it be collected, reported, analyzed?) Determining appropriate levels of aggregation (To where should it be reported and analyzed?) Setting levels of thresholds and target What will be the nature of the action (decision) once the indicator reaches the threshold?