Model for Monitoring and Evaluation of Overall Health System Performance for Comparison Based on the Study Conducted for the Ministry of Health

General Objective Comparison of overall health system performance against selected indicators between districts using one index district from each province.

The mandate was to look at routinely collected and readily generated data at district level to measure the selected indicators. Mandate

Do other countries use health system performance indicators?

Cross-Country Comparison of Concepts of Health System Performance Dimensions and Subcategories of Health System Performance OECD Proposed framework WHO framework Australia’s Proposed framework Canada’s Health System Performance framework UK’s NHS High-level Performance Framework Health Improvements/Outcomes XXXXX AppropriatenessXXX Capacity/CompetenceXX SafetyXXX ResponsivenessXX Patient Satisfaction Patient experience/ accessibility Acceptability XXXX X X X X X Accessibility (in terms of timeliness of services XXX ContinuityXX

Cross-Country Comparison of Concepts of Health System Performance…contd… Dimensions and Subcategories of Health System Performance OECD Proposed framework WHO framework Australia’s Proposed framework Canada’s Health System Performance framework UK’s NHS High-level Performance Framework EquityX Equity of Health OutcomesXX Equity of accessXXXXX Equity of FinanceXX EfficiencyXX Macroeconomic efficiencyX Overall micro efficiencyXX Unit costsXXX

PROCESS

Initially a steering committee was established consisting of ministry officials & the AHF secretariat. Based on the literature review of national and global materials and the documents available with the MoH a draft conceptual framework for measurement of health system performance was presented to Steering Committee and modified taking into account the views of the committee.

Concurrently the study team also looked at the indicators used world wide for performance measurements. After identifying the readily available indicators at provincial and national levels with the concurrence of the steering committee it was decided to place them before high level ministry officials from centre as well as the managers from the provinces.

The following indexed districts one per each province were selected for comparison with the concurrence of the ministry officials & PDHS’s. The selected districts were - Gampaha – Western Province - Ratnapura – Sabaragamuwa -Province - Anuradhapura – North Central-Province - Galle – Southern Province - Matale – Central Province - Trincomalee – North East Province - Badulla – Uva Province - Kurunegala – North Western Province

Sri Lankan Health Performance Framework- A Model HOW HEALTHY ARE THE CITIZENS? IS IT THE SAME FOR EVERYONE? WHERE IS THE MOST OPPORTUNITY FOR IMPROVEMENT? Health conditionLife expectancy and wellbeing Deaths %LBW Infectious diseases -incidence of malaria -incidence of dysentery -TB new cases Prevalence of anaemia among pregnant women Incidence of HIV/AIDS(NSACP) Life expectancy IMR MMR NMR Case fatality rate for dengue fever/DHF Health Status and Outcome Tier 1 IS THE SYSTEM GEARED TO MEET EXPECTATIONS & THE FELT NEEDS OF THE CITIZENS? ResponsivenessAccessibility & Equity Safety These dimensions are not going to be measured during this project. -Indicators not identified. Data has to be obtained from community surveys. Measuring equity in access to health services requires household survey data at district level. -Incidence of adverse drug reaction -incidence of nosocomial infection Responsiveness & Access Tier 2 EffectivenessEfficiencySustainabilityHealth Services Measles coverage Tetanus toxoid (TT2 +) given to pregnant mothers % of pregnant mothers tested for VDRL Pap smear screening rate Number of deaths within 48hrs of admission In pt /staff Hospital bed occupancy rate Average length of stay (MSU) % expenditure for health vs. total budget % of expenditure on drugs CS Rate DOTS treatment success rate Nurses or doctors/hospit al bed (AHB)MSU Hospital beds /1,000 population (AHB)MSU Doctors /100,000 population (AHB)MSU Nurses/100,0 00 population PHM/ 100,000 population (AHB)MSU Patient Transfers No. of new cases of diabetics in hospital clinics for -diabetics -Hypertension -Cancer % of medical audits done for -maternal deaths -still births No. of major surgical operations No. of minor surgical operations Health System Performance Framework Tier 3

Routine data was not available for the following indicators: Prevalence of anaemia among pregnant women Inpatient to Staff Ratio Percentage of expenditure for health vs. total budget Percentage of expenditure on drugs Patient Transfers Number of New cases of NCDs in hospital clinics for: Diabetes Hypertension Cancer Percentage of Medical Audits done for still Births

RESULTS - PILOT STUDY

Health Status and Outcome Tier 1 Health Conditions District

Deaths Infant Mortality Rate (2005) Infant Mortality Rate per 1000 live births Definition Number of deaths to infants under one year of age per 1,000 live births in a given year Trincomalee Galle Matale Average Gampaha Badulla Ratnapura Kurunegala Anuradhapura % District

Health System Performance Tier 3 Effectiveness Percentage of pregnant mothers tested for VDRL (2005) Percentage of Pregnant mothers Tested for VDRL Definition Number of mothers tested for VDRL as a percentage of total number of deliveries reported.

Average length of stay(2005) Average Length of Stay in Hospital Definition The average length of stay a patient spends in a government hospital. It is measured by dividing the total number of days stayed by all inpatients in government hospitals during a year by the number of admissions Efficiency

Caesarean Section Rate (2005) Caesarean Section Rate Trincomalee Ratnapura Anuradhapura Average Badulla Kurunegala Galle Matale Gampaha Rate District Definition The number of caesareans per 100 live births in government hospitals

Sustainability Nurses per 100,000 population(2005)

Health Services Percentage of Medical Audits done for: Percentage of Medical Audits done for Maternal deaths(2005) Percentage of medical audits done for maternal deaths

SUMMARY SHEETS FOR OVERALL HEALTH SECTOR PERFORMANCE BY DISTRICT

Direction for Establishing & Using this Model for Systematic Monitoring for Comparison of Health System Performance between Districts in the Future

POLICY ISSUES

 For future analysis the level of analysis of costing, whether provincial or district, needs to be identified. At present there is no provision to collect and compile the cost data at the district level as they are not accountable for them. Hence a policy decision needs to be taken whether performance comparison should be at district level or at provincial level.

 It is better to look at all inputs from both line ministry and the provincial ministry for an accurate comparison, as the health outcomes etc. will be dependant on all resource inputs to a particular province, rather than through provincial health sources only. This could best be done at the central level.

 It is also recommended to review the IMMR and hospital returns sent to the medical statistician and to modify them to include additional information which, at present, is already available at institution level but not collated and reported.

 Wherever national figures are available for the selected indicators these should be compiled from them as they are more accurate and reliable. Since the numerator and the denominators used will be same and for comparison across the districts.

RECOMMENDATIONS

A system for regular monitoring of health system performance should be established. Provincial health authorities need to adopt this model for their monitoring purposes & use this at review meetings with the district health authorities For this purpose planning cells of PDHS/DPDHS offices has to be strengthened. Timely & accurate electronic data flow to the DPDHS offices from the periphery for compiling the selected indicators should be established.

PDHS’s should provide expenditure data to DPDHs for monitoring purposes. Expenditure on drugs by individual institutions should be monitored at DPDHS level. These data bases from the DPDHS offices should be linked with the PDHS’s planning units & the MDPU of the MoH For the present the responsibility of M & E of Health System Performance could be a joint effort of both organizations MDPU and the AHF secretariat. This responsibility should be transferred to the D/I of the MDPU of the MoH after AHF secretariat cease to function after the project period.

It is to be noted that in other countries too most of the health status indicators are calculated by a central organization for consistency. It is recommended that to get data for the second tier - responsiveness and access, as well as for NCDs, including the risk factor prevalence a national health survey is carried out every three to four years, depending on the resource availability, or alternatively, to look at the feasibility of combining this with DHS survey to cut down costs to the health ministry.