Monitoring and Assessing Pharmaceutical Policies Mr Enrico Cinnella WHO/EMP, Geneva 18 November 2009

Outline of the Presentation Introduction Level I Level II Facility Survey Household Survey

Why countries measure? As a baseline to inform decisions Priority Setting To check how well (or badly) you are performing Two reasons to measure: Fist one is at the beginning of the cycle: to decide what to do. Second one is during the implementation: to see whether results are being achieved.

Country A Knowing the situation RATIONAL USE Last update of EML: 2009 Last update of STGs: 2008 Antiobiotics sold over the counter: NO QUALITY CONTROL System in place for quality control: NO Samples tested for post marketing surveillance: NO System in place for ADR: NO Collecting information to have an overall view of the situation of the pharmaceutical sector. Without that it is difficult to decide in which area to work and what to do.

See how well you are doing Monitoring the availability of medicines is a way of measuring whether policies are achieving their expected results. Linear monitoring across a period of time. It can be seen how availability in the Public Sector is increasing over time. Mention sources of data

Who can use the results ? Countries - focus action, prioritize, measure achievement International agencies to assess the structure and capability of countries, assess the progress, accomplishment and impact of aid Professional groups, NGOs and academia to focus advocacy activities and information campaigns Health facilities to be aware of institutional problems & improve situations Countries -Country budget planning % basis to request fund and support using data and evidence. Donors are now keen to measure progress and output. WHO EC log frame is supported by indicators with target Presentation, report, discussion , debate, backed by data.

Pharmaceutical indicators Variables that measure situations and change Numerical ( numbers, percentage, or averages) Binomials (yes” and “no)” Useful tools to track the performance of particular aspects or activities of the pharmaceutical system Linked to an important input, process, or outcome may be numerical and expressed in terms of numbers, percentage, or averages. They may also be expressed as binomials such as “yes” and “no.” Indicators are useful tools for managers to use to track the performance of particular aspects or activities of the pharmaceutical supply system as well as the performance of the overall system. A well-defined indicator is clearly linked to an important input, process, or outcome. A well-selected indicator will help managers quickly identify potential problems in critical areas. Indicators are extremely helpful to communicate important performance gains and losses to other stakeholders of the pharmaceutical supply system. Indicators can be developed for different levels of the supply system. Managers should be aware that there are well-established indicators to measure the performance of the different components of the pharmaceutical supply system. However, because systems can be organized in different ways, managers should adapt or modify internationally recognized indicators to reflect the realities of their own system if necessary. Field test idicatros before final includsion in the set and at countries with those who will collect the data

The WHO System Level I Questionnaire/rapid assessment/checklist Arrays achievement & weaknessess, illustrate sectoral approaches Level II Comprehensive monitoring of pharmaceutical strategy outcome and impact Measures attainment of objectives Level III More detailed indicators for monitoring and evaluating specific areas/components Questionnaire (Health Officials) Level I Core structure & process indicators Level I Core structure & process indicators Systematic survey Level II Core outcome/impact indicators & household survey Level II Core outcome/impact indicators & household survey Level I- Global overview of policies, structures and process Level II- Results and Outcomes through surveys conducted at facility and household level BOTH COVER the whole spectrum of EMP work Level III goes into detail of individual topics Level III Indicator tools for specific components of the pharmaceutical sector Pricing ●Traditional medicine Human Resources ● Assessing regulatory capacity Procurement and Supply

Core outcome/impact indicators LEVEL I Level I Core structure & process indicators Level II Core outcome/impact indicators & household survey Level III Indicator tools for specific components of the pharmaceutical sector Pricing ●Traditional medicine Human Resources ● Assessing regulatory capacity Procurement and Supply

Level I- A Global Survey Questionnaires sent to MoH officials every four years (1999, 2003 and 2007). Mostly yes/no question. Responses collected and compiled into a global database and global report. Questionnaire developed and send to MoH at regular intervals. Explain well how the questionnaire is sent through official channels- Regions- Countries- MoH Simple questionnaire, mostly yes/no question. Easy analysis. Global report is produced, but regions can and do produce their own reports.

Level I- What does it cover? Policies and Structures in 6 areas: Medicines Policies Regulation Procurement and Supply Financing Production and Trade Rational Use The whole Pharmaceutical situation is covered

Advantages of the level I Little Financial and HR Investment. Number of countries covered (156 in 2007). Allows An overview of the global situation Comparisons across regions Comparisons across time Financial investment is close to O Time investment is limited compared to other surveys and studies. Most member states are covered.

The Global situation in 2007 Official NMP document updated within the last 5 years It allows to have an overview of the global situation, but also to compare across regions. Most African countries have a NMP document, but not many of them have updated it recently. This may change soon as many countries are about to deliver (Mali, Uganda, Tanzania).

Have we made progress? Standard Treatment Guidelines 2003-7 Level I allow to compare the situation every 4 years. Progress has been made in all income levels. Low-income countries have performed as well as high income ones. In this case the African Region is doing particularly well.

Limitations of WHO Level I Data quality Ownership of the data Does not tell anything about outcomes and impact Data quality not always great. The level of respondents varies from country to country. Difficult to come back to them due to number of countries. Ownership of data. Countries give data, but them to access them have to write (and know that we have them). They make an effort to put information from different sources together, but do not benefit fully from this effort. Outcomes. Having STGs is fine, but what really matters is whether treatment of patients does follow them or not. Level I does not tell us anything about it.

The SADC experience Assessment of Pharmaceutical situation in 15 countries Tools similar to level I, but including indicators from other surveys Main changes in the process: Tool prefilled (more speed, less effort and duplication). Data quality- official endorsement. Data ownership- data used to build Profiles to be shared with countries. Countries will have all their information in a single document. Geographically Limited assessment. Tools similar to level I, but pulling together info from other surveys to give a more complete picture. Tool prefilled: less burden on countries, if we have the information we will not ask again for it, only to confirm. Official endorsement should better guarantee the quality of data. Information used to build profiles that will be sent back to countries. Countries will then have all their pharmaceutical information compiled in a single document.

Core outcome/impact indicators Level II Level I Core structure & process indicators Level II Core outcome/impact indicators & household survey Level III Indicator tools for specific components of the pharmaceutical sector Pricing ●Traditional medicine Human Resources ● Assessing regulatory capacity Procurement and Supply

Level I vs Level II LEVEL I Single Questionnaire Little Investment Policies and Structures Based on EMP's strategy and vision LEVEL II Survey with sampling More expensive and time-consuming Outcomes Based on EMP's strategy and vision Single questionnaire- Survey-same questions asked in a number of facilities/setting. Expensive:60 days (10x6) or 150 days (10x6) Structures vs Outcomes Both of them covered the full pharmaceutical situation, from affordability to rational use.

From Structures to outcomes Level I- Is there a EML? Level II- Is the EML available at facility level? What is the percentage of medicines prescribed that are in the EML? LEVEL II- Are policies achieving their effect? Availability- of a list of tracer medicines, and number of stock out days. Affordability- (number of days of pay to purchase treatment for selected illnesses). Quality- % Adequately labelled medicines, % expired medicines. Rational use- INRUD Prescribing indicators. Structures and Outcomes INRUD: number of medicines prescribed per patient, % of patients given antibiotics, % of patients given injections; % of medicines prescribed in generic name; % of medicines prescribed that are in the EML, What is covered by Level II

Facility Survey 5 Areas selected- The Capital (or richest), the most rural one (or poorest) and another 3. In each Areas select 1 Warehouse, 6 Public Health Care Facilities, 6 private drugs outlets. 5 Warehouses 30 Public Health Care Facilities Private Medicines Outlets Sampling of facility survey. May be adapted to country situation (e.g. Tonga)

Some results Results from Uganda. High antibiotics use. Most medicines are in the EML Generic Prescription good.

Household Survey

The household survey sampling scheme 5 regions in the country From each region select 6 public health facilities (30 reference public health facilities) In each of reference facility, select 30 households (900 households) 8

Household Survey Household situations How they access their medicines, where they get them How much they pay Identify access and affordability in relation to socio economic indicators, barriers Examine use of medicines (acute and chronic diseases) Perceptions on access, use and quality; handling of medicines Covers all areas of Medicine Policy- Again from Access to Rational Use. Household perspective. Allow more than facility survey- How many are accessing medicines? From where are they buying/getting them? Gives an idea of how people are doing in terms of access to medicines, but also on their opinions and perceptions.

Households with medicines at Home Allow to see whether different socioeconomic groups have differential access to medicines. In this case, richer people are more likely to have medicines at home.

Geographical issues COUNTRY A Do you receive free medicines at public health care facilities? 95% 56% Allow also to see whether there differences between areas of the same countries. Here, the further we go from the capital, the less likely people are to get free medicines from public facilities? 96% 97% 77%

Medicines Expenditure Putting countries together is a good way of benchmarking and have an idea of how well you are doing. Here we can that catastrophic expenditure on medicines is very different in countries that have a different financing system (free for B, completely paying for C).

Source of medicines Source of medicines, may be public facilities or private retailers. Country A private pharmacy. Country B Public Dispensary. Country C: Public Hospitals and Private Pharmacies This is very useful to see where to aim effort to control quality/promote rational dispensing.

Tools and Reports Level I Assessment 2003 Manual Level II Assessment http://apps.who.int/medicinedocs/documents/s14101e/s14101e.pdf Manual Level II Assessment http://apps.who.int/medicinedocs/documents/s14877e/s14877e.pdf Uganda Facility Survey 2008 http://apps.who.int/medicinedocs/documents/s16377e/s16377e.pdf Uganda Household Survey 2008 http://apps.who.int/medicinedocs/documents/s16374e/s16374e.pdf

Thank you for your attention