Workshop on the Vaccine Fund (Global Fund) 1. Structure, Sub-Accounts and Application Process 2. After application and approval – Reports and Monitoring Discussion Coffee Break 3. Discussion on ICCs 4. Discussion on Multi-Year and Annual Plans

Expectations Please identify up to 3 Issues or Questions you would like to address this afternoon.

Vaccine Fund - Structure, Application, Monitoring and Reports Alan Brooks, MS RN Program Officer Gates Children’s Vaccine Program at PATH June 24-27, 2001 St. Petersburg, Russian Federation

Objectives Strengthen common understandings about the Vaccine Fund. Review how to complete an effective application to the Vaccine Fund. Discuss monitoring and reporting after approval of an application

Presentation Map

1. Structure, Sub-Accounts and Application Process

Major Themes The Vaccine Fund is new and, therefore, evolving. Countries are encouraged to provide feedback to the Vaccine Fund to make the process stronger. Applications to the Vaccine Fund and on-going country activities are designed to integrate with and support immunization system components (in-country coordination, assessments, multi-year planning) The Independent Review Committee must balance between a checklist and the ability to understand the true status/operating of each component of the application. The hardest part of the work begins with the approval of an application.

The Vaccine Fund Working Capital Account (UNICEF) Vaccine Procurement/Cash GAVI Board Government ICC Assessment Multi-Year Plan Technical Financial Proposal Vaccine Fund Cycle

Overall Eligibility for Support from the Fund (Who can apply) GNP/capita </= $1000 Population > 150 million (UN) subject to “Special Negotiations” Only national governments can apply (according to WB Statistics) (140) (71) (3)(3) (Number of Countries)

Eligibility Details Changes in country GNP/Capita:  If falls below $1000, become eligible to apply to the Vaccine Fund  If rises above $1000 remain eligible for the remainder of their approved Vaccine Fund 5-year period. Applications received by Spring, 2002 may receive up to 5 years support

Application Review Process 2. Initial review by WHO – Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 1. Application submission by country

Application Review Process 1. Application submission by country 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 2. Initial review by WHO - Calculations

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 3. Russian/French Translated to English

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 4. Independent Review Committee of 7-10 people convenes. Members from Mali, Tunisia, Philippines, Ghana, Tanzania, Moldova, Thailand, Slovenia and 1 person from the US. Strong experience with immunization program management.

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 5. Each application reviewed in entirety by at least 3 people and in brief by all members (Conflict of Interest Disqualified from review)

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 6. Full group discussion and recommendations

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 7. Recommendations to Working Group

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 8. Recommendations to GAVI Board for Decision

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 9. Vaccine Fund Board endorsement of GAVI Board decision

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer)

Application Review Process 1. Application submission by country 2. Initial review by WHO -Calculations 3. French/Russian Translated to English 4. Independent Review Committee of 7-10 People Convenes (from Mali, Slovenia, Tunisia, etc) 5. Each application reviewed in entirety by at least 3 people (Conflict of Interest Disqualified from review), and in brief by all members 6. Full group discussion and recommendations 7. Recommendations to Working Group 8. Recommendations to GAVI Board for Decision 9. Vaccine Fund Board endorsement of GAVI Board decision 10. Vaccine Fund releases money based upon GAVI Board recommendation to UNICEF 11. UNICEF Releases money for infrastructure (to country) or vaccine (usually to manufacturer) 2 Months 1-6 Months

Lessons from Applications 1 General Consideration Provide a logical and realistic picture of how the general criteria (ICC, Coordination, Multi-Year Planning) are working together. Monitor progress on the implementation of recommendations from the immunization assessment, showing how the findings are being addressed. In Country Coordination Develop ICC work plan for the next 12 months including activities to strengthen ICC and address gaps in criteria

Multi-Year Plan Use realistic objectives and targets. Projections for increased DTP3 should be accompanied by appropriate, clear program strategies. For vaccine request projections, target for Hep B or Hib in routine immunization =DTP3, not necessarily 100% of birth cohort. Data in Multi-Year Plans. Data for planning should be accurate and consistent within a document and plan. Challenge of obtaining standard DTP3 estimates. Lessons from Applications 2

Financing Financial plan should reflect the whole EPI and ensure that donors do not withdraw from their current funding (GAVI will not replace current funding for HB vaccine.) Lessons from Applications 3

Application Assessment Criteria (What is looked at) 2 Levels of Criteria General (All Applications) Sub-Account Specific New and Under- Utilized Vaccines Infrastructure Strengthening

General Assessment Criteria 1. Coordination 2. Assessment 3. Multi-Year Plan Consider the challenges faced by the Independent Review Committee in attempting to evaluate these criteria in an application

General Assessment Criteria 1. Coordination Role, responsibility, and functioning of Inter-Agency Coordinating Committees (ICCs) or equivalent Terms of reference that incorporate all aspects of immunization services Frequency of meetings Minutes of the meetings and how they are circulated to members The level of the ICC chairman within the Ministry of Health The list of members Plans and budget requirements for strengthening the ICC if necessary Are the government and appropriate partners working together to comprehensively address program needs?

General Assessment Criteria 2. Assessments Data should include: External environment Health system context Immunization operations: immunization service delivery disease surveillance and MIS monitoring cold chain, logistics and transport vaccine supply, utilization, quality and stock-control advocacy and communications (including telecommunications) immunization safety Financing requirements for strengthening the ICC if necessary Is there adequate data to understand the immunization situation in the country and plan for appropriate strengthening?

General Assessment Criteria 3. Multi-Year Plans Elements to include: Immunization Operations (5 Functional Areas) Address priority elements, especially polio eradication and related activities. Further, priority areas to highlight in the application include: Management of immunization services Capacity Building Safety of Injections (Refer to Safe Injection Checklist) Sustainable financing for immunization systems New Vaccine Introduction Plan where relevant (Refer to Introduction of New Vaccines Checklist) Is it a comprehensive, reasonable plan that responds to the assessment data and moves to strengthen the immunization system? Good sample plans are available.

Presentation Map

New and Under-Utilized Vaccine Sub-Account Specific Assessment Criteria Vaccine Fund will pay for up to 5 years of vaccine: Hepatitis B (Any eligible country) Hib (Where burden of disease data) Specific Criteria: National DTP3 >50% Plan for introduction of new vaccines Action plans within multi-year plan to achieve: safe injections and safe disposal reduction of vaccine wastage sustained financing of immunization services (Mid-term report) (by 1 Year of Age) Vaccines are for routine EPI use only and excludes catch-up and adult immunization.

Infrastructure Strengthening Sub-Account Specific Assessment Criteria Vaccine Fund will pay 1 “share” for each additional child to receive third dose of DTP (DTP3) above baseline. Current share value = $20/additional child Specific Criteria: DTP3 Coverage < 80% Annual targets for increasing number of children to receive DTP3 Action plans within multi-year plan to achieve: safe injections and safe disposal reduction of vaccine wastage sustained financing of immunization services (Mid-term report) (by 1 Year of Age)

Principles of Infrastructure Strengthening Shares Share is given in the first year child reached, thus, Multi- Year Plans must include on-going support to maintain that coverage. Resources can go to country or partner agency at discretion of government and ICC. No restrictions or international monitoring of use Performance monitoring--Additional children reached with DTP3 according to district level data; 3 additional indicators determined by ICC. If no progress after 2 years, funds will be discontinued.

Current Use of Infrastructure Funds in the Region 4 countries have applied for the sub-account Criteria: DTP3 Coverage <80% by 1 year of age Measurement: Routine reporting or Survey by Card + Recall

Share Calculations: Table 1 Estimating Children

Table 2: Calculations

Table 3: Timeline for Payments

New Endorsements from GAVI Board June, 2001 Auto-Disable Syringes WHO/UNICEF Recommendation Policy provides up to 3 years of support for AD Syringes for all routine EPI Vaccines to assist transition Develop and submit plan, likely to include implementation, waste disposal and phasing out support Review & Approval: To be determined. Likely by Independent Review Committee Formal notification of requirement, including for previously approved countries will be forthcoming, from GAVI Secretariat.

New Endorsements from GAVI Board June, 2001 Short-Term Support to Countries Introducing New Vaccines, with DTP3 Coverage>80% Reports of difficulty obtaining money in short-term to pay for key aspects of multi-year plan to allow new vaccine introduction Policy provides $100,000/country (about 20 total) Funds come from Infrastructure Strengthening sub-account & handled in same manner. Money to government or ICC partner as determined in country. Monitoring of use by government & ICC. The government and ICC partners are expected to make additional resources available to ensure safe and effective introduction of vaccines nationwide. Formal notification, including for previously approved countries will be forthcoming, from GAVI Secretariat.

2. What Happens When an Application is Approved: Reports and Monitoring Sustainability Plans

Presentation Map

Reports and Monitoring Processes are designed and timed as best possible to: 1) Avoid duplication with other systems (e.g. WHO/UNICEF Joint Reporting Form) 2) Reinforce and strengthen ongoing management and reporting mechanisms used in most countries (e.g. Development and revision of Annual Work Plans) 3) Maintain primary responsibility for monitoring and evaluation with the ICC. 4) Verify performance at district level (DTP3 coverage)

Monitoring Activities 1) Government and ICC (ongoing) 2) Satisfactory Reports (One required each year)  Inception Report  Annual Progress Report, except at times of...  Midterm & Final Reviews (at end of year 2 and year 5) 3) Nationally Defined Indicators (Inception and Annual Reports) 4) Data Quality Audit (DQA)

ICC’s & Monitoring ICC should play lead role in ongoing monitoring Regular interaction between government and partners Participate in review of progress on annual work plans and multi-year plans Participate in review of financing (budget and expenditure)

Timeline of Reports & Payments

Inception Report Sent to countries in May, 2001 Due September/December, 2001 Purpose:  Track the award process, obstacles and problems.  Re-confirm or revise the quantities of new and under-used vaccines needed in the subsequent year.  Monitor progress with regard to implementation of safety plans and development of financial sustainability plans (full financial plan will not be required until the mid-term review).  Gather information on countries’ chosen indicators in management performance, disease surveillance and injection safety. Requires Endorsement of ICC before Submission Not Used to Alter “Infrastructure Strengthening” payments

Annual Progress Reports Process to determine content not yet begun Likely similar to Inception Report, consistent with principles set by GAVI Board, looking at progress towards agreed indicators. Children Immunized with DTP3, and new and under-used vaccines where applicable, by one-year of age. Data should be reported by district monthly reports. In some countries, reports will be externally verified by the data quality audit (DQA) and shares adjusted. Related to Annual Work-Plans MoH/ICC responsibility; Should not require external assistance for completion

Mid-Term & Final Reviews 1. Possible objectives of these reports: Monitor performance - (trends in number of children reached; adjustment of payments for shares based upon actual children reached) Monitor financial sustainability strategies Monitor and reinforce general assessment criteria mechanisms: In-Country Coordination; Assessments; Multi-Year Plans 2. Key issues likely to characterize the review:  External technical support may be required  Financial sustainability plan included  Possibly challenge countries to include work on Capacity Building for Programmatic and Institutional Sustainability.

Why Data Quality Audit (DQA)? Accurate administrative reporting system is needed to monitor and manage: by all immunization managers to monitor progress, detect problems and test solutions by the Partners of the ICC to see the impact of technical and financial inputs by the Vaccine Fund to assure that reward payments relate to additional children vaccinated, as substantiated by reports. Source: John Lloyd, CVP

DQA & Adjusting Share Allocations DQA verifies only the children whose immunization record is available -GAVI’s use of DQA acts as an incentive to record and report children more completely and in a more timely fashion In situations where a discrepancy is found between report and audit, DQA calculates an “adjustment factor” to increase or decrease the estimated number immunized, and therefore share payments

Implementation Plan for DQA Methodology developed by WHO To be implemented by auditors from coalition of 3 international firms. 2 staff spending 2 weeks in a country Being piloted in July-September 2001 in 9 countries receiving greatest share payments (none in the region) Review methodology in 2002, after which may be possible that countries in the region getting substantial shares could be audited

DQA doesn’t measure % DQA verifies only part of the numerator of immunization coverage ( the part that is recorded ) The denominator is: unknown below district level accuracy compromised by census projection at national level mortality and fertility estimates at district level Coverage is better estimated by survey but, surveys costly, infrequent, insufficiently accurate for management, and of variable quality

Sustainability Plan Required of all countries at Mid-Term Review (First in October, 2002) Financing Task Force process underway to further define essential plan components and strengthen understanding Plan should reflect full range of immunization program objectives and costs

Sustainability Plan - Timeframe Nov 2000-June 2001: Develop Framework Document June 2001: Workshop with Countries; New definition proposed June 2001: Preliminary report to GAVI Board & definition agreed June-Oct 2001: Refine and circulate draft plan guidelines, indicators of sustainability. Consider appropriate level of assistance for completing the plan Nov 2001-Oct 2002: Plan development Oct 2002: First plans due with Mid-Term Review

Discussion