1. Service Provision Assessment (SPA) and Service Delivery Indicators (SDI) Surveys Shouldn’t they be Merged? Presentation Prepared by Moses Busiga (USAID) and Waly Wane (World Bank) Tanzania DPG Health – November 14, 2012

2. What are SPA Surveys? Sample survey of formal sector health facilities
Provide information about the overall service environment and functioning of components of the health system that may affect the quality of services
Objective: to provide a “snap-shot” of the service environment, resources, and practices used to provide health services in formal sector health facilities

3. What questions can a SPA survey answer?
What is the overall availability of different health services?
e.g., what proportion of facilities offer child health services? family planning services?
To what extent are facilities prepared to provide these services?
What infrastructure, resources and support systems are available, e.g., regular electricity and water supply, trained staff, items for infection control, visual and auditory privacy, service guidelines, management practices,
To what extent does the service delivery process follow generally accepted standards of care?
Does the process followed in service delivery meet standards of acceptable quality and content?
Are clients and service providers satisfied with the service delivery environment?
Clients’ perception of service provided and providers’ satisfaction with the work environment

4. Demographic and Health Surveys (DHS)
INSERT GRAPHIC TO ADD MAP
MAP IS 6.17” TALL
Figure 1. Relationships between Contextual and Individual Factors, the Health System and Health Outcomes
Adapted from: J. Bertrand and Escudero, G., Compendium of Indicators for Evaluating Reproductive Health Programs, Volume One. MEASURE Evaluation Manual Series, No. 6, August HFS = Health Facility Surveys; PBS = Population-based Surveys
DHS, THMIS
SPA
Nutrition
Demographic and Health Surveys (DHS)
Demographic and Health Surveys also include the Malaria Indicator Survey (MIS) and the AIDS Indicator Survey (AIS).
Nationally representative survey on demographic and health indicators
228 completed surveys from 1985 to present
95 surveys with GPS location of surveyed clusters

5. What are the contents of SPA surveys?
Systems
Guidelines & protocols
Referral systems
Staff training and supervision
Services
Maternal & Child Health
ANC, Delivery, Newborn care
Vaccination services
Curative care for children under 5
Growth monitoring for children
Family planning (FP) services
HIV / AIDS CT
ART
PMTCT
Tuberculosis (TB)
Malaria
STI
Non-Communicable Diseases
Infrastructure
Water
Electricity
Client latrine
Items for infection control
Infection control practices
Privacy
Resources
Laboratory Diagnostics
Equipment
Pharmaceuticals

6. Standard SPA Questionnaires
Inventory questionnaire
General information: staffing levels, health care waste management, processing of equipment, methods of eliciting client opinion,
Availability of services (child health services, FP, ANC, Delivery, HIV testing, STI, TB..) and frequency (i.e., # of day/week service is offered)
Detailed information pertaining to the service, if offered: equipment, guidelines
Laboratory diagnostics services: availability of different tests, systems for external testing, QC
Medicine and supplies
Health provider questionnaire
Provider qualifications, services provided, training, continuous education
Supervision
Perception of the work environment
Observation protocol
Observation of consultations and examinations of sick children, ANC, and family planning sessions, STI, etc.
Checklists cover basic elements of service delivery; special studies needed for more info
Client exit interview questionnaire
With clients who were observed receiving services to elicit their perception on consultation and service received

7. SDI Perspective
Perspective adopted by SDI is that of citizens accessing services
Can be viewed as a service delivery report card
However, instead of using citizens’ perceptions, assemble objective and quantitative information from a survey of frontline service delivery units
Uses modules from:
Public Expenditure Tracking Survey (PETS),
Quantitative Service Delivery Survey (QSDS),
Staff Absence Survey (SAS)
Observational studies.

8. Table 1: A health service delivery report card
At the clinic: Inputs and infrastructure
Infrastructure (electricity, water, sanitation)
Medical equipment per clinic
Medical personnel: Effort and knowledge
Absence rate
Time spent counseling patients per clinician
Diagnostic accuracy in outpatient consultations
Funding: Effort in the supply chain
Health expenditures reaching primary clinics
Delays in salaries 8

9. Pilot surveys in Senegal and Tanzania
SDI piloted in Senegal and Tanzania in the spring/summer of 2010.
Main objective: test the survey instruments and to verify that robust indicators of service delivery quality could be collected with single facility-level instrument
Anglophone and Francophone country with different budget systems. 9 9

10. Sample size and design
Designed to provide representative estimates for each of the key Indicators, broken down by urban and rural location.
Stratified multi-stage random sampling design, with number of units proportional to population
Given resources available, 180 facilities to be surveyed in each sector
Sample frames: most recent list of all public primary schools and public health facilities, including information on size of population they serve. 10

11. Sample information 11

12. Survey implementation
Survey used sector-specific questionnaire administered at facility level.
Questionnaires built on international good practice for PETS, QSDS, SAS and observational surveys.
Pre-test was done by SDI team, in collaboration with in-country research partners, in early 2010.
The questionnaires were translated into French for Senegal and Swahili for Tanzania. 12

13. At the clinic: Inputs and infrastructure
Infrastructure (electricity, water, sanitation)
Medical equipment per clinic

14. Infrastructure 14

15. Medical equipment per clinic
DEFINITION:
Measured as = 1 if there is a thermometer, stethoscope and weighting scale available on the premises, and as = 0 otherwise. 15

16. Medical equipment per clinic16

17. Medical personnel: Effort and knowledge
Absence rate
Time spent counseling patients per clinician
Diagnostic accuracy in outpatient consultations

18. Absence Rate 18

19. Time spent counseling patients per clinician
DEFINITION:
Use data on time spent per patient (from the observational study)
Multiply by total number of patients treated per day
Divide by number of clinicians (on the roster)

20. Time spent counseling patients per clinician20

21. Diagnostic accuracy in outpatient consultations
DEFINITION:
Share of clinicians who reached correct diagnosis based on patient case simulations (“vignettes”)

22. What do Patient Case Simulations measure?
Knowledge/skills
(performance frontier)
PCS
Performance with
real patients

23. Diagnostic accuracy

24. Diagnostic accuracy

25. Diagnostic accuracy

26. More information: Process quality
DEFINITION:
A = Share of relevant history taking questions performed.
B = Share of relevant physical examinations performed.
Process quality is the average of A and B, averaged across 5 patient cases.

27. Process quality - number of relevant procedures

28. Case 1: Child with malaria and anaemia Per cent of clinicians who performed procedure
HISTORY TAKING
Sen
Tan
Duration of fever 82 99
Pattern (periodicity) of fever 37 66
Presence of chills, sweats 3 21
Presence of vomiting 47 58
Presence of cough 60 56
Presence of convulsions 6 31
Presence of ear problems 2 7
…and more…
EXAMINATIONS
Sen
Tan
Take temp with a thermometer 81 85
Check neck stffness 2 13
Look for palmor pallor (or other signs of anaemia) 6 27
Check ear/throat 7
General health condition (awake/lethargic) 62
Check for visible wasting 4
Look for oedema both feet 1
Check weight (against growth chart) 11
…and more….. 28

29. Funding: Effort in the supply chain
Health expenditures reaching primary clinics
Delays in salaries

30. Health expenditure reaching primary clinics per capita30

31. TANZANIA HEALTH At the Clinic 18% Medical Personnel 78% Funding 21%
Clinics with electricity, water, sanitation
Clinics with basic equipment[3]
Medical Personnel
Health workers absent on a given day
Time spent counseling patients per day
Cases diagnosed accurately by clinicians[4]
Funding
Expenditure reaching primary clinics per capita
Medical personnel experience a 2- month+ delay in salary
18%
78%
21%
29m
57%
$7.01 2%

32. What’s Next for SPA & SDI in TZ?
Already had a brainstorming session on how to merge SPA and SDI
Merging SPA and SDI has many advantages
SDI will participate in upcoming SPA stakeholder meeting in December
Both SDI and SPA teams are open to alter their timelines to consider and work on the merge
Tentatively the “merged” survey could start field operations before June 2013

33. THANK YOU