Population Size Estimation and coverage calculation for MARPs and MARA Dave Burrows, Director AIDS Projects Management Group

“Coverage” Perhaps the most mis-named, misused and least understood concept in HIV work Coverage means whatever the person using it chooses to mean Most common use: % of people ever reached (or reached in 1 year) with an intervention: this is an utterly useless statistic If 100% of IDUs are reached once with education or a new needle & syringe, or if MSM or SW are reached once with education or a condom, it will have virtually no impact on a HIV epidemic

1 st problem is PSE PSE increasingly needed for national HIV plans & GF projects: if do not know size of population, how can we estimate coverage after 5 or 6 years of programs + plan scale-up? Whatever definition of coverage is used, it almost always begins with “ % of X population (IDUs, MARA, etc) ” X population is the denominator for all further calculations related to coverage and its constituent parts: reach, regularity of reach, breadth of services, quality To find X population, population size estimation (PSE) methods are used

Why is PSE so difficult? Some populations difficult to count, especially hidden, stigmatised Usual epidemiological methods such as national household or schools surveys usually do not work Definition problems: eg, IDU has “ever injected”, “injected in past month”, injected in past year”? Even more difficult for MARA and MARY as most epidemiological statistics & estimates are not disaggregated by age (or sex)

PSE Methods Variety of methods available, but most include:  Consensus/ Delphi  Multiplier methods Other potential methods RDS: Respondent Driven Sampling Social networks

Consensus/ Delphi Asks key informants to agree on number of people in X population Can be done at:  National level  All levels from local to national Local to national seems to generate most accurate numbers Should be triangulated with other methods

Multiplier methods Recommended by UNAIDS for population size estimation, eg for reporting on UNGASS IDU indicator Uses existing data source with survey data Benchmark: Reliable, regularly collected data: IDUs accessing health services, drug treatment, overdose deaths Multiplier: Survey of as broad a sample as possible (eg not just from treatment centres)

Multiplier formula X (population) = multiplier x benchmark Example: 1000 IDUs entered drug treatment in 2007 (benchmark) 10% of IDUs surveyed said they entered drug treatment in 2007 (multiplier)  X = 1000 x 10/100 (= 10)  X = 10,000 IDUs

Triangulation Single multiplier exercises tend to be inaccurate UNAIDS recommends using 3 at least separate processes, and averaging results to find a mean estimate: Eg: Different processes may give 10,000; 8000; 11,000. Mean = 9670

RDS/ Social networks RDS uses snowball sampling in specific methods to attempt to achieve highly representative sampling: was not developed as a PSE method! Mexico AIDS Conference: meta-analysis of 200 RDS papers found no evidence that RDS is useful in PSE Promoted by many agencies with little/no evidence of accuracy; costly, time-consuming Social networks PSE: new method, currently promoted by UNAIDS PSE workshops. May have value but requires evaluation, and to date apears costly and time-consuming

APMG Tajikistan project In Tajikistan, APMG is finalising a 5-month process for UNDP (GF PR) to:  Estimate national populations of IDUs and SW  Risk behaviour of IDUs & SW in 5 sites  Capacity of implementation agencies to scale up service delivery to IDUs & SW in these sites In addition, APMG is trying to tie this process to ongoing PSE for IDUs and SW as numbers change (especially locally as IDUs & SW are chased from 1 area by police activity or attracted to an area by availability of drugs or SW clients)

Tajikistan PSE methods Expert estimation (Delphi) at rayon level, combined at oblast and national levels Survey for risk assessment included multiplier question re use of narcological services in 2008 Benchmark: narcological statistics in 2008 Results presented to national consensus meeting (September 21) to agree final numbers

Lessons learned from Tajikistan Biggest error was carrying out risk assessment and PSE simultaneously: much larger sample sizes needed for risk assessment sampling meant expert estimation could not be carried out in all rayons nationally PSE can be relatively cheap and quick if done as a stand-alone activity

Lessons learned from Tajikistan Rayon-level estimation requires national/ oblast level staff to assist local officials to come to consensus Time should be included to allow rayon estimates to be considered at oblast level, then national meeting based on oblast estimates If this process used, could set up 6-monthly monitoring by asking rayons to consider increases/ decreases over the previous 6 months. Requires oblast/ national compilation

Tajikistan lessons re MARA PSE of MARA in Tajikistan could be accomplished using the same methods (with lessons learned) BUT Definition required Definition would need to be agreed with officials from various departments Definition to be explained at rayon level

Armenia PSE of MARA in Armenia carried out by APMG and MoH staff working on GF RCC proposal (2008) Had already estimated IDUs, MSM, SW, migrants, uniformed personnel MoH wanted to include specific programs for MARA but this required a statement about projected coverage after 6 years To calculate coverage figure, PSE was needed for MARA

Armenia Methods No time available for MARA PSE study Estimate figure was calculated as 5% of all adolescents in Armenia on the basis of household and school surveys that showed at least 5% of adolescents engaging in risky sexual behaviour or illicit (not necessarily injecting) drug use Population estimate was used in RCC proposal with a note that a full PSE would be carried out as part of the grant activities RCC was approved and will begin in late 09

Macedonia PSE in Macedonia will be carried out by National Public Health Institute (NPHI) for MoH (GF PR) NPHI has decided to combine PSE with risk behaviour survey and to use RDS (against our advice) APMG ’ s role will be to examine all documents (methods, instruments, sampling frames, data analysis & reports) to recommend corrections From this process, we will be able to learn lessons about use of RDS for PSE (probably by end 09)

Some further thoughts on Coverage APMG accepts WHO Universal Access definition: % of those who need an intervention who receive that intervention APMG sees 3 aspects: Reach, including regularity of reach. What % of the total population participate? Is this a sufficient proportion to prevent/ reverse/ treat the epidemic? Breadth: Spectrum of Services. Are interventions able to prevent/ reverse/ treat the epidemic? Quality: Are interventions sufficiently attractive and effective to meet their objectives?

Coverage Calculation APMG accepts WHO Universal Access definition: % of those who need an intervention who receive that intervention E.g., for needle-syringe programs, it appears that a percentage of IDUs in a specified area need to access NSP of adequate quality ON A REGULAR BASIS to prevent/ reverse a HIV epidemic among IDUs. WHO, UNODC and UNAIDS state that the % of IDUs who have been reached by NSP regularly (at least monthly for past 12 months) should be considered as: Low coverage: <20% Medium coverage : >20– <60% High coverage : >60%

Coverage questions WHO, UNODC and UNAIDS Target Setting Guide for IDUs include:  Proportion of IDUs regularly reached by NSP  Number of pharmacies/ 1000 IDUs  NSP sites/ 1000 IDUs  Number of syringes distributed per IDU per year  % of IDUs who have been reached by NSP regularly (at least monthly for past 12 months)  % of IDUs who have been reached by NSP in the past month

Coverage questions 2  Similar questions on proportion of IDUs in substitution treatment  Similar questions on proportion of IDUs in other drug dependence treatment  Similar questions on proportion of IDUs participating in VCT and know their results  Ratio of HIV+ IDUs receiving ART to non- IDU HIV+ receiving ART (relative to proportions of HIV+ population)  Questions on TB, hepatitis C, etc

Quality Generally, view is that quality should be measured by adherence to guidelines, e.g. target setting guide asks:  Percentage of NSP sites adhering to WHO guidelines on NSP  Percentage of NSP sites adhering to UNAIDS best practice recommendations for HIV prevention among IDUs  Percentage of occasions when clients access an NSP and receive IEC  Percentage of occasions when clients access an NSP and receive condoms In Russia, APMG is helping Russian Harm Reduction Network to develop NSP quality measurement and improvement processes based on the WHO/ UNAIDS/ UNODC Guide to Starting and Managing NSPs Manual plus instruments should be available in English & Russian early 2010

Coverage for other MARPs Similar processes now under way for MSM: APMG working with Amfar, UNDP & WHO on coverage calculation, targets & breadth of services WHO working on similar processes re SW MARA and MARY not yet really included in these global processes