1. Innovative methods in assessments / surveys for challenging settings
David Prieto-Merino
LSHTM
Farr Institute, UCL
Catholic University of Murcia (UCAM)

2. The problems
Most study designs, observational and interventional, assume some ideal conditions of access to data and population.
But in many of the challenging settings we might not have easy access to an unstable / moving population and no data at all. The generation of evidence in challenging settings is not easy, often RCTs are not possible.
Methods need to be adapted. We have to make the most with the resources at hand...
So this talk is not really on “innovative methods”, but just on trying to use the old methods to face new challenges…

3. Two examples
Estimation of the prevalence of diabetes in a region in armed conflict where researchers have limited access to most of the population.
Generating evidence: Estimation of the effect of an intervention in a refugee camp where a randomised controlled trial (RCT) is not possible.

4. Estimation of prevalence with difficult access to the population
The problem: We want to estimate the prevalence of Diabetes in a certain region but we don’t have the possibility of reaching out to survey the population in that region. We do have a health facility (HF) and we can capture patients that reach HF.
The approach: Bayesian estimation of the population prevalence by comparing the relative frequency of different diseases among the patients that visit H in a particular time window (and making some assumptions).
Assume we know roughly the population size (N).

5. We want to estimate the prevalence of diabetes in the population P1 = N1/ N. Ideally we would do a random survey.
Population (N)
Health Facility (HF)
Diabetes N1
Survey

6. But we cannot access the population and can only study those that come to the HF
N1 = Number of cases of disease 1 in population
K1 = Probability of a case of 1 of visiting the HF
H1 = Number of cases of 1 in the HF
N1 * K1 = H1
Population (N)
Health Facility (HF)
Diabetes N1
N1 * K1 H1
But we might not know K1 !!

7. Consider another disease and calculate the ratio of patients in HF of the two diseases
Population
Health Facility (HF)
Diabetes N1
N1 * K1 H1
disease2 N2
N2 * K2 H2

8. Ratio between diseases in the population and in the HFK1 H1 N2 K2 H2 K2 H1 N1 N2 H2 K1

9. Estimation of prevalence with difficult access to the population
Diabetes N1 K2
disease2 N2 H1
POPULATION N
POPULATION N H2 K1
Prevalence of diabetes P(D)
Ratio diabetes / disease2
in HF
Ratio of prob of visits
Prevalence of disease2

10. Guessing unknown parameters
The solution requires us to know (or guess) some quantities:
(K2 / K1): We have to ask the question, how more likely is a patient with disease2 to visit the HF that a patient with diabetes in this time period? (can you assume they are the same?)
(N2 / N): Prevalence of disease2 in the population. The key is to choose a disease that you know well.
Even if you don’t know these parameters exactly you can make an educated guess. Or you can put a prior distribution of belief on some range and get a posterior distribution for Prevalence of diabetes with an MCMC sampling method.

11. With knowledge of the prevalence of a disease in subgroups of the population we could estimate different use of health facilities
Population
Health Facility (HF)
Group1 N1
N1 * K1 H1
Group2 N2
N2 * K2 H2

12. Estimate the ratio of access to HF (K1 / K2) between cases of different subgroups of the population

13. Advantages and limitations of the method
We don’t need to move from HF to estimate prevalence in the population.
We can stratify estimations by patient characteristics (age, sex, residence) we will need to define different K for each stratum.
We can use information from nearby health facilities or similar populations as prior information in the analysis
We can do continuous monitoring of the prevalence over time without having to repeat expensive surveys (gets more complicated with double counting of patients).
If I had reasonable knowledge of the prevalence of the population P(D), I could turn around the problem and estimate the use of health facilities by patients.

14. Generating evidence: Estimation of effect of intervention without randomisation and proper follow up.
Many interventions have shown their efficacy in ideal settings, but their effectiveness in challenging settings is not always known. But often a RCT is not possible in those settings.
Example: Estimate the effect on the adherence to a drug of a different presentation in a setting where an RCT is not possible.
The approach: regression of interrupted time series of the monthly percentage of adherence outcomes (self-reported adherence and SBP) to estimate the change of intercept and/or trend accounting for the imprecision in the estimations with a Bayesian model.

15. Time series of outcome in two time periods before an after intervention

16. Estimate the changes in the time series before and after the intervention
Estimate sudden change in level
Estimate change in slope

17. Use the changes in the time series of a control group with no intervention as a control
The control group (green) already shows a change in the slope. Discount this from the effect change of slope in the intervention group (black)

18. Advantages and limitations of the method
Comparing periods within a group we control for all characteristic of the group that remain constant.
Comparing changes with the control group helps to discount time effects independent from the intervention.
Using group level data does not require follow up of individuals that might be difficult in challenging settings.
Bayesian methods can be used to incorporate external information that we might have about the intervention or the settings

19. Conclusions…
In challenging settings we don’t have the luxury of ideal conditions of the standard study designs. We need to adapt the methods, make the most with the resources at hand, be creative…
New technologines (eHealth, mHealth, drones) will provide a wealth of data but will also pose new challenges for design and analysis.
Machine learning is not going to solve the problem (in case you were wondering). Careful thinking about models and interpretation is still a human privilege.
We need to teach statistical analysis and epidemiology in a flexible way, not as a series of recipes. Rules are there to be broken… (within reason)

20. Thanks for listening Any questions?
“Imagination is more important than knowledge. For knowledge is limited to all we now know and understand, while imagination embraces the entire world, and all there ever will be to know and understand.”
Albert Einstein