1. Analysing RWE for HTA: Challenges, methods and critique
Rita Faria
Research Fellow
Centre for Health Economics, University of York

2. The decision Information required The ideal study The reality
What is the cost-effective treatment?
The decision
Effect of treatment
In the patients seen in clinical practice
Over the short- and long-term
Relative to other treatments available.
Information required
RCT or meta-analysis of RCTs
In the population of interest
Over the relevant time horizon
Compared with the other treatment options
The ideal study
RCTs may be unfeasible.
RCT may be in a selected subgroup, over a short time horizon and using a subset of comparators.
The reality
Decision modelling
Real world evidence: non-randomised studies, observational data, administrative databases
Potential solutions
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

3. Challenges 2013 NICE Methods guide:
3.3.6: “The potential biases of observational studies should be identified, and ideally quantified and adjusted for. When possible, more than one independent source of such evidence should be examined to gain some insight into the validity of any conclusions.”
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

4. Association ≠ Causation
Is shaving good for your health?
HR for stroke=1.90 (95%CI )
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

5. Association, causation & confounders
Stroke
Infrequent shaving
Confounder
 Selection bias X
Risk factor for stroke
Infrequent shaving may be a confounding variable.
Conditional on X, there may be no association between stroke and infrequent shaving.
RWE is at high risk of selection bias.
Selection bias arises from differences in the characteristics that have an independent influence on the outcome between the individuals in the treated and the control groups.
TSD17 p13.
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

6. NICE DSU TSD 17 on non-RCT data
What to do in practice?
How to know it is right?
Methods & algorithm for selection
Does treatment selection ONLY depends on the observed data?
OVERLAP: Are treatment groups comparable?
QuEENS checklist for critical appraisal.
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

7. Objective: obtain the true (unbiased) effect of treatment.
Methods
Selection on observables
Regression adjustment
Inverse probability weighting
Matching (±regression; w/o propensity score)
Regression on propensity score
Doubly robust
Objective: obtain the true (unbiased) effect of treatment.
Selection on unobservables
Instrumental variables
Natural experiments
Differences-in-Differences
Regression discontinuity
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

8. Critical appraisal: QuEENS checklist
Selection on observables
General issues
Robustness of results
Consistency across studies
Statistic validity
Valid assumptions
Matching methods
Methods using propensity score
Instrumental variable methods
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.

9. Final recommendations
Remember the objective: obtain the true (unbiased) effect of treatment.
Data
Assumptions
Methods
Sensitivity analysis
Reporting
Analysing RWE for HTA: Challenges, methods and critique.
Rita Faria, Research Fellow, University of York.