1. A consideration of potential implications on health economic analyses of using adaptive designs
Sue Todd
Department of Mathematics and Statistics
University of Reading, UK
ICTMC/SCT Conference May 2017

2. Outline Types of adaptive designs Benefits and limitations
Approaches to economic evaluation
Consequences for economic evaluation of adaptive designs
Conclusions
ICTMC/SCT Conference May 2017

3. Types of adaptive designs
DEFINITION: A trial which incorporates interim analyses where data are examined in order to make a decision concerning the rest of the trial’s conduct
Can incorporate a wide range of procedures, for example in the confirmatory setting may see
Sample size reassessment
Group sequential tests
Flexible adaptive designs
Response adaptive designs
ICTMC/SCT Conference May 2017

4. Use of adaptive designs
Use of adaptive designs has grown due to the influence of ETHICS ECONOMICS ENTHUSIASM But there are issues with implementation surrounding ADMINISTRATION APATHY ANALYSIS
ICTMC/SCT Conference May 2017

5. Practical issues with implementation
ADMINISTRATION Timing of interim analyses Ensuring interim data quality How to budget appropriately for costs and people’s time APATHY More complex to design More complicated to organise More time involved in data collection and processing
ICTMC/SCT Conference May 2017

6. Statistical issues with implementation
ANALYSIS Conventional P-values may be inaccurate. Conventionally calculated point estimates may be biased and confidence intervals may have incorrect coverage (depending on type of AD) Although the meaning and interpretation of these traditional data summaries remain as for standard conventional designs, they may require alternative calculation in the setting of adaptive designs
ICTMC/SCT Conference May 2017

7. Statistical issues with implementation
Example bias plot (group sequential design)
ICTMC/SCT Conference May 2017

8. Statistical issues with implementation
P-values: E.g. Whitehead (2000); Bauer & Köhne (1994)
Estimates / CIs - primary endpoint literature: E.g. Whitehead (1986a); Pocock & Hughes (1989); Todd et al. (1996); Emerson & Kittelson (1997); Shen (2001); Stallard et al. (2008); Posch et al. (2005); Bauer et al. (2009); Kimani et al. (2013, 2015)
Estimates / CIs - secondary endpoint literature: E.g. Whitehead (1986b); Hall & Yakir (2000); Whitehead et al. (2000)
Software exists E.g. EaSt; ADDPLAN; RCTdesign
ICTMC/SCT Conference May 2017

9. Statistical issues with implementation
Issues with analysis are therefore well documented, but in practice, are often not considered
“Many trials which are terminated at an interim analysis are reported with analyses which take no statistical account of the interim analysis … Most authors present naïve estimates of the treatment effect, but these are known to be biased ... The analyst should also consider the impact of interim analyses on the analysis of secondary variables which are correlated with the primary variable assessed during interim analyses”
Facey & Lewis (1998)
“Of the 46 RCTs that were stopped early, only 3 (7%) reported the use of appropriate statistical methods for bias correction of point estimates of the intervention effect, and associated CIs and P-values”
Stevely et al. (2015)
ICTMC/SCT Conference May 2017

10. Consequences for economic evaluation
How might the above practical and statistical issues with implementing adaptive designs impact on health economic evaluation?
In short, it depends on what economic evaluation is being conducted...
... and when
ICTMC/SCT Conference May 2017

11. Types of economic evaluation
DEFINITION: “Comparative analysis of alternative courses of action in terms of both their costs and consequences”
Drummond et al. (2015)
Incorporates a wide range of approaches within the broad categories of
Cost-effectiveness analysis
Value of information analysis
ICTMC/SCT Conference May 2017

12. Types of economic evaluation
Cost information comes from a range of sources e.g. the Personal Social Services Research Unit document of unit costs for a range of health and social care services in England and the British National Formularly for drug costs
Consequences are measured using a health related utility such as quality adjusted life years(QALY)
Sometimes summary measures are used to compare costs/consequences. Alternatively, decision analytic models may be developed
ICTMC/SCT Conference May 2017

13. Economic evaluation at the end of an adaptive trial
Economic evaluations are often ‘added on’ to existing trials. They are not separately powered. If a trial stops early, will have even less information with which to carry out the evaluation
If use is made of the treatment effect relating to the primary endpoint in a decision analytic model, then bias in the estimate may influence the economic model
If calculate a secondary economic endpoint e.g. QALY, it is likely to be biased as correlated with the primary endpoint
ICTMC/SCT Conference May 2017

14. Economic evaluation alongside clinical evaluation in an adaptive trial
Could consider economic evaluation informally during an adaptive design (e.g. In the context of a flexible adaptive design). Practicalities of conducting interims must then be considered such as timing and data required
Could consider formally monitoring measures of clinical and cost effectiveness at each interim analysis. This leads to bivariate monitoring. Previous literature in the context of efficacy and safety monitoring E.g. Jennison & Turnbull (1993); Cook & Farewell (1994) and in the case of two endpoints more generally E.g. Todd (1999; 2003)
ICTMC/SCT Conference May 2017

15. Conclusions
Adaptive design methodology and health economic analyses have evolved largely independently, yet if more adaptive designs are going to be used and health economic analyses are going to become even more embedded in trials then the interface between the two must be explored
Research to this effect has been initiated
Practical implementation is also developing
ICTMC/SCT Conference May 2017