2. Conflict of Interest
I have acted as a Consultant on an education workshop organised by Gilead Sciences

3. Acknowledgements
David Glidden (UCSF)

4. Traditional outcome measures: trials with a no-treatment arm
Group
No. infections
Follow-up (PY)
Incidence (per 100 PY)
90% CI
TDF-FTC 3
243
1.2
0.4–2.9
No PrEP 20
222
9.0
6.1–12.8
Incidence rate ratio (IRR) = 1.2/9.0 = 0.14
Effectiveness = ( )/9.0 = 1–1.2/9.0 = 86%
Data from PROUD trial (Lancet, 2015)

5. Trials of experimental PrEP agents
Unlikely to include a placebo arm because of ethical considerations
Instead, non-inferiority design with oral TDF-FTC as an active control arm
TDF-FTC highly effective => expect few HIV endpoints
Variance of IRR is inversely related to the observed number of endpoints

6. Two hypothetical studies
Study
Incidence rate*
IRR
(90% CI)
TDF-FTC
PrEP X A
1 (20)
1.00 (0.56,1.77) B
2 (40)
1.00 (0.66,1.48)
* per 100 PY (number of endpoints)
2000 PY observation per arm
IRR = incidence rate ratio

7. Add in background (placebo) HIV incidence in the trial population
Study
Incidence rate*
IRR
(90% CI)
Placebo
TDF-FTC
PrEP X A 2
1 (20)
1.00 (0.56,1.77) B
2 (40)
1.00 (0.66,1.48)
* per 100 PY (number of endpoints)
2000 PY observation per arm
IRR = incidence rate ratio

8. Ratio of averted infections (RAI)
RAI= λ P − λ E λ P − λ C
λ = incidence rate
P= placebo arm, E= experimental arm, C = control arm
Measures the proportion of infections that would be averted by using the experimental drug rather than the control drug

9. Two hypothetical studies
Study
Incidence rate*
IRR
(90% CI)
RAI
Placebo
TDF-FTC
PrEP X A 2
1 (20)
1.00 (0.56,1.77)
1.00 (0.59,1.68) B
2 (40)
1.00 (0.66,1.48)
Not defined
* per 100 PY (number of endpoints)
2000 PY observation per arm
IRR = incidence rate ratio, RAI = ratio of averted infections

10. Two more hypothetical studies
Study
Incidence rate*
IRR
(90% CI)
RAI
Placebo
TDF-FTC
PrEP X C 5
1 (20)
1.00 (0.56,1.77) D
2 (40)
1.00 (0.66,1.48)
* per 100 PY (number of endpoints)
2000 PY observation per arm
IRR = incidence rate ratio, RAI = ratio of averted infections

11. Two more hypothetical studies
Study
Incidence rate*
IRR
(90% CI)
RAI
Placebo
TDF-FTC
PrEP X C 5
1 (20)
1.00 (0.56,1.77)
1.00 (0.88,1.14) D
2 (40)
1.00 (0.66,1.48)
1.00
(0.78, 1.28)
* per 100 PY (number of endpoints)
2000 PY observation per arm
IRR = incidence rate ratio, RAI = ratio of averted infections

12. Three arm “gold standard” trials
Trial arm
Incidence rate*
IRR
(90% CI)
RAI
Placebo
1.99 (52) –
TDF-FTC
0.48 (21)
TDF
0.71 (31)
1.481 (0.85,2.57)
0.85
(0.69,1.04)
0.85= 1.99− −0.48
* per 100 PY (number of events)
1. Relative to TDF-FTC
Data from Partners-PrEP (Baeten et al. LID, 2014)

13. Two arm trials
Most future trials will not include a placebo or no-treatment arm
RAI requires an estimate or informed guess of the placebo incidence we would have observed – the counterfactual value

14. Two arm trials Trial arm Incidence rate* IRR (90% CI) RAI Placebo
1.99 (52) –
TDF-FTC
0.48 (21)
TDF
0.71 (31)
1.481 (0.85,2.57)
???
* per 100 PY (number of events)
1. Relative to TDF-FTC
Data from Partners-PrEP (Baeten et al. LID, 2014)

15. Counterfactual placebo incidence

16. RAI: alternative expression
RAI= λ C − λ E + θ C λ E θ C λ C
θ C = effectiveness of the active control drug in the current trial

17. Counterfactual control drug effectiveness

18. The big tough question
How to estimate the counterfactual parameter? Either
placebo incidence
effectiveness of control drug

19. Estimating counterfactual parameter
Placebo incidence
Control drug effectiveness
Run-in period without active treatment
Meta-analysis of previous trials comparing active control to placebo
Infer from tests for recent infection on baseline samples
Measure adherence within trial and infer effectiveness from PK/PD models or using meta-regression
Recent studies in the same geographic region in population with similar characteristics
Estimates from epidemiological surveillance systems
Infer from ecological association between incidence of HIV and other STIs

20. Conclusions (1)
In a two-arm active-control trial, the rate ratio (experimental versus control) cannot be considered in isolation
Essential to consider counterfactual placebo incidence and/or effectiveness of control drug
Can either make implicit or explicit (quantitative) assumptions

21. Conclusions (2)
The Ratio of Averted Infections is a useful measure of effectiveness with a direct clinical interpretation
Natural way of defining “preservation of effect” in non-inferiority trials
Preliminary work suggests substantial sample size saving compared with current approach: can decrease PYFU by at least 30-50%

22. Extra slides

23. Counterfactual placebo incidence

24. Counterfactual control drug effectiveness

25. Estimating counterfactual placebo incidence
Approach
Comments
Run-in period without active treatment
Ethical issues. May over-estimate incidence due to variability between individuals in risk of infection
Infer from tests for recent infection on baseline samples.
Number of recent infections usually small and estimates therefore imprecise.
Recent studies or trials in the same geographic region in population with similar characteristics
Unreliable if HIV incidence changing rapidly
Estimates from epidemiological surveillance systems
Under-estimate incidence if PrEP attracts particularly high-risk individuals
Measure STI incidence within trial; calibrate from ecological association between incidence of HIV and other STIs
Ecological association not strong. HIV incidence less stable than that of common STIs such as gonorrhoea.

26. Estimating counterfactual control arm effectiveness
Approach
Comments
Meta-analysis of previous trials comparing active control to placebo
Hinges on “constancy” assumption i.e. that effectiveness can be validly extrapolated from meta-analysis (studies often highly heterogeneous)
Measure adherence within trial and infer effectiveness from PK/PD models or using meta-regression
Elicited adherence often inaccurate. Drug levels more reliable but expensive to collect samples.

27. Approach in current active-control trials
Non-inferiority designs
Make assumptions about control arm effectiveness
Inference based on the measure
log λ P − log λ E log λ P − log λ C = log ( λ P / λ E ) log ( λ P / λ C )