1. Statistics in Drug Regulation: The Next 10 Years Thomas Permutt Director, Division of Biometrics II Center for Drug Evaluation and Research The views expressed are those of the speaker and not necessarily of FDA.

2. Statutory Standards Substantial evidence of efficacy All tests reasonably applicable for safety Balance not explicit, but history clear

3. Risk/Benefit Formerly: –Very good evidence about direction of mean treatment effect Too good? No. –Adverse events: Common: statistical but unimportant Rare: nonstatistical but important

4. What’s New? Rofecoxib Rosiglitazone LABA

5. Rofecoxib Heart attacks Large outcome trial –which was trial in new indication Now need outcome studies for COX-2 and maybe nonselective

6. Rosiglitazone Nissin meta-analysis We do meta-analysis You do meta-analysis You do outcome trial, maybe

7. Meta-analysis Hard Nonstatistical Statistical Both different in regulatory setting

8. Meta-analysis: Nonstatistical Better information, but … Doesn’t fit usual protocol-driven regulatory framework, either Do it anyway, but … Nobody will believe you (or us), so … ? –sensitivity analysis important

9. Meta-analysis: Statistical Fixed vs. random effects –doesn’t matter much for global null, but –this doesn’t apply to noninferiority Attributable vs. relative risk –relative risk “stable” across settings different length of study, at least –but attributable risk is what matters –what about zeroes Nissin to Congress: “no information”

10. What triggers this? “Signal” –Class effects –Someone else’s meta-analysis For diabetes, everything For COX-2, probably everything –other COX?

11. LABA Believed to cause death –not “side effect,” death from asthma Effect mostly “seen” without steroid So, with steroid?

12. With Steroid, Show What? Noninferior to nothing? –i.e., combination therapy vs. steroid Noninferior to realistic alternative? –e.g., increased dose of steroid –why not superior? because of benefit Interaction with steroid? –i.e., already “know” without steroid: Is with different? –maybe can’t do without steroid anyway

13. Noninferiority Margins Not “1.3” –COX-2 –diabetes –asthma! Risk-benefit –for direct measures –for surrogates

14. Surrogate Everyone likes “hard” endpoints but … They mostly don’t measure benefit They are correlated with benefit

15. Correlation with Benefit Does drug produce benefit or modify correlation? (anti-arrythmics, maybe glitazones) Qualitative validation hard enough Quantify benefit very hard –estimate strength of relationship –and hope it holds

16. Patient-Reported Outcomes Hard endpoints are “nice” but they don’t measure utility PRO are squishy but relevant Psychometrics is not evil (now)

17. Linking Risk and Benefit Expected utility –mean efficacy outcome –incidence of AE –(mean effect) X (goodness) – (AE rate) X (badness) Other formulas are incorrect –provided utility is linear wrt effect

18. It Isn’t Linear For surrogates For PROs

19. Utility Calculations: Example 50% symptom-free 50% intolerable adverse events Good or bad? –How bad were symptoms? –How bad were adverse events?

20. Two Drugs Women have efficacy Men have adverse events Women have efficacy Women have adverse events Men have nothing

21. Two Drugs Women have efficacy Men have adverse events Useful drug –provided AEs are reversible Women have efficacy Women have adverse events Men have nothing Useless drug “Expected utility” does not distinguish!

22. Why Doesn’t Expectation Work? Because you don’t really measure benefit –benefit at timepoint (or average over time) is surrogate for long-term benefit –don’t get long-term benefit if you drop out –LOCF makes it worse “Mixing up” safety and efficacy is … –not illegal –not even stupid –“individualized medicine” dropout is good biomarker!