1. L.J. Wei Harvard University Moving beyond the comfort zone in practicing translational statistics

3. Why are we staying in a “Comfort Zone” ? Generally following a fixed pattern for conducting studies Are we like lawyers? Avoiding delay of review processes?

4. What is the goal of a clinical study? Use efficient and reliable procedures to obtain robust, clinically interpretable results with respect to risk-benefit perspectives at the patient’s level.

5. What are the problems? The conventional way to conduct trials gives us fragmentary information Lack of clinically meaningful totality evidence Difficult to use the trial results for future patient’s management

6. A Few Methodology Issues 1. Estimation vs. testing P-value provides little clinical information about treatment effectiveness The size of the effects (efficacy and toxicity) matters Design using interval estimates is quite flexible Almost everything we want to know via testing, we can get from estimation

7. TREAT study for EPO CV safety If we follow the patients up to 48 month, the control arm's average stroke-free time is 46.9 months and the Darb arm's is 46 months. The difference is 0.9 month (0.95 CI: 0.4m, 1.4m) with p<0.001 (very significant).

8. 2. How do we define a primary endpoint with multiple outcomes? What is current practice? Component/composite analyses Efficacy and toxicity (how to connect them together?) Disease burden measure? Competing risks problem? Informative dropout?

9. Example : Beta-Blocker Evaluation of Survival (BEST) Trial (NEJM, 2001) Study Bucindolol vs. placebo patients with advanced chronic heart failure -- n = 2707 Average follow-up: 2 years Primary endpoint: overall survival Hazard ratio for death = 0.90 (p-value = 0.1)

10. BEST Trial

11. Possible solutions? Using the patient’s disease burden or progression information during the entire followup to define the “responder” Creating more than one response categories: ordinal categorical response Brian Claggett’s thesis paper

12. BEST Example: 8 Categories 1: No events 2: Alive, non-HF hospitalization only 3: Alive, 1 HF hosp. 4: Alive, >1 HF hosp. 5: Late non-CV death (>12 months) 6: Late CV death (>12 months) 7: Early non-CV death (<12 months) 8: Early CV death (<12 months)

13. 3. How to handle dropouts or competing risks? LOCF? BOCF? MMRM (model based) Pattern mixture model (cannot handle non-random missing) Using responder analysis with different ways to define informative dropouts for sensitivity analysis

14. 4. Analysis of Covariance Compare two treatments with baseline adjustments via regression models For nonlinear model, different adjustments may lead to incoherent results The inadequacy of the Cox ANCOVA

15. Possible solutions? Using the augmentation method by Tsiatis et al; Tian et al. No need to pre-specify the baseline covariates, but a set of potential covariates in the adjustment process

16. 5. Data monitoring Heavily utilizing p-value or conditional power A low conditional power may indicate that the sample size is too small or there is no real treatment difference Using estimation and prediction for monitoring?

17. 6. Stratified medicine (personalized medicine)? A negative trial does not mean the treatment is no good for anyone A positive trial does not mean it works for everyone The usual subgroup analysis is not adequate to address this issue Need a built-in pre-specified procedure for identifying patients who benefit from treatment

18. 7. Identify patients who respond the new therapy (predictive enrichment)

19. 8. How to monitor safety? What is the conventional way? Component-wise tabulation or analysis? No information about multiple AE events at the patient level Graphical method?

21. 9. Quantifying treatment contrast (difference)? Should be model-free parameter Using difference of means, median, etc. For censored data, using a constant hazard ratio (heavily model-based)? Model-based measure is difficult to interpret or validate

22. Issues for the hazard ratio estimate Hazard ratio estimate is routinely used for designing, monitoring and analyzing clinical studies in survival analysis

23. Model Free Parameter for Treatment Contrast * Considering a two treatment comparison study in “survival analysis” * How do we quantify the treatment difference? Median failure time (may not be estimable); t-year survival rate (not an overall measure)? A constant hazard ratio over time with the log-rank test

24. Eastern Cooperative Oncology Group E4A03 trial to compare low- and high-dose dexamethasone for naïve patients with multiple myeloma The primary endpoint is the survival time n=445 The trial stopped early at the second interim analysis; the low dose was superior. Patients on high-dose arm were then received low-dose and follow-up for overall survival were continued.

25. A Cancer Study Example Group 1 Group 2

26. The proportional hazards assumption is not valid The PH estimator is estimating a quantity which cannot be interpreted and, worse, depends on the study-specific censoring distributions Any model-based treatment contrast has such issues (need a model-free parameter) The logrank test is not powerful

27. Conventional analysis: Log-rank test: p=0.47 Hazard Ratio: HR=0.87 (0.60, 1.27)

28. What is the alternative way for survival analysis? Using the area under the curve of Kaplan-Meier estimate up to a fixed time point Restricted mean survival time Model-free and a global measure of efficacy Can be estimated even under heavy censoring

29. Cancer Study Example Restricted Mean (up to 40 months): 35.4 months vs. 33.3 months Δ = 2.1 (0.1, 4.2) months; p=0.04 Ratio of Survival time = 35.4/33.3 = 1.06 (1.00, 1.13) Ratio of time lost = 6.7/4.6 = 1.46 (1.02, 2.13)

30. 10. Post-marketing/safety studies ? It is not appropriate to use an event driven procedure to conduct a safety study. The event rate is low, the exposure time matters Requires lot of resources (large or long-term study) Meta analysis; observational studies

31. CV safety study for anti-diabetes drugs Event driven studies, that is, we need to have a pre-specified # of events so the resulting confidence interval for the treatment difference is “narrow” For example, the upper bound of 95% confidence interval is less than 1.3

32. The EXAMINE trial (alogliptin) NEJM, October 3, 2013

33. RMST (24 months): Placebo 21.9 (21.7, 22.2) Alogliptin 22.0 (21.8, 22.3) Difference -0.08 (-0.39, 0.24) Ratio 1.00 (0.98, 1.01) RMST (30 months): Placebo 27.1 (26.7, 27.4) Alogliptin 27.2 (26.9, 27.5) Difference -0.12 (-0.56, 0.33) Ratio 1.00 (0.98, 1.01)

34. 34 What if a smaller study? 95% confidence intervals for various measures All data25%20%15% N=16492N=4123N=3298N=2427 Hazard Ratio (0.89, 1.12)(0.80, 1.26)(0.78, 1.28)(0.76, 1.36) Difference in event rate at Day 900 [%] (-1.2, 0.9)(-2.3, 2.0)(-2.6, 2.2)(-2.9, 2.6) Difference in RMST at Day 900 [days] (-5, 4)(-9, 9)(-11, 10)(-12, 12)

35. 11. Meta analysis for safety issues

36. Nissen and Wolski (2007) performed a meta analysis to examine whether Rosiglitazone (Avandia, GSK), a drug for treating type 2 diabetes mellitus, significantly increases the risk of MI or CVD related death.

37. Example Effect of Rosiglitazone on MI or CVD Deaths Avandia was introduced in 1999 and is widely used as monotherapy or in fixed-dose combinations with either Avandamet or Avandaryl. The original approval of Avandia was based on its ability in reducing blood glucose and glycated hemoglobin levels. Initial studies were not adequately powered to determine the effects of this agent on micro- or macro- vascular complications of diabetes, including cardiovascular morbidity and mortality.

38. Example Effect of Rosiglitazone on MI or CVD Deaths However, the effect of any anti-diabetic therapy on cardiovascular outcomes is particularly important because more than 65% of deaths in patients with diabetes are from cardiovascular causes. Of 116 screened studies, 48 satisfied the inclusion criteria for the analysis proposed in Nissen and Wolski (2007).  42 studies were reported in Nissen and Wolski (2007), the remaining 6 studies have zero MI or CVD death  10 studies with zero MI events  25 studies with zero CVD related deaths

39.  Event Rates from 0% to 2.70% for MI  Event Rates from 0% to 1.75% for CVD Death

40. Log Odds Ratio MI 95% CI: (1.03, 1.98); p-value = 0.03 (in favor of the control) CVD Death 95% CI: (0.98, 2.74); p-value = 0.06 Log Odds Ratio ??????

41. Questions Rare events? How to utilize studies with 0/0 events? Validity of asymptotic inference? Exact inference? Choice of effect measure? Between Study Heterogeneity? Common treatment effect or study specific treatment effect? The number of studies not large?

42. MI 95% CI: (-0.08, 0.38)% P-value = 0.27 95% CI: (0.02, 0.42)% P-value = 0.03 Exact InferenceAsymptotic Inference

43. CVD Death 95% CI: (-0.13, 0.23)% P-value = 0.83 95% CI: (0.00, 0.31)% P-value = 0.05 Exact InferenceAsymptotic Inference

44. Summary Could we modify our statistical training? Teaching young generations “how, where and what to learn” Learning from doing a project with mentoring? Could we have a coherent approach from the beginning to the end for a research project? George Box: Instead of figuring out the optimal solution to a wrong problem, try to get A solution to a right problem. Asking ourselves “What is the question?”