1. Design & Interpretation of Randomized Trials: A Clinician’s Perspective Francis KL Chan Department of Medicine & Therapeutics CUHK

2. Common problems of RCTs  Originality  Hypothesis  Allocation concealment & randomization  Evaluation of baseline data  “Intention-to-treat” analysis  Subgroup analysis

3. Is the study original?  Ground breaking research?  Does this work add to the literature in any way?  Bigger, longer?  More rigorous methodology?  Results add to a meta-analysis of previous studies?  Different population (age, sex, ethnic groups)?

4. Hypothesis & End point Many RCTs did not explicitly state their study hypotheses “The aim of this study was to compare the efficacy of a new treatment with the standard treatment…” Hypothesis 1: Treatment A is superior to the standard treatment Hypothesis 2: Treatment A is equivalent to the standard treatment

5. Hypothesis? Sample size estimation None!

6. Failure to detect a difference = Equivalence?

7. Superiority Trial The new treatment (µ N ) is superior to the standard treatment (µ S ) if the difference exceeds by a clinically important amount (  ). Test hypothesis (H): µ N - µ S > 

8. Equivalence trial The new treatment is equivalent to the standard treatment if the maximal allowable difference does not exceed by a clinically important amount.

9. Equivalence trial -  0 +  Equivalent Difference Not equivalent Favors new treatment Not equivalent Favors standard treatment New agent is not inferior to the standard

10. Assume non-inferiority if the lower limit of 95% CI is less than –5%, N=904 per group!

11. Allocation concealment & randomization  Concealment of allocation (investigators and patients not knowing the assigned treatment before randomization)  Was treatment assigned by an independent staff?  What was the method of allocation concealment?  contact with central office  blinded packages  sealed (opaque) envelopes

12. Allocation concealment

13. Comparison of baseline data Chan et al. Lancet 1997 Does P>0.05 indicate comparability of treatment groups?

14. Baseline data AzathioprinePlacebo Mean age54.754.9 Serum bilirubin (  mol/L) 37.230.9 Stage I disease %1412 Stage II disease %4443 Stage III disease %15 Stage IV disease %2730 Christensen et al. Gastro 1985 Effect of azathioprine on the survival of patients with primary biliary cirrhosis

15. Baseline data AzathioprinePlacebo Mean age54.754.9 Serum bilirubin (  mol/L) 37.230.9 Stage I disease %1412 Stage II disease %4443 Stage III disease %15 Stage IV disease %2730 Christensen et al. Gastro 1985 Effect of azathioprine on the survival of patients with primary biliary cirrhosis

16. Unadjusted P=0.10 Adjusted for bilirubin P=0.01

17. P=0.04 P=0.02 Columbus Investigators. NEJM 1997

18.  Significant imbalance may not affect outcome Comparison of baseline data  Non-significant imbalance may affect outcome  Significance tests for baseline differences are inappropriate.

19. Significance tests for baseline differences Chan et al. Lancet 1997 INAPPROPRIATE

20.  Significant imbalance may not affect outcome Comparison of baseline data  Non-significant imbalance may affect outcome  Significance tests for baseline differences are inappropriate.  Table of baseline data should focus on factors affecting outcome.

21. 45 baseline factors!

22.  Significant imbalance may not affect outcome Comparison of baseline data  Non-significant imbalance may affect outcome  Significance tests for baseline differences are inappropriate.  Table of baseline data should focus on factors affecting outcome.  Analysis adjusted for baseline factors that are known to strongly influence the outcome (Covariate-adjusted analysis).  Analysis of covariance for a quantitative outcome  Logistic regression for a binary response  Cox’s-proportional hazard model for time-to-event data

23. “Intention-To-Treat” Analysis “…results were analyzed according to the ITT principle.” Question: How were missing outcomes/ protocol violators handled in the so called “ITT” analysis?

24. “Intention-To-Treat” Analysis Endpt Savage et al. NEJM 1997

25.  Minimize missing response on primary outcome Recommendations for ITT Analysis  Follow up subjects who withdraw early  Investigate & report the effect of missing response  Report all deviations and missing response

26. Subgroup Analysis Randomised trial of home-based psychosocial nursing intervention for patients recovering from myocardial infarction. Frasure-Smith et al. Lancet 1997 “…The poor overall outcome for women, and the possible harmful impact of the intervention on women, underlie the need for…”

27. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87. Subgroup Analysis SteroidPlaceboP Pre-ecclampsia 21.2% (7/33) 27.3% (9/33) 0.57 No pre-ecclampsia 7.9% (21/267) 14.1% (37/262) 0.021

28. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87. Subgroup Analysis SteroidPlaceboP Pre-ecclampsia 21.2% (7/33) 27.3% (9/33) 0.57 No pre-ecclampsia 7.9% (21/267) 14.1% (37/262) 0.021 Difference 6.1%

29. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276-87. Subgroup Analysis SteroidPlaceboP Pre-ecclampsia 21.2% (7/33) 27.3% (9/33) 0.57 No pre-ecclampsia 7.9% (21/267) 14.1% (37/262) 0.021 Difference 6.1% Difference 6.2% P value depends on effect size & SE

30. Evaluation of Subgroup Analysis Tests of interaction (assess whether a treatment effect differs between subgroups) rather than subgroup P values Diff in Subgroup A – Diff in Subgroup B SE of the above Diff Z =

31. Trial of vitamin D supplements in pregnancy to prevent infant hypocalcemia. BMJ 1980;281:11-4. Interaction Test Difference = 0.42Difference = 0.15 0.42 – 0.15 = 0.27 SE of this Diff = 0.22 Z = Diff / SE = 1.23 P = 0.2 No evidence that the effect of Vit D is different between bottle-fed and breast-fed infants

32. General points regarding subgroup analysis  Emphasis should remain on overall comparison  More convincing if confined to a limited number of pre-specified subgroup hypothesis  Rely on interaction tests, not P values  View subgroup findings as exploratory (to be confirmed in subsequent trials)