1. THE ROLE OF SUBGROUPS IN CLINICAL TRIALS Ralph B. D’Agostino, Sr., PhD Boston University September 13, 2005

2. OUTLINE Clinical trial scenarios for subgroup (subset) analysis Illustrations of possible outcomes –GOOD and PROBLEMATIC PRACTICES Role of formal Interaction Test Statistical properties of analyses Closing Comments

3. POSSIBLE SCENARIOS FOR SUBSET ANALYSIS Primary Hypothesis is for an overall statistically significant effect (All data combined) If Yes –Subgroups examined for consistency –Special subgroups examined for additional effect –Latter two activities are secondary analyses Primary Hypothesis anticipates subgroup effect –Subgroups examined for equal effect to justify pooling and an overall analysis

4. Primary Hypothesis is for an overall statistically significant effect Primary hypothesis is that two treatments are significantly different If yes to above, concern then is that this should be consistently seen in all relevant subgroups (THIS IS A SECONDARY ANALYSIS)

5. OVERALL TEST IS SIGNIFICANT

6. GOOD: T 1 better than T 2 Events Plots over Study Time 0.0.25. 50 30.0.75 1.00 00.51.01.52.02.53.0 TREATMENT 1 TREATMENT 2 - Time from randomization in years

7. 0.1 1 10 Overall Gender Males females Age < 65 >65 Previous Condition Hazard Ratio yes no 1 better 2 better Note: Many subgroups may not show statistical significance, but do show consistentcy

8. 0.1 1 10 Overall Gender Males females Age < 65 >65 Years with Condition Hazard Ratio yes no Location X NO YES 1 better 2 better Special subgroup

9. 21

10. Relative Risk Reduction by Qualifying Condition IS n = 6431 MI n = 6302 PAD n = 6452 Total n =19185 30 20 10 0 10 20 Clopidogrel BetterAspirin Better

11. OVERALL TEST IS NOT SIGNIFICANT TECHNICALLY YOU CANNOT GO BEYOND THIS WITH ANY STATISTICAL STATEMENTS USEFUL TO LOOK AT SUBSETS AS EXPLORATORY ANALYSIS (NOT EVEN APPROPRIATE TO CALL IT A SECONDARY ANALYSIS)

12. PROBLEM: T1 not better Events Plots over Study Time 0.0.25. 50 30.0.75 1.00 00.51.01.52.02.53.0 TREATMENT 1 TREATMENT 2 - Time from randomization in years

13. 0.1 1 10 Overall Gender Males females Age < 65 >65 Previous Condition Hazard Ratio yes no 1 better 2 better Note: Only age > 65 is “significant.” Do we believe it?

14. 0.1 1 10 Overall Gender Males females Age < 65 >65 Years with Condition Hazard Ratio yes no Location X NO YES 1 better 2 better Location is “significant”. Was it pre- specified as primary? No

15. Primary Hypothesis anticipates possible subgroup effect Subgroups identified by pre-randomization or post-randomization stratification Subgroups tests for equal vs. unequal effect. This is done formally by INTERACTION TEST Procedure If significant interaction, do not pool and test groups If no significant interaction pool May need to add variable in analysis for groups

16. Primary Hypothesis anticipates subgroup effect (continue) INTERACTION TEST may be avoided and subgroups can be tests separately with control of error rates For example, if there are two groups then each can be tested at 0.025 level of significance For example, groups can be tested sequentially with error rate control

17. Location X: YES 0.0.25. 50 30.0.75 1.00 00.51.01.52.02.53.0 TREATMENT 1 TREATMENT 2 - Time from randomization in years

18. Location X: NO 0.0.25. 50 30.0.75 1.00 00.51.01.52.02.53.0 TREATMENT 1 TREATMENT 2 - Time from randomization in years

19. 0.1 1 10 Overall Hazard Ratio Location X NO YES 1 better 2 better Location: Interaction of location is significant Do not pool

20. Statistical properties of analyses Overall test If Primary hypothesis of overall significance is satisfied Then as secondary analyses we can examine subgroups and control error rates (that is, can control chance of identifying falsely significant subgroups) for specified subgroups If number of subgroups is unspecified, then analysis is exploratory even here

21. Statistical properties of analyses Overall test (continue) If Primary hypothesis of overall significance is not met (that is, we do not achieve statistical significance), then we cannot control error rate (falsely identifying significant subsets). We have used “alpha” on overall test

22. Primary hypothesis anticipates subgroup differences Level of significance is chance of rejecting at least one false null hypothesis and it can be controlled. If there are potentially k subgroups this error rate may be as high as k(0.05) if each subgroup is tested at 0.05 level of significance. With k=2, 0.10; k=3, 0.15, k=5, 0.25

23. Closing Comments Error rates (level of significance) can be controlled even for looking at subgroups if structure of statistical approach is clearly stated. We must not confuse test for subgroups stated in a pre-specified manner from post hoc identification and tests