When Is Stratification Detrimental to a Clinical Trial Design? Part II Katherine L. Monti, Ph.D. Senior Statistical Scientist and Director of the Massachusetts Office, Rho, Inc. Adjunct Associate Professor, Biostatistics University of North Carolina Gretchen Marcucci, M.S. Biostatistician, Rho, Inc.

2  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Introduction The method of randomizing clinical trial subjects to treatments in the presence of a (possibly important) prognostic factor requires a design decision: –To stratify on the factor or not to stratify? –Preconception: “Stratification can’t hurt.” This paper assesses that preconception for a particular design problem.

3  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Outline Motivation for the assessment Description of the approach Results Conclusions

4  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Motivation A drug company’s design: –120 subjects – 4 treatments (placebo, three drug doses) – 30 sites – 1 prognostic factor with 2 levels (hi and low levels, continuous covariate) Randomization: –At each site, NOT centralized –In blocks of 4 within factor level within site

5  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Motivation Those designing the study thought that randomizing within level of the (prognostic ?) factor –would increase balance in the design –“couldn’t hurt” Others argued that randomizing within factor level would lead to –operational difficulties –greater imbalance in the design

6  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Operational Difficulties Drug supply requirements increased. (~ 33%) Packaging/shipping costs increased. Additional training visits to sites were needed in order to explain the more complex randomization scheme. The project management burden increased. The misassignment of subjects to treatment was more likely.

7  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Imbalance 120 subjects/ (30 sites) = 4 subjects per site Perfect balance with 4 treatments 120 subjects/ (30 sites x 2 level) = 2 subjects per site for each level Balance not assured with 4 treatments

8  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Imbalance Although 120/30 = 4 is the “expected” number of subjects per site, 4 subjects enrolling at each site is not really expected! –We don’t know what the real enrollment pattern will be, but we know that it is not likely to be exactly 4 subjects per site. Therefore we anticipate some overall treatment imbalance when randomizing within site. However, additionally restricting randomization to within level of the prognostic variable within site could only increase the imbalance.

9  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Simulation I set out to compare the magnitude of the treatment imbalance if randomization were performed in permuted blocks of 4: –within site (WS) (30 strata) –within level of the factor within site (WLWS) (60 strata) Used simulation.

10  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Simulation Used SAS PROC PLAN to generate treatment assignments in permuted blocks of 4 –for 30 sites and –for 30 sites with 2 factor levels per site

11  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Enrollment Patterns To compare the balance in treatment assignment when randomizing WS and WLWS, we need to assign subjects to treatments –at each site and then reassign –in each level at each site To assign subjects to treatments, we need to know the the number of subjects ( N ij ) –in site i (i=1-30) who have –factor level j (j=1,2) The N ij are unknown….

12  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Enrollment Patterns …So we make assumptions However, instead of prescribing the exact N ij for each i and j in the simulation, I defined 9 different enrollment patterns for the 60 site/level combinations.

13  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Enrollment Patterns Each enrollment pattern assumed a distribution of the number of subjects in the 60 site/levels, so that there were: –30 sites, 2 levels per site –120 subjects –0-8 subjects in each of the 60 site/level strata Some enrollment patterns forced balance between the factor levels: N.1 = N.2 = 120/2 = 60

14  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. An Enrollment Pattern EX: A simple pattern: –20 site/levels strata w/ 3 subjects –20 site/levels strata w/ 2 subjects –20 site/levels strata w/ 1 subject Total of 60 site/level strata w/ = 120 subjects Here, N ij = 1, 2 or 3 Given that pattern –The 60 strata were randomly paired to construct 30 sites. –N i1 / N i2 for site i could be any of the following: 3/3, 3/2, 3/1, 2/3, 2/2, 2/1, 1/3, 1/2, or 1/1

15  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Evaluation Criteria For each enrollment pattern, we want to be able to compare the treatment balance when randomization is performed WS and WLWS using permuted blocks of 4 treatments.

16  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Evaluation Criteria There are two types of treatment balance: –Overall balance The extent to which the treatment assignments are balanced. –Within-level balance The extent to which the treatment assignments are balanced within each of the 2 factor levels.

17  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Evaluation Criteria 4 criteria used to assess the randomization results of each simulated run are reported here: –N[1] = smallest N of the 4 treatments –N[1] + N[2] = smallest total sample size for any comparison of treatments –% loss of power compared to a completely balanced design for the comparison based on N[1] + N[2] for a study designed for 90% power –N[4] – N[1] = maximum difference in sample sizes

18  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Evaluation Criteria Overall, treatment balance would be achieved if –N1 = N2 = N3 = N4 = 120/4 = 30 –N[1] = 30 –N[1] + N[2] = 60 (No loss of power) –N[4] – N[1] = 0

19  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Simulation Scheme Do for enrollment pattern (EP) = 1 to 9 –Do 10,000 replications: Generate 60 strata of subjects (per EP). –Assign random numbers to each subject (to determine order of enrollment at the site and the order of enrollment in the level at the site). –Assign a random number to each stratum (to identify the levels, 1 vs 2). Randomly pair the 60 strata into 30 sites. (The stratum with the lower random number is level 1.)

20  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Simulation Scheme Randomly assign treatments WS based on the order of enrollment in the site, then -determine the sample size of each treatment -compute the evaluation criteria Randomly assign treatments WLWS based on the order of enrollment in the site/level, then -determine the sample size of each treatment -compute the evaluation criteria –Retain all the evaluation criteria Go to next enrollment pattern

21  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Results Reported For each EP there are 10,000 values of the evaluation criterion –when randomizing WS –when randomizing WLWS

22  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

23  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

24  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

25  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

26  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

27  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent.

28  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Conclusions With a relatively large number of treatments: –Randomizing within numerous small sites can lead to some treatment imbalance and loss of power. –Randomization within levels of a prognostic factor in those small sites will generally Increase the treatment imbalance overall Increase the loss of power in overall pairwise comparisons Do little to reduce the treatment imbalance within the levels of the prognostic factor

29  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Conclusions “When is stratification detrimental to a clinical trial design?” –I did not address this question directly. Is it the case that stratification by prognostic factor “can’t hurt”? –NO: In some cases, stratification can hurt Statistically (power) Operationally (money)

30  2003 Katherine L. Monti. All rights reserved. No part of this document may be copied without express written consent. Contact Information Slides:

* Plans were constructed with and without forcing balance between strata. No. Subjects Enrollment Plan 12*3*4* Entries are the number of strata having the indicated number of subjects.