Outline of Stratification Lectures Definitions, examples and rationale (credibility) Implementation –Fixed allocation (permuted blocks) –Adaptive (minimization) Rationale - variance reduction

Stratification A procedure in which factors known to be associated with the response (prognostic factors) are taken into account in the design (e.g., randomization) Pre-stratification refers to a stratified design; post-stratification refers to the analysis

Pre- versus Post Stratification and Precision (Variance Reduction) As a general rule, the precision gained with pre- versus post-stratification is less than one might expect The gain in precision is greatest in small studies (where you need it the most) because the risk of chance imbalance is greater. Covariate adjustment for prognostic factors is usually carried out with regression (e.g., linear, logistic, or proportional hazards regression.

Stratification Can Increase Precision Simple versus stratified random sampling. Snedecor and Cochran note (p. 520): “If we form strata so that a heterogeneous population is divided into parts each of which is fairly homogeneous, we may expect a gain in precision over simple random sampling”. Ref. Snedecor and Cochran, Statistical Methods

Stratification Can Increase Precision Randomized block versus completely random design. Snedecor and Cochran note (p. 299): “Knowledge (about predictors or response) can be used to increase the accuracy of experiments. If there are a treatments to be compared,…first arrange the experimental units in groups of a, often called replications. The rule is that units assigned to the same replication should be as similar in responsiveness as possible. Each treatment is then allocated by randomization to one unit in each replication…Replications are therefore usually compact areas of land…This experimental plan is called randomized blocks.”

Pre-stratification Does Not Matter. Peto et al note: “As long as good statistical methods,…,are used to analyze data from clinical trials, there is no need for randomization to be stratified by prognostic features.” Keep it simple so investigators are not discouraged from participating. Post-stratified analysis is needed with pre- stratification anyway. Improvement in sensitivity (precision) with pre- stratification compared to letting stratum sizes be determined by chance is small. Peto R et al., Br. J Cancer, pp ,1976

Stratified Design for Comparing Treatments StratumAB m1m1 m2m2 m3m3 m4m4 nana nbnb Treatment m 1A m 2A m 3A m 4A m 1B m 2B m 3B m 4B Typical situation:m1m1 ≠m2m2 m3m3 m4m4 ≠≠ Study is designed/powered based on n a and n b Goal: m iA = m iB for all i.

How much of a price does one pay with respect to precision by trusting randomization to achieve reasonable balance? Consider the relative efficiency of a stratified design to an unstratified design: Var (treatment contrast with stratification) Var (treatment contrast with no stratification in design, but post-stratified analyses) RE =

Pooling Estimates Estimates: E, E Var (E ) =  w E + w E Pooled Estimate: Best Pooled Est: w = , w =  w  + w  Variance Pooled Est: w + w (w + w )

Continuous response, equal variance - effect of chance imbalance n A =total number randomly assigned to A n B =total number randomly assigned to B g =fraction of those given A with prognostic factor h =fraction of those given B with prognostic factor Treatment Stratum A B S1S1 S2S2 A n g B n h nAnA nBnB (1-g)nAnA (1-h)nBnB RE = A n g B n h (1-g) (1-h) A n g B n h + + A n g B n h + A n B n+ 1 - 

RE obtained by noting: 1)Var( y - y ) = 1A1B 1 + n g A 1 n h + ()  2 2)Var(y - y ) = 2A 2B n A n B ( )  2 (1-g) (1-h) 3)Pooled variance is: Var Pooled (y - y ) = AB wiwi 2 Var (y - y ) iAiB 2  w i  2 w 1 = A n g B n h + A n g B n h A n B n (1-g) (1-h) + w 2 = A n B n (1-g)(1-h) B 1 1

For Stratified Design, g = h w = 1 n A n B + n A n B g 2 n A n B + n A n B (1 - g)

Assume A n B n = RE = g(1-g)h(1-h) g + h       e.g., block randomization used Consider the case of g = 2h: 0.10, , , , g, h RE

Bernouli Response Loss of efficiency = h2n2h2n2 1 - h = lack of balance 2n = number in each stratum Ref: Meier (Controlled Clinical Trials, 1981)

This can be seen by noting: 1)Stratified design, for stratum 1 Var (p - p ) = AB since n = n = n A1 B n1n A 1 + 1n1n B 1 p1q1p1q1 () 2n2n = p1p1 q1q1 (()) Note: q 1 = 1- p 1

The ratio of these variances is proportional to: 2)No stratification in design; post-stratification in analysis Var (p - p ) = A B1 1 1 n+h + 1 n-h p1q1p1q1 () 1/n+h + 1/n-h 2/n = h2n2h2n2 1 -

n = 10 1(11, 9)0.99 2(12, 8)0.96 4(14, 6)0.86 5(15, 5)0.75 h RE (n, n ) 1 A 1 B

Example: Brown et al. Clinical Trial of Tetanus Anti-toxin in Treatment of Tetanus. Lancet, ;1960 (see also Meier, Cont Clin Trials, 1981; a slightly different approach is taken here). Anti- Toxin (A) Alive Dead No Anti- Toxin (B) p= overall death rate = = ^

= 20/41 = p ^ A = 29/38 = p ^ B p ^ A p ^ B - = Var p ^ A p ^ B - () = + A 1n1n B 1n1n ][ p ^ 1p ^ - () = [(())] + = SE p ^ A p ^ B - () =

Time from first symptoms to admission turned out to be an important prognostic factor; therefore, post-stratification was carried out. A Alive Dead B A Alive Dead B 13 8 < 72 Hours ≥ 72 Hours

Stratum 1: < 72 hours Stratum 2: ≥ 72 hours p ^ 1A p ^ 1B - = p ^ 1A = p ^ 1B = p ^ 1 = SE( ) = p ^ 1A p ^ 1B - p ^ 2A p ^ 2B - = p ^ 2 = SE( ) = p ^ 2A p ^ 2B -

Weighted diff. ( ˆ p A - ˆ p B ) w ˆ Let G = fraction of patients in Stratum 1 = 58/79 = ( ˆ p A - ˆ p B ) w = ˆ G( ˆ p 1A - ˆ p 1B )+ (1- G)( ˆ p 2A - ˆ p 2B ) = compared to unweighted VAR(p - p ) = G VAR(p - p ) ˆ A ˆ B w ˆ 1A ˆ 1B ˆ SE(p - p ˆ A - ˆ B ) w = (1- G ) VAR( ˆ p 2A - ˆ p 2B )= ˆ ˆ

Gain in precision achieved with post- stratification Var(post-stratification) Var(no stratification) RE = = = % reduction

How much gain in precision would be achieved if stratification was used in the design?

Force balance within stratum Assume ˆ p ‘s don’t change ij ˆ 1A ˆ 1B ˆ 2A ˆ 2B A Alive Dead B 29 A Alive Dead B < 72 Hours≥ 72 Hours SE(p - p ) = instead of SE(p - p ) = instead of 0.191

Var(stratified design) Var(no stratification) RE 1 = (0.096) (0.109) = % reduction = 2 2 Var(stratified design) Var(post-stratification) RE 2 = (0.096) (0.097) = % reduction = 2 2 SE stratified design = (same weights are used)

GREP 1. P 2. Gp 1. (1-p 1. ) + (1-G)p 2. (1-p 2. ) [ Gp 1. + (1-G)p 2. ] [1 – Gp 1. – (1-G)p 2. ] RE = If p 1. = p 2. Then RE = 1

1)the distribution of the prognostic factor in the population; 2)the relative strength of the prognostic factor; and 3)the expected endpoint rate in the group studied. The reduction in variance achieved with post- stratification depends on:

Scott’s Survey of Trials Published in Lancet and N Eng J Med in 2001 Stratification Permuted block43/150 Minimization6/150 Other adaptive3/150 Other19/150 Unspecified79/150 Scott et al. Cont Clin Trials 2002; 23:

Kahan’s Survey of 258 Trials Published in Four Major Medical Journals in 2010 Method of Randomization Simple: 4 Permuted blocks, no stratification: 40 Permuted blocks, stratification: 85 Minimization: 29 Other: 4 Unclear: 96 Kahan BC et al. BMJ 2012; 345:e5840

Conclusions 1.Usually there is little loss of efficiency with post- stratification as compared to a stratified design. 2.Loss of efficiency results from large chance imbalances for important prognostic factors, which are more likely in small studies. 3.Stratified designs should be considered in small studies (n < 50) with important prognostic factors. 4.Strictly speaking, analysis should account for pre-stratification.

Recommendation for Multi-Center Trials: Always Consider Stratification on Center 1.Clinic populations differ. 2.Treatment differs from clinic to clinic. 3.Each center represents a replicate of overall trial – can investigate treatment x clinic interactions. 4.In some trials (surgery), it may be better to stratify on surgeon within clinic. 5.If there are a very large number of clinical sites, small block size may have to be used and site combined into a priori defined larger strata (e.g., region or country) for analysis

General Recommendations Large trials –Block randomization with stratification by center –Stratification on other factors not necessary (I am a lumper) –If needed, usually okay to carry out block randomization within each stratum Small trials –Block randomization with stratification by center –If stratification on other factors is considered, may have to use an adaptive approach These are consistent with Freidman, Furberg and DeMets (see page 111)