Rerandomization in Randomized Experiments Kari Lock and Don Rubin Harvard University JSM 2010

The “Gold Standard” Why are randomized experiments so good? They yield unbiased estimates of the treatment effect They eliminate (?) confounding factors… … ON AVERAGE. For any particular experiment, covariate imbalance is possible (and likely)

Rerandomization Suppose you are doing a randomized experiment and have covariate information available before conducting the experiment You randomize to treatment and control, but get a “bad” randomization Can you rerandomize? Yes, but you first need to specify a concrete definition of “bad”

Randomize subjects to treated and control Collect covariate data Specify a criteria determining when a randomization is unacceptable; based on covariate balance (Re)randomize subjects to treated and control Check covariate balance 1) 2) Conduct experiment unacceptable acceptable Analyze results with a Fisher randomization test 3) 4)

Unbiased To maintain an unbiased estimate of the treatment effect, the decision to rerandomize or not must be  automatic and specified in advance  blind to which group is treated Theorem: If the treated and control groups are the same size, and if for every unacceptable randomization the exact opposite randomization is also unacceptable, then rerandomization yields an unbiased estimate of the treatment effect.

Mahalanobis Distance Define overall covariate distance by M = D’r -1 D D j : Standardized difference between treated and control covariate means for covariate j k = number of covariates D = (D 1, …, D k ) r = covariate correlation matrix = cov(D) Choose a and rerandomize when M > a

Rerandomization Based on M Since M follows a known distribution, easy to specify the proportion of rejected randomizations M is affinely invariant Correlations between covariates are maintained The variance reduction on each covariate is the same (and known) The variance reduction for any linear combination of the covariates is known

Rerandomization Theorem: If n T = n C and rerandomization occurs when M > a, then and

Difference in Covariate Means Difference in Outcome Means

(theoretical v a =.16)

(theory =.58) Equivalent to increasing the sample size by a factor of 1.7 Difference in Outcome Means Under Null

Conclusion Rerandomization improves covariate balance between the treated and control means, and increases precision in estimating the treatment effect if the covariates are correlated with the response Rerandomization gives the researcher more power to detect a significant result, and more faith that an observed effect is really due to the treatment