To Prevent Selection Bias Minimal Balance is Sufficient Wenle Zhao, PhD Medical University of South Carolina, Charleston, SC, 29425, USA Society for Clinical Trials 36th Annual Meeting Arlington, VA, USA - May 17-20, 2015

Contents Where does Selection Bias Come From? How to prevent selection bias? How to avoid random serious imbalance?

A Completed Trial with Suspicious Selection Bias. The Worst Thing in the World of Clinical Trials Funding? Recruitment? A Completed Trial with Suspicious Selection Bias.

Defense Measurements against Selection Bias Treatment Allocation Real-time Subject Randomization Subject Enrollment Outcome Assessment Random Allocation Allocation Concealment Treatment Masking The only reliable protection left against selection bias.

Allocation Randomness Target allocation ratio Random variable ~U(0,1) To balance treatment distribution Permuted Block Randomization Biased Coin, Urn Design To balance baseline covariate Stratified Randomization Minimization Allocation Randomness Complete Randomization Permuted Block Randomization Minimization

Predictability Defeats Concealment & Masking Deterministic Assignment 100% ~ 87% Proportion of DA 50% 33% 25% 20% B = 2 B = 4 B = 6 B = 8 Minimization Permuted Block Randomization

Evidence of Selection Bias in Randomized Trials 1. Heparin for myocardial infarction E 2. University Group Diabetes Program P 3. Talc and mustine for pleural effusions P 4. Tonsillectomy for recurrent throat infection in children P 5. Oxytocin and amniotomy for induction of labor P 6. Western Washington Intracoronary Streptokinese Trial E 7. RSV immune globulin in infants and young children E 8. A trial to assess episiotomy E Selection bias evidence identified P 9. Canadian National Breast Cancer Screening Study P 10. Surgical Trial P 11. Lifestyle Heart Trial E 12. Coronary Artery Surgery Trial P 13. Etanercept for children with juvenile rheumatoid arthritis E 14. Edinburgh Randomized Trial of Breast Cancer Screening P 15. Captopril Prevention Project P 16. Göteborg (Swedish) Mammography Trial E 17. HIP Mammography Trial E 18. Hypertension Detection and Follow-up Program P 19. Randomized Trial to prevent vertical transmission of HIV-1 P 20. Effectiveness trial of diagnostic test E 21. S African trial of high-dose chemotherapy for metastatic breast cancer P 22. Randomized study of a culturally sensitive AIDS education program P 23. Runaway Youth Study Suspicious election bias due to p-value < 0.05 P 24. Cluster randomized trial of palliative care P P 25. Randomized trial of methadone with or without heroin P 26. Randomized NINDS trial of tissue plasminogen activator for acute ischemic stroke P 27. Norwegian Timolol Trial P 28. Laparoscopic versus open appendectomy P 29. The Losartan Intervention for Endpoint reduction in Hypertension Study P 30. The Heart Outcomes Prevention Evaluation Study

P > 0.3 Protect Trials Against Selection Bias P < 0.05 ? Selection bias will result small p-values Complete randomization may (5% chance) see a p-value < 0.05 P < 0.05 ? P > 0.3 P > 0.2

The Logic Real-time complete randomization Eliminates selection bias due to allocation predictability Eliminates selection bias due to allocation concealment failure Totally eliminates selection bias Without selection bias, complete randomization may still have Imbalance in treatment distribution  Power loss is trivial Imbalance is baseline covariate distribution  Adjustment, not balancing, is the solution Serious baseline covariate imbalance with p-value < 0.05  5% chance for any covariate  60% chance for at least one in 10 covariates  Suspicion of selection bias  Trouble in trial result interpretation

Options We Have Stratified Restricted Randomization Permuted Block Randomization Biased Coin Design - Efron Urn Design - Wei Big Stick Design – Soares & Wu Maximal Procedure – Berger et al. Block Urn Design – Zhao & Weng Unnecessarily tighten control imbalances. Disabled when number of strata getting large. Minimization Most assignments are deterministic. Dynamic Hierarchy Balancing Hierarchy order is hard to justify.

Minimal Sufficient Balance Procedure Subject ready for randomization T-test for continuous var. χ2 test for categorical var. N Any serious imbalance? Any p-value < 0.2? Y Current assignment can effectively reduce imbalances? Y N Complete randomization Biased coin assignment The proportion depends on p-value threshold and biased coin probability Next subject

Example : NINDS rt-PA Stroke Study p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810 Serious imbalances found in 2 of the 11 baseline covariates.

Example : NINDS rt-PA Stroke Study NINDS rt-PA Stroke study data. Simulations = 5000 Baseline covariates: Severity (NIHSS) Age Onset to treat Glucose Center

Example : NINDS rt-PA Stroke Study Balance 11 baseline covariates Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled* NINDS rt-PA data, Imbalance control limit p-Value ≥ 0.3. ξ= 0.65, simulation = 1000/scenario. p-value Site NIHSS Age OTT Glucose Stroke Subtype Sex Fibrinogen Weight Systolic BP Diastolic BP Low 2.5% boundary 0.226 0.259 0.237 0.262 0.244 0.214 0.245 0.239 0.252 0.246 Low 5% boundary 0.276 0.295 0.279 0.288 0.280 0.292 0.277 0.278 0.281 0.287 Low 10% boundary 0.309 0.341 0.316 0.323 0.315 0.326 0.322 0.319 0.330 Median 0.605 0.631 0.626 0.624 0.638 0.609 0.634 0.610 Observed in the Original Study 0.9987 0.1398 0.0289 0.8662 0.7804 0.0733 0.6265 0.1808 0.0111 0.5968 0.2810

Summary Complete randomization eliminates selection bias due to allocation predictability. Real-time randomization eliminates selection bias due to allocation concealment failures. Minimization method has the highest proportion of deterministic assignments, and therefore is vulnerable to selection bias. Power loss due to treatment imbalance is trivial. Justification, not balancing, is the solution for covariate confounding effects. Using Minimal Sufficient Balancing to prevent random serious imbalances, while maintaining a high level of allocation randomness.

Thank You! Contact me at: zhaow@musc.edu

Some of my works on Randomization Zhao W, Ciolino J, Palesch Y. Step-forward randomization in multicenter emergency treatment clinical trials. Acad Emerg Med. 2010 Jun;17(6):659-65. doi: 10.1111/j.1553-2712.2010.00746.x. PMID: 20624149 . Zhao W, Weng Y, Wu Q, Palesch Y. Quantitative comparison of randomization designs in sequential clinical trials based on treatment balance and allocation randomness. Pharm Stat. 2012 Jan-Feb;11(1):39-48. doi: 10.1002/pst.493. PMID: 21544929 Zhao W, Weng Y. Block urn design - a new randomization algorithm for sequential trials with two or more treatments and balanced or unbalanced allocation. Contemp Clin Trials. 2011 Nov;32(6):953-61. doi: 10.1016/j.cct.2011.08.004. PMID: 21893215 Zhao W, Hill MD, Palesch Y. Minimal sufficient balance--a new strategy to balance baseline covariates and preserve randomness of treatment allocation. Stat Methods Med Res. 2012 Jan 26. [Epub ahead of print] PMID: 22287602 Zhao W. Selection bias, allocation concealment and randomization design in clinical trials. Contemp Clin Trials. 2013 Sep;36(1):263-5. doi: 10.1016/j.cct.2013.07.005. Epub 2013 Jul 19. No abstract available. PMID: 23871796 Zhao W. A better alternative to stratified permuted block design for subject randomization in clinical trials. Stat Med. 2014 Dec 30;33(30):5239-48. doi: 10.1002/sim.6266. PMID: 25043719 Zhao W, Durkalski V. Managing competing demands in the implementation of response-adaptive randomization in a large multicenter phase III acute stroke trial. Stat Med. 2014 Oct 15;33(23):4043-52. doi: 10.1002/sim.6213. Epub 2014 May 22. PMID: 24849843 Zhao W, Mu Y, Tayama D, Yeatts SD. Comparison of statistical and operational properties of subject randomization procedures for large multicenter clinical trial treating medical emergencies. Contemp Clin Trials. 2015 Mar;41:211-8. doi: 10.1016/j.cct.2015.01.013. Epub 2015 Jan 29. PMID: 25638754