Regression Discontinuity for the Replication of Randomized Controlled Trial Results: An Application to the Mycotic Ulcer Treatment Trials Catherine Oldenburg, ScD MPH University of California, San Francisco May 9, 2017

Regression Discontinuity Takes advantage of exogenous source of variation in treatment assignment Same family of study designs as instrumental variable analysis CD4 Count-Based Eligibility for Treatment ART Initiation Mortality General Health Status SES etc

Threshold Rules Continuous measures (assignment variable) that determine treatment assignment at a certain number Most compelling are assignment variables measured with error Examples of clinical applications: CD4 count Blood pressure IOP

Conceptualizing RCTs as RD Designs Randomly select number 1-10 from uniform distribution Threshold Rule (1:1 randomization):  5: Treatment; > 5: Placebo Assignment Variable: Uniform distribution from 1-10 Treatment Placebo 1 2 3 4 5 6 7 8 9 10

Conceptualizing RCTs as RD Designs Randomly select number 1-10 from uniform distribution Threshold Rule (1:1 randomization):  5: Treatment; > 5: Placebo Assignment Variable: Uniform distribution from 1-10 Treatment Placebo 1 2 3 4 5 6 7 8 9 10 How do RCT estimates compare to RD estimates?

Mycotic Ulcer Treatment Trials (MUTT) Two RCTs designed to estimate efficacy of treatment strategies for fungal keratitis MUTT I: topical natamycin versus topical voriconazole MUTT II: topical voriconazole plus oral placebo versus topical voriconazole plus oral voriconazole Assignment to MUTT I or MUTT II depended on baseline visual acuity

Fungal keratitis Natamycin Voriconazole Baseline Three Weeks Three Months

Assessed for Eligibility for MUTT I or MUTT II Is visual acuity worse than 20/400? NO YES MUTT I Randomize to: A: Topical Natamycin B: Topical Voriconazole MUTT II Randomize to: C: Topical Voriconazole + Placebo D: Topical Voriconazole + Oral Voriconazole Regression Discontinuity Comparison: MUTT I: Topical Natamycin versus MUTT II: Topical Voriconazole + Placebo

RD Baseline Characteristics   MUTT I Natamycin (<20/400) N=162 MUTT II Voriconazole (>20/400) N=54 Age (median, IQR) 48 (39 to 58) 51.5 (45 to 65) Female sex 73 (45.1%) 20 (37.0%) Occupation Agriculture Non-agriculture 80 (49.4%) 82 (50.6%) 26 (48.2%) 28 (51.9%) Duration of symptoms (median, IQR), days 5 (3 to 9) 8.5 (6 to 14) Baseline BSCVA 0.66 (0.38 to 0.92) 1.70 (1.30 to 1.80) Baseline infiltrate/scar size 3.09 (2.45 to 3.99) 5.41 (4.69 to 6.69)

Continuity in Baseline Characteristics Age Sex

Regression Discontinuity Methods Assess the effect of natamycin versus voriconazole (plus placebo) in RD design Baseline visual acuity is the assignment variable Outcomes include three—month visual acuity and probability of perforation and/or TPK

Visual Acuity Outcome Natamycin Voriconazole

Visual Acuity Outcome Natamycin Voriconazole

Probability of Perforation/TPK Natamycin Voriconazole

Probability of Perforation/TPK Natamycin Voriconazole

Regression Discontinuity Randomized Controlled Trial RD versus RCT   Regression Discontinuity Randomized Controlled Trial 3-month BSCVA (mean logMAR difference, 95% CI) -0.39 (-0.61 to -0.17) -0.18 (-0.30 to -0.05) Perforation/TPK (OR, 95% CI) 0.31 (0.12 to 0.77) 0.42 (0.22 to 0.80)

Voriconazole versus Voriconazole Check the assumptions of regression discontinuity Expect no difference between two groups   Overall Sample Fusarium spp Only 3-month BSCVA (mean difference, 95% CI) -0.21 (-0.47 to 0.05) -0.59 (-1.19 to 0.02) Perforation/TPK (OR, 95% CI) 0.48 (0.19 to 1.20) 0.35 (0.05 to 2.37)

Limitations Sample size Use of placebo in MUTT II

Conclusions Regression discontinuity is a potentially useful tool when randomization is not possible Results were broadly similar between RD and RCT, although the RD was an overestimate of effects Ideally, presenting results of observational studies using methods with different assumptions for causal inference will improve confidence in results

Thank you! UCSF: Aravind Eye Hospitals: N. Venkatesh Prajna Tiruvengada Krishnan Revathi Rajaraman Muthiah Srinivasan UCSF: Kathryn J. Ray Kieran O’Brien Travis C. Porco Nisha R. Acharya Jennifer Rose-Nussbaumer Thomas M. Lietman Contact: catherine.oldenburg@ucsf.edu