Reducing Surveillance Bias in Adverse Events Reporting in an Unmasked Treatment Trial Mae Gordon, Julia Beiser, Patricia Morris, J. Philip Miller, Michael Kass The Ocular Hypertension Treatment Study Washington University School of Medicine, St. Louis, MO Society for Clinical Trials, May 22, 2012 Miami, FL

Reasons for Unmasked Trial Unfeasible to mask Unfeasible to mask Difficult to maintain masking Difficult to maintain masking Wanted participants to commit to randomization assignment Wanted participants to commit to randomization assignment Efficacy measures were quantitative and evaluated by masked graders at Reading Centers Efficacy measures were quantitative and evaluated by masked graders at Reading Centers

Ocular Hypertension Treatment Trial Purpose Purpose To determine the safety and efficacy of ocular hypotensive medication in preventing primary open angle glaucoma

In January1997, the OHTS protocol for reporting adverse events was made more rigorous because of large clinic-to-clinic variation in the completion of the adverse-event forms. Therefore, data from the adverse-event forms are reported from January 1997 to the present.

Patient found eligible for OHTS Medication n=817 Topical ocular hypotensive medication Observation n=819 No topical medication Randomization n=1,636 Determination of Abnormality Masked readers at Reading Centers Tests for Primary Open Angle Glaucoma Visual Field 6 mo. Optic disc 12 mo. Adjudication to Attribute Abnormality to Glaucoma Masked Endpoint Committee

Safety Outcomes: Unmasked Medical and ocular history- unmasked Medical and ocular history- unmasked Patient Reported Outcomes-unmasked Patient Reported Outcomes-unmasked –QoL : SF-36 –Ocular Symptom checklist

Number of Adverse Event Forms by Time and Randomization Group before 1997 DSMB Report 9/6/96, 12.7

Adverse Event Forms before 1997 OBSMED N % P-value Adverse Events238 31%536 69% < Serious Adverse Events %148 56% 0.05 DSMB Report 9/6/96, 12.12

Adverse Events by Organ System before 1997 OBSMED N % P-value Ocular96 26%276 74%< Respiratory18 24%56 76%< GI18 33%37 67%0.016 Neurologic 8 27%22 73%0.018 DSMB report 9/6/96, 12.14

Percent of Pts. with > 1 AE by Site before 1997 DSMB Report 9/6/96, 12.7

Variation Among 22 Sites in Percent of Pts. with > 1 AE Before 1997 Median22% Max66% Min6% Range60%

Patient Reported Outcomes

Percent of Pts. reporting Ocular Symptoms in Both Eyes in Self-Administered QSS before 1997 DSMB report 9/6/96, 6 mo

Relative Risk of Ocular Symptoms at before 1997 SymptomsRelative RiskP-value Burning1.50<0.05 Dryness0.70<0.05 Itching0.86>0.05 Tearing0.92>0.05 DSMB report 9/6/96, 6 mo

SF-36 No differences detected between randomization groups p<0.10 No differences detected between randomization groups p<0.10 –Bodily pain –Health perceptions –Mental health –Physical functioning –Role emotional –Role physical –Social functioning –Vitality

Data and Safety Monitoring Committee Recommended that AE reporting start denovo after 1997, except for mortality data Recommended that AE reporting start denovo after 1997, except for mortality data Recommended that adverse events reporting be more rigorous Recommended that adverse events reporting be more rigorous –Retrieve hospital discharge summaries –Script medical & ocular history –Implement edits checks –Retrain clinic coordinators

After 1997: Medical History Form completed by Clinic Staff a. High Blood Pressure (Hypertension)  1 Yes  2 No b. High Blood Pressure (Hypertension)  1 Yes  2 No c. Diabetes, or sugar in the blood  1 Yes  2 No k. Other conditions_________________  1 Yes  2 No Medical History 3. Have you seen a doctor since (state last FV date)? If yes, for what problem(s)? _________________ ________________________________________  1 yes  2 no  9 unknown 4. Have any new health problems been diagnosed since (state last FV date)? _________________ ________________________________________  1 yes  2 no  9 unknown (if yes, complete Adverse Event Form) If yes, check appropriate boxes:......

Results

Number of AE Forms by Time and Randomization Group before and after 1997

Before and After 1997 Percent of FV Forms with at least one AE by Site Before 1997

Before and After 1997 Percent of FV Forms with at least one AE by Site Before 1997 After 1997

Adverse Event Forms Before and After 1997 Before 1997 OBSMED N % P-value Adverse Events %536 69%< Serious Adverse Events %148 56%0.05 DSMB Report 9/6/96, ;5/1/98, 12.22

Adverse Event Forms Before and After 1997 Before 1997After 1997 OBSMEDOBSMED N % P-valueN % P-value Adverse Events %536 69%< %945 56%< Serious Adverse Events %148 56% %206 50%1.0 DSMB Report 9/6/96, ;5/1/98, 12.22

Adverse Events by Organ System before and after 1997 to 6/96 OBSMED N % P-value Ocular 96 26%276 74%< Respiratory 18 24%56 76%< GI 18 33%37 67%0.016 Neurologic 8 27%22 73%0.018 DSMB report 9/6/96,12.14: 5/1/

Adverse Events by Organ System before and after 1997 Before 1997After 1997 OBSMEDOBSMED N % P-valueN % P-value Ocular 96 26%276 74%< %311 64%< Respiratory 18 24%56 76%< %101 52%0.614 GI 18 33%37 67% %74 56%0.190 Neurologic 8 27%22 73% %48 47%0.482 DSMB report 9/6/96,12.14: 5/1/

Results: Safety No differences No differences –Participant reported outcomes –Hospitalizations –Visual acuity –New medical conditions –Worsening pre-existing conditions –Mortality

Reflections Should assess expectation of medication side-effects Should assess expectation of medication side-effects Use expectation as a covariate for patient and clinic reported safety outcomes Use expectation as a covariate for patient and clinic reported safety outcomes Don’t be afraid to jettison biased data to get a clean start Don’t be afraid to jettison biased data to get a clean start

With Appreciation to the OHTS DSMC

Thank-you! Questions invited!