Architecture Design of Generic Outcome Adjudication in CTMS Wenle Zhao, PhD Medical University of South Carolina, Charleston, SC, 29425, USA Society for Clinical Trials 36th Annual Meeting Arlington, VA, USA - May 17-20, 2015

Contents 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

Patients Randomization Treatment ATreatment B Outcome AOutcome B Analysis Accurate Outcome ( Endpoint) Assessment is Important The validity of the trial results relies on the accuracy of outcome assessment.

Outcome Safety OutcomeEfficacy Outcome AE is Serious?Quality of Life Modified Ranking Scale NIH Stroke Scale Hamilton Rating Scale for Depression Hamilton Rating Scale for Anxiety Most Outcome Assessments are Subjective AE is Expected? AE is Related to Study Treatment? Glasgow Outcome Scale Extended The Problem: Uneven expertise and potential bias The Solution: Independent central adjudication

1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

Example 1: A Simple SAE Adjudication SAE CRF Submitted Is serious?Is expected?Is related? Done CRF Completed? May be not. How to notify the adjudicator? Auto . When it is expected? No schedule. Need more information? May be yes. CRF Updated after submission? Could happen.

Four challenges for coordinating outcome adjudication  Adjudication procedures vary among projects No fixed model for every procedure.  Timing of outcome event unpredictable No schedule.  Narrow time window for completion of adjudication “Time is of the essence.”  Data changing after adjudication procedure started The target is moving.

1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

The Generic DB Structure for Adjudication Procedures 1 ∞ ∞ 1 ∞ ∞ 1 ∞ 1 Adjudication Procedure ID Form ID Procedure Name Adjudication Step ID Procedure ID Adjudication Step Name Enable-logic condition Disable-logic condition Question 1 Answer 1 Code Group ID …… Question 5 Answer 5 Code Group ID Edit User Group ID View User Group ID Adjudication Result ID Adjudication Step ID Record ID Freeze Record ID Answer 1 …… Answer 5 General Comments Locked Data (CRF) Form ID Form Name Table Name Form Table ID Subject ID Visit ID Q01 Q02 …… Updated By Updated On Form Freeze Table Freeze Record ID Subject ID Visit ID Q01 Q02 …… Updated By Updated On ID The Model Allow multiple steps. Allow multiple updates. Limit ≤ 5 close-ended Qs.

Two Tables are included in the Database Template

One table specifies definitions of each adjudication step. The other table stores adjudication results.

A freeze table is created for each data form table with adjudication

Adjudication Step Enable logicDisable logic Question 1 (Q1) Question 2 (Q2) Question 3 (Q3) Step 1 PM Review (S1) {New SAE CRF submission} {S2 submitted after S1} OR {S3 submitted after S1} CRF properly completed? 0 = No 1 = Yes Requires MSM review? 0 = No 1 = Yes Step 2 MSM Adjudication (S2) {S1Q1=1} AND {S1Q2=1} AND {no S2 submitted after S1} AND {no S3 submitted after S1} {S2 submitted after S1} OR {S3 submitted after S2} Serious? 0 = No 1 = Yes Unexpected? 0 = No 1 = Yes Related to study drug? 1 = Unrelated 2 = Unlikely 3 = Possible 4 = Probably 5 = Definitely Step 3 PM Review (S3) {S2 submitted} AND {no S3 submitted after S2} {S3 submitted after S1} MedWatch Form needed? 0 = No 1 = Yes Example 1: Definition of Three-step SAE Adjudication

Adjudication StepEnable logicDisable logic Question 1 (Q1) Question 2 (Q2) Step 1 Project Manager (PM) Review {New SAE CRF submission}{S2 submitted after S1} CRF properly completed? 0 = No 1 = Yes Requires CEC review? 0 = No 1 = Yes Step 2 Clinical Event Coordinator (CEC) Review {S1Q1=1} AND {S1Q2=1} AND {no S2 submitted after S1} {S1 submitted after S2} OR {S3 submitted after S2} OR {S4 submitted after S2} OR Even packet complete? 0 = No 1 = Yes Event type? 1 = Neurological 2 = Cardiac 3 = Systemic Step 3 Adjudicator 1 Review (S2Q1=1) AND {no S5 submitted after S3} {S1 submitted after S3} OR {S5 submitted after S3} Outcome category=? 1 = ischemic stroke 2 = symptomatic hemorrhagic transformation of an ischemic stroke 13 = transient ischemic attack Death type =? 1 = ischemic 2 = hemorrhagic vascular 3 = nonvascular Step 4 Adjudicator 2 (ADJ2) Review (S2Q1=1) AND {no S5 submitted after S4} {S1 submitted after S4} OR {S5 submitted after S4} Step 5 Adjudicator 3 (ADJ3) Review {S3 submitted after S1} AND {S4 submitted after S1} AND {S3Q1≠S4Q1 or S3Q2≠S4Q2} AND {no S6 submitted after S5} AND {S1 submitted after S5} OR {S6 submitted after S5} Step 6 Adjudication Committee Chair (ACC) Review {S5 submitted after S1} AND {S5Q1≠S3Q1 or S5Q2≠S3Q2} AND {S5Q1≠S4Q1 or S5Q2≠S4Q2} AND {no S1 submitted after S6} AND {S6 submitted after S1} Example 2: Definition of Six-step SAE Adjudication

The Generic Table for Adjudication Results

1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

Site submits a Case Report Form Form has adjudication procedure defined? System evaluates enable logics for each adjudication steps Enable logic = true & disable logic = false? System sends to [Edit] user group members User views a Case Report Form User has adjudication step [Edit] permission & Enable logic = true & disable logic = false? System displays [Adjudication Edit] button for the user User adds/edits adjudication result and submits adjudication record The last freeze record is the same as the current CRF record? System creates new freeze record, links it to the adjudication record. Y Y Y N Y N N N Automated Coordination of Adjudication Activities

Site submits SAE CRF S1Q1 = Yes? Step S3 = PM Review S3Q1: MedWatch Form required? System composes MedWatch Form draft N Y Y N Step S1 = PM Reviews S1Q1: CRF properly completed? S1Q2: Requires MSM adjudication? Auto #1 to PM Auto #3 to MSM Auto #2 to Site S1Q2 = Yes? Step S2 = MSM Adjudication S2Q1: Serious? S2Q2: Unexpected? S2Q3: Related to study treatment? Site adds/edits SAE CRF Auto #4 to PM S2Q1 = Yes & S2Q2 = Yes & S2Q3 = Possible/ Probably / Definitely? S3Q1 = Yes? Auto #6 to Site Site edits MedWatch PM Closes the Adjudication Procedure Auto #7 to PM PM finalizes MedWatch and submits to FDA Auto #8 to All Sites N Y Auto #5 to PM Y N Example 1: Three-step SAE Adjudication Procedure

Site submits SAE CRF S1Q1 = Yes? Step S3 = ADJ1 Review S3Q1: Outcome category? S3Q2: Death type? Y N Step S1 = PM Reviews S1Q1: CRF properly completed? S1Q2: Requires CEC review? Auto #1 to PM Auto #3 to CEC Auto #2 to Site S1Q2 = Yes? Step S2 = CEC Adjudication S2Q1: Event packet complete? S2Q2: Event type? Site adds/edits SAE CRF and upload the event packet Auto #7 to ADJ3 Auto #9 to PM PM Closes this Adjudication Procedure N Y Auto #5 to ADJ1 Y Step S5 = ADJ3 Review S5Q1: Outcome category? S5Q2: Death type? Step S4 = ADJ2 Review S4Q1: Outcome category? S4Q2: Death type? S3 = S4? Auto #6 to ADJ2 N System determines adjudicators 1, 2, and 3 based on S2Q2 and the Random number R. Auto #4 to Site Y S2Q1 = Yes? N Auto #8 to ACC Y Step S6 = Chair Review S6Q1: Outcome category? S6Q2: Death type? S3= S5 or S4 = S5? N Example 2: Six-step Outcome Adjudication Procedure

In this case, the Clinical Event Coordinator (CEC) completed the adjudication within few hours after the Study Manager (SM) ‘s completeness review passed.

In this case, the procedure has not been completed yet after two months, due to CRF data change.

1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

Adjudication Procedure From stepTo step Event count 1st quartile (25%) 2nd quartile (50%) 3rd quartile (75 %) hrs Three-step SAE adjudication CRF submitPM review PM reviewMSM review MSM reviewPM close Six-step outcome event adjudication CRF submitPM review PM reviewCEC review CEC reviewADJ1/2 review ADJ1/2 reviewADJ3 review ADJ3 reviewACC review PM: project manager MSM: medical safety monitor CEC: clinical event coordinator ADJ1/2: adjudicators 1 and 2 ADJ3: adjudicator 3 ACC: adjudication committee chair Our Experiences

The Strategy to Success  Allows multiple adjudication procedures per CRF.  Allows multiple steps for each adjudication procedure.  Allows different questions be asked for each adjudication step.  Only close-ended questions are allowed.  No more than 5 questions are allowed.  Using automated s to help adjudication coordination.  Allows multiple CRF submissions after adjudication started.

Acknowledgement This research is partly supported by following NIH/NINDS grants: U01NS (NETT Palesch, Y. & Durkalski, V.) U01NS062778(ProTECT, Palesch Y.) U01NS062835(POINT, Palesch, Y.) U01NS087748(StrokeNet, Palesch, Y. & Zhao, W.)  The followings are contributed to the design, development, validation and user feedback collection to this work: Keith Pauls(DCU senior programmer) Jaemyung Kim(DCU senior programmer) Catherine Dillon(DCU senior data manager) The Clinical Coordination Center of the NINDS NETT at University of Michigan  Thousands investigators in hundreds institutions and thousands patients participating our clinical trials.

Thank You! Contact me at: