Dirk Hasenclever Oana Brosteanu ADAMON Risk adapted monitoring is not inferior to extensive on-site monitoring Dirk Hasenclever Oana Brosteanu Institute for Medical Informatics, Statistics & Epidemiology (IMISE) Clinical Trial Centre (ZKS) University of Leipzig on behalf of the ADAMON project group ICTMC/SCT 2017 May, 8th 2017 Liverpool

ADAMON objective and design Is trial-specific, risk-adapted, focused on-site monitoring as effective as extensive on-site monitoring in avoiding the occurrence of major or critical findings as ascertained in a final audit? Rando-misation risk-adapted monitoring extensive monitoring Trial Sites Clinical Trials ADAMON audit in every trial site final Stratified, cluster-randomised study Trials are included (strata in analysis) Trial sites are randomised (cluster) Individual patients are evaluated (observation unit)

Planned interim analysis ADAMON trial Trial sites monitored with risk-adapted strategy End of trial or Planned interim analysis Participating clinical trial Preparation trial conduct Trial sites fully monitored 12 clinical trials to be included: broad range of medical conditions differences in complexity of design different methods of data capture (paper-based, eCRF) non-commercial sponsor multi-centre (at least 6 sites) risk classification K2 (intermediate) or K3 (low) Definition of risk-ADApted MONitoring strategies based on the risks identified by published, structured procedure for risk analysis in clinical trials. Brosteanu O et al. "Risk analysis and risk adapted on-site monitoring in non-commercial clinical trials" Clin Trials 2009

Planned interim analysis ADAMON trial Trial sites monitored with risk-adapted strategy Participating clinical trial End of trial or Planned interim analysis Preparation trial conduct Trial sites fully monitored Trial specific risk analysis and classification Definition of key data Preparation of the 2 monitoring manuals for the trial Training of the trial monitors (at least 2 persons!) Check of requirements for central oversight and data management Central randomisation of the trial sites Risk: Major or critical findings concerning GCP objectives: Rights, integrity and confidentiality of trial subjects protected Safety ensured Data and reported results credible.

Planned interim analysis ADAMON trial Trial sites monitored with risk-adapted strategy Participating clinical trial End of trial or Planned interim analysis Preparation trial conduct Trial sites fully monitored Supervision of the trial on site monitoring, if necessary additional training for monitors Assessment of monitoring parameters Extensive monitoring on-site Check existence / informed consent for all patients Source data verification (SDV) of all data for all patients Risk-adapted monitoring on-site according to risk class K2 – intermediate risk  key data of 20 – 50% of patients K3 – low risk  one visit, 20% key data versus

Planned interim analysis ADAMON trial Trial sites monitored with risk-adapted strategy End of trial or Planned interim analysis Participating clinical trial Preparation trial conduct Trial sites fully monitored Assessment of the primary endpoint Trial specific definition of major audit findings; Audit manual On site audits in each trial site assessing each patient Patients with at least one major or critical finding in five error domains Informed consent Patient selection Intervention Endpoint assessment SAE reporting

11 trials participated Indication 6 Oncology 2 Hematologic 4 Solid tumours Trauma surgery Parkinson‘s disease Heart failure & depression Severe sepsis Neonatology Type of intervention 8 pharmaceutical 2 surgical methods 1 surgery on top of standard of therapy Monitoring risk class 8 K2 (intermediate) 3 K3 - low

Effort of extensive monitoring compared to risk-adapted monitoring Monitoring time on site Monitoring time on-site Total monitoring time Number of monitoring visits per patient Overall 2.7 2.2 2.1 Monitoring class K2 1.8 K3 5.2 3.6 3.5 Factor: Effort with extensive monitoring / Effort with risk-adapted monitoring (geometric mean of factors in individual trials)

Primary and secondary endpoints Patient-level finding rate 996 / 1618 patients (62%) with 1+ findings in any error domain: IC-process 241/1618 ~15% patients with 1+ findings Total: 292 Selection 331 /1618 ~ 20% patients with 1+ findings Total: 436 Intervention 405 /1618 ~25% patients with 1+ findings Total: 758 Endpoints 420 /1618 ~26% patients with 1+ findings Total: 614 SAE-reporting 295 /1618 ~18% patients with 1+ findings Total: 356 2456 individual ADAMON audit findings Observed finding rates varied by trial between 18 and 99%.

Primary Endpoint No evidence of heterogeneity No risk class subgroup effect. pre-specified tolerance margin 0.6 on log odds scale Non-inferiority shown

Secondary endpoints 95%-Confidence intervals of primary endpoint and all its components pre-specified tolerance margin.

Interpret log adds ratios in terms of rates Observed finding rates 18 - 99%. Potential benefit of extensive on-site monitoring is small relative to overall finding rates.

Typical types of findings I Informed Consent (N=292) dating the signature by the participating patient (N=180) not by qualified staff (N=38) Patient Selection (N=436) measurements required for the assessment of eligibility (N=175) not performed, not performed in a timely manner, out of range in Prohibited co-medication (N=89 in one trial) Not in target population (N=18) Eligibility not verifiable (N=53).

Typical types of findings II Intervention (N=758) Treatment modification rules in complex treatment schemes: ignored (N=227) trigger not measured (N=166) Relevant dose deviations (N=90) Endpoint assessments (N=629) Not assessed (N=95) Measured inadequately (N=181) Not on schedule (N=68) Serious Adverse Event reporting (N=356) Not reported (N=73) Reported with delay (N=217)

error prone trial rules error prone complex clinical settings. High finding rates Evidence for error prone trial rules error prone complex clinical settings. Room for improvement in study rule formulations. Explorative analysis: At least half of formal trial rule violations could be avoided by improved rules serving the same clinical objective. Domain of methodical/biometrical research Recurrent problem areas: Wrong granularity/ pseudo-precision Over-specification and unnecessary interference with standard clinical practice Non-transparent modality of study rules: strict rule versus loose guidance of local decision making

Risk adapted monitoring Conclusions Risk adapted monitoring is not inferior to extensive on-site monitoring requires less than 50% of extensive on-site monitoring resources. Complicated, overly specific or not properly justified protocol requirements contributed to the overall frequency of findings. Risk-adapted monitoring has a part to play in quality control and source data verification. However, no monitoring strategy can remedy defects in quality of design. Monitoring should be embedded in a comprehensive quality management approach covering the entire trial lifecycle.

ADAMON project group Ulrike Zettelmeyer CTC Cologne Anke Strenge-Hesse ECRIN Office / KKS Network Ursula Paulus formerly: CTC Cologne Gabriele Schwarz BfArM Dirk Hasenclever IMISE Leipzig Oana Brosteanu CTC Leipzig Peggy Houben Anja Schneider ADAMON project group