1. Site Monitoring in Clinical Trials: the evidence, new approaches and trial manager perspectives Athene Lane

2. Overview: presentation/discussion Rationale for site monitoring Systematic review of on-site monitoring systems New approaches to site monitoring Peer review site monitoring system (PRIME) Evaluation of PRIME in the ProtecT trial Trial managers experiences and practices

3. Monitoring rationale: ICH- GCP The rights of participants are protected Data is accurate, complete & verifiable (SDV) Conduct adheres to the protocol and GCP ‘Generally there is a need for on-site monitoring, before, during and after the trial’ European Clinical Trials Directive based on ICH- GCP for all IMP trials became UK law in 2004

4. Trial monitoring systems TSCDMC CI & TMG Trial conduct & data collection Regulators & Sponsors CTU Trial sites Site monitoringTrainingData checks Protocol & SOPs EthicsMHRA

5. Systematic review of on- site monitoring systems Rhiannon Macefield, Andrew Beswick, Jane Blazeby

6. PRISMA flow diagram Records from database search (Embase: 529, Medline: 536) Removed duplicates n = 678 Records from other sources n= 28 (Hand search of CT/CCT/CCT, referenced in other papers, personal knowledge) Records screened n = 678 Records excluded n = 526 Full-text articles assessed n =152 Excluded (n = 123) Safety monitoring or central monitoring e.g. radiotherapy QA, no details/methods, not full paper, unavailable (2) Articles included n = 57 Included n = 29

7. Publication characteristics (n)  1 RCT of site monitoring intervention – stopped early  Individual trial reports (30, 26 trials)  Heart disease (33%), cancer (23%)  Group or organisation descriptions (21)  16 groups: e.g. USA NCI cooperative groups, EORTC and pharmaceutical industry  Cost simulations (2) and surveys (3)

8. Published monitoring structures Frequency: multiple visits to all sites (where stated) Range from 2 months to 3 years Monitors varied: sponsors, ‘independent evaluators’, DSMB/TSC, coordinating centre staff, trial coordinators, data managers, clinical investigators, CRA 1 to 8 monitors, typically up to 3

9. Site monitoring activities Review trial documents View site facilities/clinic tours Walk through/discuss procedures with site staff Interview site staff Observe trial procedures Often inadequately described: “A full onsite review”

10. Site monitoring assessments Consent verification SDV and data management Protocol adherence Drug accountability Safety monitoring and ethical approvals Site operation including accrual and retention Staff training

11. Feedback and reports Some exit interviews to outline findings and problems (NCI cooperative groups) Report templates (NIDA CTN, NHBLI, VA) Written report distributed to: Local investigator/site director Sponsors/funders CI and trial coordinator TSC and performance review committees

12. Benefits of site monitoring (5) Identified problems: procedural errors/data inconsistencies Issues resolved quicker, e.g. increased recruitment (2) Improved protocol adherence and GCP compliance (3) Interactions of staff between sites and central staff Shared best practice between sites Opportunities for training

13. Site monitoring disadvantages Costs: typically for one day but up to four days EORTC = £600 direct costs in 1991 NIDA £1000 direct costs in 2009 Staff time regarded as a major cost but not measured 50% of site staff in a survey found visits annoying

14. Summary and ongoing research Most publications from non-commercial trials & groups No consistency in systems Little evaluation of costs and benefits to trials Abstract in Clinical Trials2010;7:428 (paper under review) New trials of site monitoring: Pharma standard v risk-adapted monitoring trials: France: OPTIMON (V Journot) CC Trials 2011 32: 16. Germany: ADAMON (O Brostaneau) C Trials 2009 6:585.

15. New approaches Monitoring workshop of best practice in 2012 N West HTMR, C Tudur-Smith & other hubs Effective and efficient monitoring research CTTI & FDA in USA, M Landray, CTSU, Oxford FDA draft guidance on risk adapted monitoring ECRIN: QA working party on monitoring V Journot, Bordeaux [MHRA risk-adapted processes M Ward]

16. Peer Review Intervention for Monitoring and Evaluating Sites Athene Lane, Rhiannon Macefield, Julia Wade, Liz Down, Sue Bonnington, Pete Holding, Teresa Lennon, Amanda Jones, Liz Salter, David Neal, Freddie Hamdy & Jenny Donovan

17. Report to CIs & local PI Annual PRIME visits to all sites (1-2 d) PRIME structure Exit meeting & problem solving SOP & report template Peer reviewers TM & 2 site nurses (from 5)

18. PRIME Intervention & Evaluation ComponentObjectivePRIME activityHours OrientationTrainingOrientation & trial progress meeting 0.5 Site performancePerformanceSite recruitment and attrition rates Protocol adherenceGCPObservation, feedback & meetings6 Data collectionGCPObservation of CRF completion1 Safety monitoringGCPReview process & documentation0.5 DocumentationGCPSite file review1 Training Site staff training discussion0.5 Site organisationPerformanceCoordinating centre communication0.5

19. Trial conduct observation Recruitment & follow-up appointments Individual feedback given to site staff Errors difficult to identify otherwise: Local exclusion criteria Weight taken with shoes on

20. Evaluation of PRIME ProtecT: prostate cancer treatment trial ISRCTN20141297, HTA funded trial Three years of PRIME site reports analysed Resource use [Survey of site nurses]

21. Findings by component and year GCP adherencePerformanceTraining

22. PRIME benefits and costs Benefits: site performance gains, e.g. Increased radiotherapy CRF return (65%) Study cohesion & communication Identifies individual and study training needs “Useful for ensuring everything is in order! Good for sharing good practice” (staff survey) Annual costs: staff time (32-56 d) & £5,600

23. PRIME visits annually to all trial sites Standardises trial conduct & good practice Site staff focus including as peer reviewers Improves GCP compliance Performance gains Summary

24. PRIME recent research Used in two other studies: SFP Cymru (SEWTU) DUTY (BRTC & SEWTU) PRIME SOP adapted for these studies Benefits from site visits and training Currently seeking additional trials? Evaluate in other trials, including costs

25. Any questions? Group work and discussion Athene.lane@bristol.ac.uk

26. Group work and discussion 1. Survey on site monitoring practice 2. Small groups for 15 minutes to discuss: Your experiences of on-site monitoring List benefits and disadvantages Other ways to monitoring trial conduct at sites? What sort of trials could benefit from PRIME? 3. Feedback main points to the whole group