Sia Gravani 10th May 2017 4th ICTMC & 38th SCT, Liverpool The ATLAS study ANALYSIS OF TRIAL-SPECIFIC TRAINING DURING THE SITE INITIATION PHASE: The ATLAS study Sia Gravani 10th May 2017 4th ICTMC & 38th SCT, Liverpool Hello, My name is Sia and I am going to present you part of my PhD work on the ‘Analysis of trial-specific training during the site initiation phase’ of trials.

What we know so far about trial training Quality Assurance: To improve design and conduct of trials Rely on appropriate training of site staff, tailored to trial role Why is trial-specific training of staff crucial? Trial participants’ rights are protected High quality data is collected Trial is successfully conducted Systematic Review findings [under publication]: Substantial variability of training processes So as we all know, quality assurance systems should be in place in RCTs to improve their design and conduct. However, their successful implementation relies on appropriate training of site staff, tailored to their trial role. So why is trial-specific training of site staff so important? Well, it is important because by training staff to fully understand their job roles/responsibilities in the trial – for example how to screen, approach and consent patients, randomise them (when it comes to randomised trials), deliver the trial interventions and collect data, we ensure that first of all trial participants’ rights are protected but also we ensure that high quality data is collected over the course of the trial and analysed, thus ensuring the successful conduct of the trial. So, to inform my work on trial-specific training, I did a systematic review, which showed that there is substantial variability of training processes Across RCTs of different intervention types, disease areas and sample sizes Aim, content, methods, frequency, duration, location, evaluation

ATLAS study Investigate how trial-specific training is provided The ATLAS study Investigate how trial-specific training is provided Explore trial managers’ and site staff views on trial-specific training Make recommendations for improving the support provided to staff To further explore trial-specific training, we designed the ATLAS study to: Investigate how trial-specific training is provided during site initiation Explore trial managers’ and site staff views on the importance of trial-specific training Make recommendations for improving the support provided to staff during site start-up

Methods Six studies Interviews The ATLAS study Methods Six studies Interviews trial managers (n=6) healthcare professionals (n=13) Observations of trial-specific training sessions (n=13) Questionnaires for trial managers and site staff (n=120) As part of ATLAS, we purposefully selected six studies to serve as case studies. Within each case study, trial managers facilitating trial-specific training sessions as well as healthcare professionals receiving the training were interviewed either face-to-face or over the phone between April 2015 and April 2016. Moreover, Non-participant observations of trial-specific training sessions (n=13) as well as questionnaires completed by trial managers and site staff (n=120) were also used to gain an overview of site staff and facilitators’ experience. Interviews were transcribed verbatim and analysed thematically. Analysed thematically

ATLAS case studies Study no. Study type Intervention Number of Sites The ATLAS study ATLAS case studies Study no. Study type Intervention Number of Sites Site Initiation training - Mode Site Initiation training - Methods T01 RCT Drug 1 Face-to-face Presentation Demonstration Discussion T02 RCT (pilot) Procedure/ Surgery 5 T03 Device 100 Online (recorded) Self-study (Videos, Manual) T04 Radiotherapy 10 Teleconference T05 Non-randomised (feasibility) Low-friction bedding 3 T06 15 Simulation As you can see in this table, the six studies taking part in ATLAS, covered a range of different interventions, number of sites as well as a variety of training modes & methods of site initiation, although there were some similarities in some studies.

Key findings (1/2) Face-to-face interaction most beneficial in: The ATLAS study Key findings (1/2) Face-to-face interaction most beneficial in: Facilitating learning Enhancing personal relationship building & networking Impractical for large trials Purpose of site initiations: Trial Managers: Check everything is in place at site So what did we find? First of all, although a variety of training modes (face-to-face, teleconference, online) was used to deliver training across the six case studies (as we saw in the previous table), TMs and site staff felt that face-to-face interactions were most beneficial in facilitating learning of the trial and most importantly enhancing personal relationship building and networking of the trial team. However, they also discussed about the practicalities of organizing face-to-face training in larger trials. Another interesting finding was that the purpose of site initiations slightly differed between trial managers and site staff, with Trial managers focusing more on checking that everything was in place at the site before recruitment started and that staff were aware of their roles and responsibilities in the trial, whereas, on the other hand, staff viewed site initiations as the best time to meet and connect with the trial team and in fact, connect names with faces. Site staff: Meet and connect with the trial team

Key findings (2/2) Decision-making process: Evaluation of training: The ATLAS study Key findings (2/2) Decision-making process: Important in selecting level and mode of training Overlooked at early planning stages & poorly documented Influenced by Trial Unit’s established processes & funding Evaluation of training: Best practice to identify key areas where follow-up training might be required Not routinely conducted The TMs also discussed the importance of the decision-making process in selecting the appropriate level and mode of training required, this process was often overlooked at early planning stages of trials and was poorly documented Although evaluation of site initiation training was considered best practice to identify key areas of the trial that staff felt unsure about and where follow-up training might be required, it was not routinely conducted unless there was process evaluation element embedded in the trial. Last but not least, TMs and staff emphasised the need for ongoing additional training and support over the course of the trial (rather than only during site initiation) to ensure the trial’s successful conduct. However, this was mainly provided on an ad-hoc basis. Additional training & Support: Ongoing, important in ensuring trial’s successful conduct Mainly provided on an ad-hoc basis

The ATLAS training toolkit The ATLAS study Qualitative findings Regulations & guidance documents Literature on learning, training and development Specify staff training needs Decide on training mode Design training Perform & Document training Evaluate training Additional training needs? End No Yes STEP 1 STEP 2 STEP 3 STEP 4 STEP 5 Start Based on my analysis, the current regulations and literature around learning/training, I developed the ATLAS training toolkit which: Consists of five steps each focusing on a particular element of the training cycle. I’m not going to go into much detail, however, as you can see from the overview flowchart, the first step includes specifying the staff training needs and deciding on the training mode for the trial, followed by designing the training , then performing and documenting the training and lastly evaluating the training with the aim to identify if there are any additional training needs and if there are, then the training cycle resumes. Each step is supplemented by various supporting documents My first attempt to evaluate the ATLAS training toolkit was by going back and interviewing the TMs of the trials taking part in ATLAs and also by attending two trial managers’ meetings and discussing the usability of the toolkit. First evaluation via: Follow-up interviews with TMs (n=6) TMs’ meetings (n=2)

The ATLAS training toolkit The ATLAS study Decision-making tools Training plan template Feedback forms From the ATLAS toolkit, there were 3 components that were considered most useful by TMs: Decision-making tools: to provide prompts to assist TMs in choosing the appropriate mode of training for the trial Training plan template: to document the decisions made and list all planned training activities over the course of the trial Feedback forms (staff & facilitators): identify key areas of the trial staff feel unsure about and where additional training can be given with the aim to improve future training sessions.

Summary The ATLAS study Face-to-face training considered most beneficial Not practical in all trial settings Planning & Evaluation of trial-specific training = Best practice Not routinely done ATLAS training toolkit Developed to support trial managers Reference document encouraging active thinking of staff training Further evaluation needed To summarise, the ATLAS study showed that f-2-f training was considered most beneficial however it might not be practical for all trial settings Early planning and evaluation of trial-specific training were considered best practice, however, they are not routinely done To address these gaps, we designed the ATLAS training toolkit which was developed and act as a reference document encouraging active thinking of training. However, further evaluation is needed.

Acknowledgements PhD Supervisors: Studies taking part in ATLAS The ATLAS study PhD Supervisors: Prof. Chris Rogers Dr. Athene Lane Dr. Marcus Jepson Dr. Caroline Wilson Studies taking part in ATLAS I would like to thank my supervisors, all studies taking part in ATLAS as well as the MRC conduct hub for supporting my work. MRC ConDuCT-II Hub for Trials Methodology Research Contact: A.Gravani@Bristol.ac.uk