Implementing and Running an online Trial A Case Study Using The REACT Trial Duncan Appelbe1,2, Heather Robinson3, Susie Dodd1,2, Andrew Walker3, Paula Williamson1,2 and Fiona Lobban3 1: Clinical Trials Research Centre, The University of Liverpool 2: North West Hub for Clinical Trials Methodology Research, Department of Biostatistics, The University of Liverpool. 3: SPECTRUM Centre for Mental Health Research, Lancaster University

Disclaimer The REACT project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA), 14/49/34. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HTA, NIHR, NHS or the Department of Health.

Participants are the Relative or close friend What is REACT? REACT is an online study to compare the effectiveness of a Relatives Education And Coping Toolkit (REACT) with an online Resource Directory. Half the people in the study will receive the REACT toolkit, and half will receive the Resource Directory. Participants are the Relative or close friend All Online !!!!

Participant Journey Through the Trial – Getting to the site Participants directed to the REACT site from: Posters in Clinic Promotion by Mental Health Charities Facebook adverts Internet Search Support Groups

Participant Journey Through the Trial – Getting Randomised Check Geographic area Online form, translated to word then E-Mailed Check that we have a working contact number Require Baseline Questionnaires to be completed prior to randomisation

Participant Journey Through the Trial – Access to the arms TAU REACT

Participant Journey Through the Trial – Follow up 12 Weeks x3 2x2 factorial £10/20 (un)conditional 24 Weeks

& How did we get here …? Typical Trial Team CI Trial Manager Data Manager IT External Supplier for intervention site Intervention site based on data collected in a Pilot Trial &

Anything but straight forward …. Identifying an external web design company to design and implement the intervention site, within the trial budget Would also host site Could not implement study questionnaires Using typical trial databases would not be appropriate as they would turn off the subjects Intervention Site – WordPress Registration/DC Site: C# MVC, MySQL

Collect data …. Not just PROMs Need to know how many people stop at each stage Need to know why people have failed to progress Started off with UA (linked to a user) Data showed that events were missed, leading to a bespoke implementation e.g. in a test of ~10 minutes interaction with ~30 page changes GA reported ~20% only!!

Collect data …. Not just PROMs Recorded how much of a video participants watch - Videos on YouTube - Utilise YouTube JS API to determine percentage watched - AJAX call to server every 5s on a page with a video Call limited to pages with videos, otherwise we could overwhelm server, but more importantly Would effect data usage from participant Currently ~1M records recorded

Linking the two sites AJAX Calls to C# end points sharing data via JSON Secured using a secret key that changes on an hourly basis Password reset Reminder E-Mails

Automated Alerts …. Need to send “info” if subjects answer GHQ or wellbeing answers are specific values – advice that would normally be given if face to face Timed alerts if user is “stuck” at a given point for too long Alerts to supporters on new posts

Study Dashboard

68% 65% Where are we up to ..? 1st Randomisation April 2016 Recruitment to date 546 out of 666 12 24 68% 65%

Lessons Learnt Focus Groups are important – But You need to know when to say stop changing things You need to redress the balance of the trial team – more IS specialties involvement This also includes the TMG!

Lessons Learnt Think about what data you want in advance Think about what metrics you need to help you manage the study

The Good news … Online trials still have challenges, but can be rewarding Caveat : Think about the demographic!

Acknowledgements Fiona Lobban Susie Dodd Heather Robinson Lizzi Collange Sue Flowers Andrew Walker Susie Dodd Paula Williamson Naomi Bacon Emma Bedson Clare Jackson