Session 3: Trial management Sarah Miller (Queens, Belfast) Laura Dunne (Queens, Belfast)

RCT MANAGEMENT: KEY ISSUES AND LESSONS LEARNED… Dr Laura Dunne & Dr Sarah Miller

Centre for Effective Education | SCHOOL OF Education “An ounce of prevention is worth a pound of cure” Related issues: Recruitment –achieving required sample and power Retention –Treatment attrition –Measurement attrition Communicating with schools and developers Introduction:

Centre for Effective Education | SCHOOL OF Education RCT evaluation of Barnardo’s Ready To Learn afterschool programme: 16 schools and 8 data collection sweeps over three years RCT evaluation of a classroom based social emotional learning programme: 74 schools and 5 data collection sweeps over four years Two examples…

Centre for Effective Education | SCHOOL OF Education The programme: Aims to raise achievement by focusing on children’s early literacy skills and enhance parental engagement in children’s learning Programme runs during term time for one hour after school on Tuesday, Wednesday and Thursday afternoons Activities in the programme include story telling, drawing, singing, puppets, talking and listening RCT Evaluation of Barnardo’s Ready To Learn Afterschool Programme

Centre for Effective Education | SCHOOL OF Education The evaluation: A cluster randomised design combining outcome data with experiences of participants (over three years, 8 data sweeps) 16 schools, 564 Year 1 pupils (9 intervention; 7 control) Measures Battery of age-appropriate tests, focus groups and parental interviews Principal interviews RCT Evaluation of Barnardo’s Ready To Learn Afterschool Programme

Centre for Effective Education | SCHOOL OF Education The Programme: A universal programme delivered on a whole-class basis for one academic year 27 lessons based around 9 themes delivered by a trained RoE instructor Lessons revolve around a monthly classroom visit by an infant and parent The parent-infant visit serves as a springboard for discussions about understanding feelings and infant development and effective parenting practices RCT evaluation of the Roots of Empathy programme

Centre for Effective Education | SCHOOL OF Education The Evaluation: Cluster randomized controlled trial: 74 primary schools randomly allocated to control and intervention groups Schools in the intervention group receive the programme in their selected Year 5 class for one academic year Schools in the control group do not receive the programme but receive it the following year RCT evaluation of the Roots of Empathy programme

Centre for Effective Education | SCHOOL OF Education School recruitment Actively engage with schools re: both the programme and the methodology e.g. attend information sessions, meetings with service and Principals. Make it clear to schools what their participation will involve (MoU) Meetings with service and Principals Consider a wait list control Financially compensate for teacher time Consent prior to randomisation Key issues

Centre for Effective Education | SCHOOL OF Education Retention Regular communication: newsletters each term, study updates, Christmas cards, annual principal interviews, school visits Minimise disruption and be flexible - pilot measures, testing P1-3, stickers Document reasons for dropout and monitor attrition between intervention and control groups If a school drops out of the intervention, continue to collect data (ITT) Process evaluation: observation and field notes Key issues

Centre for Effective Education | SCHOOL OF Education Communication Make clear to schools who in which team is responsible for each element of the study (MoU), provide a timetable of events Identify a key contact to liaise with in the school Minimise the number of different people the school has to deal with Agree a research plan and be clear who is responsible for what Regular (and efficient) communication between programme developer(s) and research team (Skype, etc) Set realistic expectations of the possible outcomes of the evaluation Key issues