The research challenges of large scale RCTs with volunteers and volunteering organisations Facilitator: Dr Matt Ryan Presenter: Professor Peter John

RCTs, trials, randomised evaluations, experiments Randomly allocate something to a group and compare with a group that does not get it Needs to be staged and for certain things to be in place – randomisation, and tying of outcomes to random allocation See diagram Unbiased estimate of a programme Considered to be the gold standard They are hard to do - worth it, and lots of leaning and interactions with researchers to the benefit of all

Figure 7.1 Simple Random Allocation Design Simple Random Allocation Design

Key distinguishing features of RCTs Research needs to deliver or have delivered an intervention at the same time as research So more activities to carry out as well as measurement Need for effective coordination of activities Real world constraints need to be anticipated Need to ensure procedures follow special conditions of a trial – attrition, selection etc.

Interactive design: The Glass Bead Game Step 1: Assessment of fit for a RCT Step 2: Recruitment of partner(s) Step 3: Selection of the research site and subjects Step 4: Determination of the treatments Step 5: Specification of control condition Step 6: Calculation of sample size Step 7: Agreement on procedure for random allocation Step 8: Collection of data on the subjects Step 9: The preparation of research plans Step 10: Assessment of ethical principles

Threats to experiments Problem 1: Loss of subjects Problem 2: Failed randomization Problem 3: Treatment not given or modified Problem 3: Interference between treatment and control Problem 4: Unavoidable confounds Problem 5: Poor quality data Problem 6: Loss of cooperation of partners Problem 7: Unexpected logistical challenges and expenses

Volunteering experiments We are the first of these (bar one) Sector has had a lot of attention but with giving e.g. experiments on matching (see BIT), e.g. on JustGiving Volunteering is different – more lumpy, differentiated, hard to match demand and supply Need to recruit volunteering organisations – really enthusiastic and interested in the methods – issue was the practical constraints RCTS need: enough numbers good data and its management those who understand experiments

Some reactions to the offer: no fear of white coats!

Politicophobia? Concerns about mentioning politicians, especially with governing boards

Data protection and management Need to ensure transfer of the data is legal, sensitive – university example Managing data

12

Recruitment to the project, volunteering High attrition to staying in the study Just getting volunteers to volunteer!

Measuring outcomes Hard to find records of the volunteers, information not routinely kept or easily found

Summing up It is possible to get the experiments to work, but new grounds – so needed our great partners to make it happen Might be easier now if this knowledge disseminates Also experience generally is growing about RCTs >> helps volunteering experiments Can we make it easier, get better quality data