New Cochrane risk of bias tool for cluster randomised tools ICTMC May 2017 New Cochrane risk of bias tool for cluster randomised tools Sandra Eldridge Pragmatic Clinical Trials Unit Centre for Primary Care and Public Health Marion Campbell (Aberdeen) Michael Campbell (Sheffield) Amy Drahota (Portsmouth) Bruno Giraudeau (Tours) Jeremy Grimshaw (Ottawa) Julian Higgins (Bristol) Barney Reeves (Bristol) Nandi Siegfried (Cape Town) Title Authors Centre for Primary Care & Public Health Blizard Institute I am director of the pragmatic clinical trials unit at Barts and the London school of medicine and dentistry. The core staff in the unit are located in the brick building in this photograph which is the Centre for Primary Care and Public Health, and that in turn is part of the Blizard Institute which is mostly located in the large glass building that our building is reflected in.

Outline Why is this work important? Methods and scope Focus of this talk: Allocation concealment Blinding of participants Missing clusters Timing of identification and recruitment of individual participants/randomisation https://sites.google.com/site/riskofbiastool/

Cluster randomised trials 708 trials in last year, only 252 in 2011 In recent review of systematic reviews about 10% include cluster randomised trials

Issues in including cluster randomised trials in systematic reviews Analysis not accounting for clustering Difficulty interpreting bias Incorrect weighting when included in meta-analysis (usually given too much weight) Studies may be deemed as high or low risk of bias incorrectly

Methods and overall results Sub-group of group updating Cochrane risk of bias tool – ROB v2 Mostly teleconferences and emails over a long period of time! Adaptation of ROB v2 Used expertise and personal collections of trials Piloting amongst group and externally Continuous referral back to main updating group Of 5 domains in updated main tool: Alterations to three domains Added one new domain

Scope Clusters are schools, villages, medical practices, hospitals, families etc. Clusters are individuals with observations made on, for example, teeth, eyes etc. Stepped-wedge designs in which units of randomisation are clusters

Outcomes and participants Tool assumes risk of bias being assessed for one outcome at a time For cluster randomised trials “participants” = individuals on whom it is planned to collect the outcome of interest IRIS trial Clusters: general practices Intervention: to increase identification and referral for victims of domestic violence No individual participant recruitment Data collection: routine records of all women in practice aged 16 plus Open access urology trial Clusters: general practices Intervention: guideline-based open  access urological investigation service Outcome: general practitioners’ compliance with referral guidelines Outcome: waiting time for patients’ referral

Allocation concealment Key fact: clusters can be randomised sequentially, in batches, or all at once IRIS trial Clusters randomised sequentially Cluster identified, details emailed to allocator, allocator emailed back allocation status Diabetes manual trial Clusters randomised in batches Stratification factors largely aggregated participant baseline data Comments: Less easy to subvert randomisation in cluster randomised trials More difficult to judge whether baseline imbalances are indicative of problems with randomisation process

Likely that information reported is limited Lack of blinding of participants and others who could affect delivery of intervention Those with opportunity to introduce deviations to the intervention might have little inclination to do so The more complex the intervention the more difficult to identify such deviations Likely that information reported is limited Key fact: Interventions often multifaceted, aimed at various participants and participants and others cannot be blinded Do participants (for the outcome of interest) know they are in a trial? (sometimes no) If they know they are in a trial, are they aware of their assigned intervention? (usually yes) Are there other people involved in the intervention who know the allocation of participants? (often yes) If participants or others are aware of assigned intervention did this affect the way the intervention was delivered beyond what one would expect in usual practice?

Example of algorithm for judging risk of bias

Missing clusters Key fact: Missingness can occur at both cluster and individual level Comment: Need to be careful about interpretation of bias because will depend on reasons for missingness at both levels

Timing of recruitment and randomisation ukBEAM pilot Aim: To improve back pain Clusters: UK General Practices Intervention: offer of exercise classes, physiotherapy etc. Control group: 66 recruited Intervention group: 165 recruited, suffering from milder back pain Explanation: participation in the trial very attractive in intervention arm Cluster ‘consent’ Cluster randomisation BIAS e.g. ukBEAM pilot study Participant consent to randomisation? Consent Randomisation UKBEAM 66 in control 54% of predicted, 165 in intervention, 41% more than predicted In the UKBEAM trial pilot study (table 1.4), 26 practices were randomised to offer active management or usual care to patients presenting with low back pain. Patients within the active management arm were also individually randomised to receive spinal manipulation, exercise classes or advice alone. After one year practices in the control arm (traditional care) had recruited 66 patients, 54% of the number predicted based on practice list size, while those in the active management arm recruited 165 patients, 41% more than predicted. In addition participants from the active management arm were suffering from milder back pain than those in control practices. It is likely that the offer of exercise classes or physiotherapy made participation in the trial an attractive option for the GPs and their patients in the active management arm, whereas there was such benefit for patients in the control arm. Following the pilot study, the trial was redesigned as an individually randomised trial comparing different methods of delivering active management. Participant consent to data collection, participation

Timing of recruitment and randomisation Key fact: The timing of cluster randomisation, participant identification and participant recruitment (if relevant) can be a potential source of bias in cluster randomised trials in a way that is not possible in individually randomised trials New domain: Timing of identification and recruitment of individual participants in relation to timing of cluster randomisation

Scenario 4 (identical to 6) Scenario 6 (identical to 4) Cluster randomization Identification of potential individual participants Identification of individual participants Recruitment of individual participants Participants not directly recruited Potential for identification/recruitment bias although this could be avoided through trial design No potential for identification/recruitment bias because randomization happens after UK BEAM pilot (Farrin et al 2005)

Two further examples in which identification/recruitment bias possible Scenario 2: Feeding strategies for critically ill patients in intensive care Clusters: Intensive care unit (ICU) wards Intervention: Guidelines developed by ICU staff Outcome: Hospital discharge mortality Participants not directly recruited but identified by ICU staff (though no evidence of bias) Scenario 3: Hip protectors for preventing hip fractures Clusters: Elderly care units within community based health centres Participants identified prior to randomisation but approached after randomisation Recruited: 31% in intervention and 9% in control group

Conclusion In cluster randomised trials: Less easy to subvert randomisation but more difficult to judge if this has happened Lack of blinding could potentially cause bias but, where appropriate, participants not knowing they are in a trial can protect against this Clusters are less likely to be missing than individuals but if they are potential bias may be more difficult to interpret The timing of cluster randomisation, participant identification and participant recruitment (if relevant) can be a potential source of bias Authors and peer reviewers should be aware of key elements to include in their reports to reassure readers regarding bias