1. The Campbell Collaborationwww.campbellcollaboration.org Some challenges in reviewing the effectiveness of behavioural group-based parenting programmes for childhood conduct problems Mairead Furlong 1, Sinead McGilloway 1, Tracey Bywater 2, Judy Hutchings 3, Susan Smith 4, Michael Donnelly 5 Institutional affiliations: 1 National University of Ireland Maynooth; 2 University of York, England; 3 Bangor University, Wales; 4 Royal College of Surgeons in Ireland; 5 Queens University Belfast, N.Ireland Cochrane Colloquium 2012

2. The Campbell Collaborationwww.campbellcollaboration.org Overview Background – why the review was conducted Brief overview of inclusion criteria and results of review Some methodological challenges

3. The Campbell Collaborationwww.campbellcollaboration.org Background Conduct problems in young children are common – 5% to 10% in UK, US and Europe Rates in socially disadvantaged areas -> 20% to 35% Negative long-term psychological, social and economic consequences (e.g. Farrington, 2007) Evidence that group-based parenting programmes, based on behavioural and social learning principles may reduce conduct problems (e.g. Webster-Stratton, 1988; Sanders, 2000)

4. The Campbell Collaborationwww.campbellcollaboration.org Background – cont’d Previous systematic reviews have been problematic with regard to: - lack of statistical meta-analyses and defined against established criteria as ‘effective’ (eg. Brestan & Eyberg,1998; Mihalic et al.2002) - failure to report heterogeneity and confidence intervals - no intention-to-treat or sensitivity analyses undertaken - results of group-based behavioural programmes combined with other types of parenting programmes (e.g. individual programmes; different theory base; those with adjunctive treatments) Thus, the evidence for behavioural group-based parenting programmes remains unclear

5. The Campbell Collaborationwww.campbellcollaboration.org Objectives and inclusion criteria  To examine the effectiveness and cost-effectiveness of behavioural and cognitive-behavioural group-based parenting programmes for children (aged 3-12 year) with early-onset conduct problems in improving child conduct problems, parental mental health and parenting skills Studies were included if: (1) Randomised or quasi-randomised controlled trial; (2) Based on parent interventions only (i.e. did not involve children); (3) Incorporated an intervention group versus a waiting list, no treatment or standard treatment control group. Excluded parent programme as control

6. The Campbell Collaborationwww.campbellcollaboration.org Inclusion/exclusion criteria Included studies that: Used at least one standardised instrument to measure clinically significant child conduct problems. Reported on conduct problems co-morbid with ADD and ADHD if they reported outcomes for conduct problems separately from ADD and ADHD outcomes Excluded studies that involved children with comorbid physical and intellectual impairments (e.g. ASD, Down Syndrome, tic disorders, significant language delay and learning problems) Only included costs or cost-effectiveness studies conducted alongside, or subsequent to, RCTs that met our eligibility criteria

7. The Campbell Collaborationwww.campbellcollaboration.org  13 effectiveness studies met inclusion criteria – 10 full RCTs, 3 quasi- RCTs; two cost-effectiveness studies included (Incredible Years studies)  1078 participants (646 intervention, 432 control)  5 studies conducted in USA, 7 in Europe and 1 in Australia  In 6 studies, children had a diagnosis of Conduct Disorder; in 7 studies they scored above clinical cut-off on a validated measure of childhood conduct problems  9 studies were Incredible Years; 1 Triple P; 1 Barkley’s Parent Training Programme; 1 Parent Management Training and 1 Comet Parent Management Training

8. The Campbell Collaborationwww.campbellcollaboration.org Results cont’d All measures were short-term outcomes Moderate statistically significant improvements in child conduct problems, whether assessed by parents (SMD -0.53; 95% CI -0.72 to - 0.34) or independently (SMD -0.44; 95% CI -0.77 to -0.11) Sub-moderate improvements in parental mental health (SMD -0.36; 95% CI -0.52 to -0.20) and moderate improvements in positive parenting skills

9. The Campbell Collaborationwww.campbellcollaboration.org Results maintained across different settings (research and service), levels of disadvantage, diagnostic categories, and within services demonstrating high implementation fidelity Results remained robust to all but one of the sensitivity analyses Insufficient evidence to test for child internalising problems and cognitive/educational performance. The parent-training cost approximately $2500 (£1712/€2217) per family to bring the average child with clinically significant conduct problems into the non-clinical range.

10. The Campbell Collaborationwww.campbellcollaboration.org Challenge 1 Were we right to exclude studies that did not provide evidence of children having clinical levels of conduct problems? No: Exclusion of preventative studies; Exclusion of studies which indicated the inclusion of children with conduct problems but provided no evidence Yes: Other reviews had combined preventative and clinical studies -> but results for programmes were unclear. We had a more focussed question

11. The Campbell Collaborationwww.campbellcollaboration.org Challenge 2 Were we right to exclude children with comorbid physical and intellectual impairments? No – ‘real world’ research suggests conduct problems often comorbid with learning difficulties Yes - none of the excluded studies had minor comorbid learning difficulties but rather Downs Syndrome, Autism etc. Condition under investigation changes if those with severe impairments are included Already too many sub-groups – precluded any subgroup analysis based on children with mild learning/physical problems.

12. The Campbell Collaborationwww.campbellcollaboration.org Challenge 3 Should we have included all children with ADHD as it’s often comorbid with conduct problems? Evidence on both sides to say that CP and ADHD overlap - but also different aetiology and treatment… Compromised => included studies with diagnosed ADHD if comorbid with conduct problems and if they reported separate outcomes for conduct problems and attention/hyperactivity Five included studies reported a low level of comorbidity with hyperactive symptoms; excluded in Zwi review as focus on CP

13. The Campbell Collaborationwww.campbellcollaboration.org Challenge 4 How do you include all measures of an outcome within a study into the review? Typical advice in handbook….. Parent TrainingControl Std. mean difference Study or subgroupMeanSdTotalMeanSdTotalWeightIV, Random, 95% CI Gardner 2006 66.729.93473.534.72622.30%-0.55 [-1.07, -0.03] Hutchings 2007a 122.335.1104144334918.00%-0.63 [-0.97, -0.28] Kling 2010 34.925.658139.845.94015.80%-0.77 [-1.19, -0.35] Martin 2003 99.8822.3916126.0928.111117.70%-1.02 [-1.85, -0.20] McGilloway 2009 121.340.7103144.933.24617.70%-0.61 [-0.96, -0.25] Webster-Stratton 1984 106.5120.3313138.9120.33117.70%-1.54 [-2.47, -0.61]

14. The Campbell Collaborationwww.campbellcollaboration.org What would you do? Some other Cochrane reviews - included only the most commonly used measure (e.g. the Eyberg Child Behaviour Inventory)… We were wrong - focused only on means and SDs for continuous outcomes! Calculate mean effect size and SE for each individual outcome within each study; enter using generic inverse variance method (using SMDs) ParentControl Std. mean difference Study or subgroupStd. mean differenceSETotal WeightIV, Random, 95% CI Barkley 2000 0.150.2206394215.30%0.15 [-0.28, 0.58] Hutchings 2007a -0.430.17811044918.00%-0.43 [-0.78, -0.08] Kling 2010 -0.70.2128584015.80%-0.70 [-1.12, -0.28] McGilloway 2009 -0.590.18331034617.70%-0.59 [-0.95, -0.23] Total (95% CI) 451276100.00%-0.50 [-0.76, -0.24]

15. The Campbell Collaborationwww.campbellcollaboration.org Challenge 5 Reviewing costs studies = reviewing ‘in the dark’? Completed both the ‘Drummond’ and ‘Evers’ checklists Read all reviews conducted by Ian Shemilt and ‘Chapter 15’ of Handbook A lack of standardised risk of bias table for costs studies Key areas for improvement in costs studies in this area: Need to adopt a ’complex intervention approach’ – wider range of costs/benefits Reporting measures of variance for all parameters Clearly delineating frequency of resource use from unit costs Providing a more detailed description of the comparator and associated costs Selecting outcome measures that can be compared with other studies

16. The Campbell Collaborationwww.campbellcollaboration.org Conclusion: Other miscellaneous challenges and issues Inference challenges – lack of control group at long-term follow up Did not include qualitative research which can help to clarify adverse outcomes Our control group did not include other parenting programmes; did have treatment as usual…what should be in the control group? Not a challenge Did not expect to receive as much help from authors of included (and other) studies so thank you! Rewards of conducting a review in this area

17. The Campbell Collaborationwww.campbellcollaboration.org With special thanks to… The Health Research Board in Ireland who funded the review (Cochrane Fellowship Scheme) The Cochrane Developmental, Psycho-social and Learning Problems group and most especially: Geraldine MacDonald, Laura MacDonald, Chris Champion, and Jane Dennis Anonymous peer reviewers for their helpful feedback The Campbell Collaboration

18. The Campbell Collaborationwww.campbellcollaboration.org P.O. Box 7004 St. Olavs plass 0130 Oslo, Norway E-mail: info@c2admin.org http://www.campbellcollaboration.org