Evidence based parenting programmes – examining the effectiveness of a large scale implementation NFPF/NERA Congress, University of Aarhus, Copenhagen, 8-10 March 2012. ‘School-Home Cooperation’ network Dr Stephen M. Cullen Mrs Mairi-Ann Cullen The Centre for Educational Development, Appraisal & Research (CEDAR), The University of Warwick, England.

Evaluation teams Lindsay, G., Strand, S., Cullen, M.A., Cullen, S., Band, S., Davis, H., Conlon, G., Barlow, J., Evans, R., Hasluck, C., & Stewart-Brown, S.

Overview Evidence based parenting programmes: efficacy and effectiveness The Parenting Early Intervention Programme (PEIP) in England (2008-2011) The national evaluation of the PEIP Identifying key factors in the successful large-scale implementation of the PEIP Some comparisons with parent management training in Norway (1996-2006)

1. Efficacy and Effectiveness There is extensive evidence that parenting programmes can have positive effects on parent outcomes and children’s well-being and behaviour Overviews of the evidence include: United Nations Office on Drugs & Crime (UNODC) Compilation of Evidence-Based Family Skills Training Programmes (2010); which covers 13 evidence-based programmes The evidence for the success of programmes is, usually, based on scientific testing utilising randomized controlled trials (RCTs). RCTs provide evidence of the efficacy of programmes However, when programmes are implemented on a large scale, optimum conditions (as for RCTs) are not present, as programmes are expected to meet public needs, not research standards. What is needed in these circumstances are evaluations of the effectiveness of programmes.

2. The Parenting Early Intervention Programme (PEIP) The Pathfinder (PEIP) ran, in England, from September 2006-March 2008. Government provided £7.6million of funding to 18 Local Authorities (LAs) to implement one of three evidence-based parenting programmes for children aged 8-13. The pathfinder was followed, from 2008-2011, by the Parenting Early Intervention Programme (PEIP) for all English LAs. Roll out accomplished in two stages, Wave 2 (from 2008) & Wave 3 (from 2009), with a further two evidence based programmes being added to the original three. The programmes were: Triple P; Incredible Years (school age); Strengthening Families, Strengthening Communities; FAST; Strengthening Families 10-14.

3a. The Evaluation The evaluation of the three stages of the PEIP roll out was carried out by CEDAR, the University of Warwick. The evaluation of the first stage – Wave 1 – was reported on in 2008 (Lindsay et al 2008). The evaluations of Wave 2 (2008-11), and Wave 3 (2009-11) were reported on in 2010 & 2011 (Lindsay et al, November 2010 and May 2011) The evaluations adopted a mixed methods approach, using quantitative and qualitative methods.

3b. Methodology The data underpinning this paper is drawn from both types of data sources undertaken with 22 LAs from Wave 2, & 17 LAs from Wave 3 Quantitative data: the data relevant to this paper were drawn from 6,143 completed pre and post course parent questionnaires and questionnaires completed by 1277 parenting programme facilitators, giving details of their qualifications and previous parenting programme experience Qualitative data: 429 interviews were undertaken, of which: 73: Local Authority (LA) strategic leads 92: LA operational leads 13: LA strategic and operational leads (combined role) 77: parenting programme facilitators 83: other parenting support professionals 16: school representatives 75: parents (mainly mothers)

3c. The outcomes for parents & children Parents completed pre and post course questionnaires consisting of established self-reporting measures which assessed: Parental well-being: Warwick-Edinburgh Mental Well-being Scale Parental laxness: Parenting Scale Parental over-reactivity: Parenting Scale Children’s behaviour: Strengths & Difficulties Questionnaire By the end of the PEIP parenting group 74% showed a reduction in parenting laxness 77% showed a reduction in the score for over-reactivity 79% showed an increase in mental well-being The % of children with significant behaviour problems fell from 56% to 38% The % of children whose behaviour difficulties had a substantial impact on the family fell from 62% to 36% And these results remained largely the same after one year

4. Effectiveness The parent & child outcomes showed that, overall, the PEIP parenting programmes were effective in this large-scale roll out. Effective = combining ‘reach’ in terms of numbers of programmes delivered + programmes leading to good quality outcomes. Variations in LA effectiveness – combined methods analysis What were the factors that underpinned effectiveness? A) The question of organisational models – was there a single successful model? B) The role of leadership C) The recruitment of parenting programme facilitators D) Links with other initiatives E) Engaging parents

A) Models of organisation LAs were free to organise the PEIP to suit local contexts, but essentially there were three basic models of organisation: i) a core team of facilitators ii) a multi-agency approach iii) commissioned out to a third sector provider No single model emerged as being the best way of delivering the PEIP. The important element was that the model chosen matched local conditions.

B) The role of leadership A number of key elements were central to the successful roll out of the PEIP in LAs. Although models of local organisation could be suited to local needs, leadership was most successful when: There was a single operational lead in post, co-ordinating daily work on PEIP & possessing an overview of the roll-out There was a strategic level lead in a high-level post in the LA structure The strategic lead role was important to support PEIP at the highest levels of LA decision-making, to argue for the place of PEIP among competing priorities, and to support the operational lead.

C i)Parenting programme facilitators Facilitators were drawn from a wide variety of backgrounds in terms of their level of education & experience Key success criteria were: workload capacity to deliver programmes having qualities, skills, knowledge & experience to enhance parents’ experience of the programmes. One view was that facilitators should have degree level qualifications in relevant disciplines, e.g., psychology however, this proved to offer no advantage; in fact, lead facilitators who were not educated to this level were associated with greater improvement than graduate lead facilitators in terms of parental mental-wellbeing

C ii)Parenting programme facilitators The facilitator factors that contributed to quality outcomes were: diversity among facilitators: age, ethnicity, employment background, educational level etc range of skills (e.g., group skills), and personal & inter-personal qualities (e.g., empathy, respectful) good quality programme training & additional training (e.g., group skills) where needed investment in pre-group engagement of parents (e.g., visits) supervision to support their delivery of programmes All this supported recruitment, engagement & retention of parents.

D) PEIP links to other initiatives at LA level It was important that: a variety of local agencies were involved in delivering the PEIP, thereby enhancing sustainability – this was facilitated were LAs already had a multi-agency approach to working non-LA partners, including third sector bodies were able to be involved in delivery of the PEIP.

E) Recruiting parents to the programmes The PEIP was conceived as a targeted initiative focused on parents of children at risk of anti-social behaviour and poor life outcomes. However, LAs chose not to restrict recruitment: parents recruited from a range of backgrounds; some referred by courts, others responded to universal advertising about the PEIP. overall, delivery was weighted towards the targeted group group programmes not suitable for all parents; some given one-to-one support first (or instead).

5(i) England and Norway - comparisons Norway provides a similar case of the efficacy and effectiveness question The implementation of Parent Management Training, Oregon Model (PMTO), 1999-2006 The PMTO roll-out in Norway possessed a number of similarities and differences with the PEIP in England PMTO reported by Ogden et al (2005).

5(ii) PMTO Norway & PEIP Similarities in approach: National governments making decision to roll-out Central funding for evidence based programmes (EBP) Role for local implementation Evaluation research on the implementation process

5(iii) PMTO Norway & PEIP Differences, for example: Norway: one programme; England: 3-5 Norway: a national implementation & research centre – a permanent centre: ‘The Behaviour Centre’ Norway: comprehensive recruitment & training programme; England left to LAs in conjunction with the programme originators Norway: clinical outcome research undertaken; in England, this was a part of the national evaluation Norway: costs split between centre & local; in England, all initial costs funded by central government, with local sustainability for future.

Finally… Time for your comments and questions We are very keen to learn more about parenting support initiatives in Scandinavian countries We are also keen to develop dialogue on parenting support and the role of the state S.M.Cullen@warwick.ac.uk M-A.Cullen@warwick.ac.uk

References: Lindsay, G., Strand, S., Cullen, M.A., Cullen, S., Band, S., Davis, H., Conlon, G., Barlow, J., & Evans, R., (2011) Evaluation of the Parenting Early Intervention Programme, Research Report DFE-RR121 (a) London, Department of Education (DFE) Lindsay, G., Strand, S., Cullen, M.A., Band, S., & Cullen, S. (2010) Parenting Early Intervention Programme 2nd Interim Report, DFE-RR047, London, DFE Lindsay, G., David, H., Strand, S., Band, S., Cullen, M.A., Cullen, S., Hasluck, C., Evans, R., & Stewart-Brown, S. (2008) Parenting Early Intervention Pathfinder Evaluation, Research Report RW054, London, Department of Children, Schools & Families (DCSF) Ogden, T., Forgatch, M.S., Askeland, E., Patterson, G.R., Bullock, B.M. (2005) ‘Implementation of Parent Management Training at the National Level: the Case of Norway’, Journal of Social Work Practice, 19:3, 317-329 United Nations Office on Drugs & Crime (UNODC) (2010) Compilation of Evidence-Based Family Skills Training Programmes, UN, New York