POSITIVE CLASSROOMS, POSITIVE CHILDREN Evaluating the Incredible Years Teacher Classroom Management programme in Ireland Dr Sinead McGilloway Principal Investigator Mental Health and Social Research Unit Department of Psychology NUI Maynooth

Overview The Incredible Years Ireland Study: our role and who we are Study background: - EBD in the classroom - research objectives/questions - study design Outline of key findings Conclusion

The research evaluation: Our role To undertake a 4-year national community-based evaluation of the Incredible Years programme in Ireland: - Parent (BASIC) training - Teacher Classroom Management training** - Combined parent and small group (Dina) training To work collaboratively with Archways and other community- based organisations and schools in executing the research To generate high quality findings that will impact on policy and practice in Ireland (and elsewhere) -> improve the lives of children with EBD and their families

The Research Team NUI Maynooth Dr Sinead McGilloway Lynda Hyland Dr Grainne NiMhaille Yvonne Leckey Paul Kelly Prof Donal ONeill Mairead Furlong Church of Ireland College of Education Dr Anne Lodge Bangor University, Wales Dr Tracey Bywater Trinity College Dublin Prof Catherine Comiskey Queens University Belfast Dr Michael Donnelly

EBD in the classroom Child conduct problems in school academic under- achievement, poor attendance and early school leaving poor mental health, juvenile delinquency Impact on the teachers ability to teach the class effectively Negative impact on classroom environment and the teacher-child relationship Can affect teacher job satisfaction, stress and likelihood of burnout DES survey (2006): 32% of trainee teachers had fair or ineffective classroom management skills

The Incredible Years TCM programme Social and behavioural learning principles Help teachers promote pro-social behaviour Reduce undesirable and aggressive behaviour Establish more positive relationships with pupils and facilitate peer-to-peer bonding Encourage teachers to collaborate with parents and promote parent involvement in school (Webster-Stratton, 2005) The intervention A brief group-based programme Five one-day sessions (each month) Videotape modelling, role plays and discussions

Research questions 1.Does the IY TCM programme improve classroom management competencies and provide skills to better manage challenging behaviour? 2.To what extent does the TCM programme improve child behaviour in the classroom? 3.How do outcomes for teachers and children change over time? 4.What are the experiences of teachers and which factors facilitate or inhibit the effective implementation of the programme? 5.How cost-effective is the programme?

Research Design Three separate studies: 1.Main study: A clustered Randomised Control Trial (RCT) design (intervention and control group) 2.A smaller qualitative study assessing experiences and views of teachers who took part in the RCT 3.A cost analysis to assess the cost-effectiveness of the programme

Method 1. RCT 22 teachers from 11 schools in Limerick area (445 children aged 4-7yrs from Junior and Senior Infant classes; 53% female) - Child behaviour (teacher & researcher-completed) - Teacher behaviour and self-report Initial screening - low, medium and high EBD (n=217) Assessed at baseline (pre-intervention) and 6 months later 2. Qualitative sub-study 1:1 interviews with 11 teachers from 6 schools (baseline and 6- mth follow-up only) Thematic analysis 3. Cost analysis Detailed cost data on: (1) non-recurrent initial training costs of facilitators; (2) pre-delivery cost of programme; (3) group training costs; and (4) teacher substitution costs outcome data

RCT Flow Diagram Schools (n=12) in the greater Limerick area approached by Archways to participate in the research Names of 2 other schools provided to the research team by Archways. Both consented to participate Three schools were unable to participate (reluctant to participate [n=2] and time concerns [n=1]) Nine schools provided informed consent All teachers completed and returned questionnaire measures T-SDQ child scores computed and index children identified. Parental informed consent obtained Randomisation process conducted; 11 intervention group teachers and 110 index children; 11 control group teachers and 107 children Baseline (T1) observations carried out in all classrooms (n=22) with all teachers (n=22) and index children (217) Intervention group teachers (n=11) receive TCM training (110 Index children ) Follow-up (T2) assessments conducted immediately after TCM training completed (ie. 6 months after baseline assessment). Total number of participating schools (n=11), teachers (n=22), children (n=445) 234* index children were selected for observation from the initial sample of 445 on the basis of their total SDQ scores * 17 children were lost to follow-up.

Mean SDQ Total Difficulties scores across the sample range

Key findings: 1. RCT Baseline: 26% of children showed significant social, emotional and behavioural difficulties at baseline (SDQ) Several statistically significant improvements in pre-post intervention teacher behaviour: - use of fewer negative classroom management strategies - give children more time to comply with questions/instructions - positive upward trend in positive classrm management strategies Increase in the intervention group in self-reported frequency of use and perceived usefulness of positive classroom management strategies All teachers reporting feeling more confident in their ability to manage conduct problems

Teacher no opportunity for Intervention and Control Groups

Child outcomes Children in intervention group: - showed significant reductions in emotional symptoms - positive trends with respect to overall behaviour Those most at risk showed most significant improvements: - overall behaviour - peer problems - general distress and social impairment

SDQ Total Difficulties score for high risk children

2. Qualitative study (Lynda Hyland) Theme 1: Reality of the classroom environment All teachers reported significant behavioural challenges in their classrooms; some also reported physical and verbal aggression: …he has thrown things behind his head and hit children with them. And he has even been violent towards adults as well too, children and adults… numerous teachers Id say at this stage have gotten kicked and punched when he lashes out. (T1) Teacher stress: Some days I went home and just cried, because I was just so exhausted and frustrated with the situation. Because, you know, I could see it in the other children, and I could see it in the other parents as well. (T4) Reluctance to seek support

Theme 2: TCM learning experience Increased awareness: Course helped crystallise theoretical knowledge into conscious practical applications in the classroom Experiential learning: A very positive aspect of the programme Im delighted that I went on the training. As I said overall, youre a way more confident teacher dealing with behaviour. You just have better and more resources and different things to draw upon. (T11) Role of the Group leaders: Building relationships and facilitating learning … (The group leaders) were very, very good. They were very open and… theyd ask and they would try their best - even give you ideas to try this or that or try the other. (T1) I am not alone: Social support and the learning process

Theme 4: Impact on child behaviour - A therapeutic milieu? A number of reported improvements in pupil behaviour: overall conduct staying more on task - less distracted fewer attention-seeking behaviours better peer relations They would be encouraging each other. They have started to get on better with each other and theyre co- operating amongst themselves even. (T1) …for the children…,you know who were doing well and who might often get overlooked, they were encouraged and you know, they were boosted and their behaviour improved as a result. (T2)

Theme 5: The positive classroom Teachers reported having much more positive classroom experiences, and seeing the classroom as a happy place to work and learn They reported feeling much less stressed and better equipped to deal with situations when they arise: Weve been having more and more really, really good days where theyve been enjoying it and Ive been enjoying it. And you know, they have been saying things like, oh you should get a sticker too teacher …you were very good today and all this kind of thing. So I think it helps me personally feel like I can cope with anything now. (T7) I think its good. I think its very good…I definitely would recommend people to do it (T3) All interviewees would highly recommend programme to another teaacher/school

3. Cost Analysis (Prof.Donal ONeill) Total cost of delivery: 22, Average cost/teacher: Average cost/child: (average class size=20) Approx 1% of total expenditure per primary school child (Eurostat, 2007) Combined with outcome (SDQ) data -> per 1 point change in SDQ scores (ICER); e.g. it would cost approx to bring the average highest scoring child in sample to within normal range of behaviour Much lower than other education-based interventions ($4863-$20,972 per child/youth)

Conclusion Findings support those from elsewhere Significant improvements in key areas: - Teacher skills/competencies in managing behavs - Child behaviour - Classroom atmosphere/ positive classroom envt Programme was regarded as being very worthwhile Other benefits: teacher stress, process of self- reflection, better home-school collaboration Very modest costs TCM programme is effective, cost-effective, highly acceptable to teachers, well suited to an Irish context and consistent with policy and practice guidance in Ireland heooThe good nTeh ews…

Thanks to… All participating schools, their principals, teacher participants The staff of Archways The Atlantic Philanthropies & Dormant Accounts fund The Expert Advisory Committee and Peer Reviewers All members of the research team and especially the MHSRU team at NUI Maynooth