The Importance of Treatment Fidelity in Delivering Parent Programmes Presentation for the Incredible Years Wales Conference Llandrindod Wells, 7th February.

Slides:



Advertisements
Similar presentations
Amber Zank, M.S.E. Michael Axelrod, Ph.D. University of Wisconsin- Eau Claire NASP Conference 2011 Training Individuals to Implement a Brief Experimental.
Advertisements

ED PACE Educational Development for Planning and Conducting Evaluations West Virginia Department of Education.
February 2010 Researching the Incredible Years Toddler Parenting Programme in Wales Nia Griffith Bangor University Supervised by Professor Judy Hutchings.
Researching the IY Therapeutic Dino School Programme with high risk KS1 children in Gwynedd Schools: The BIG Lottery Grant Professor Judy Hutchings.
Evaluation of the Incredible Years TODDLER Parent Training Programme for nursery staff working with 2-3 year old children living in ‘high risk’ disadvantaged.
Incredible Years Programmes in Powys
Resource for: Stage 1, Stage 2 mentor preparation and ongoing annual Mentor Updates. 10 Chapters: Chapter 1: Mentorship – an overview. Chapters 2 – 9:
VIG Project Dissemination Event July 26 th, 2012 Robin Sen (University of Sheffield) 1.
Using Student Data as a Basis for Feedback to Teachers Ronnie Detrich Wing Institute Cal-ABA, 2011.
Promoting Rational Drug Use in the Community Monitoring and evaluation.
References: Tough Kid Social Skills, Skillstreaming Curricula Question 4: Can teachers implement CW-FIT with high levels of fidelity? Yes % of CW-FIT.
Parent Connectors: An Evidence-based Peer-to-Peer Support Program Albert J. Duchnowski, Ph.D. Krista Kutash, Ph.D. University of South Florida Federation.
Joined up services: making services work for families Professor Judy Hutchings OBE Centre for Evidence Based Early Intervention, Bangor University Children.
 Is extremely important  Need to use specific methods to identify and define target behavior  Also need to identify relevant factors that may inform.
Gwinnett Teacher Effectiveness System Training
1 Organizational, Time Management, and Planning Treatment for Children with ADHD (OTMP Study) NIMH-funded R01 New York University – Howard Abikoff, PI.
Oregon Supported Employment Center for Excellence Individual Placement and Support Fidelity Overview Oregon EASA Conference November 10, 2010 In Partnership.
IMPLEMENTATION OF PMTO IN ICELAND EVIDENSEKONFERENCE – DENMARK 2012 Margrét Sigmarsdóttir.
Researching The Incredible Years Therapeutic Dinosaur School Programme Funded by the Big Lottery.
Dr. Tracey Bywater, Prof Judy Hutchings, Dr. Dave Daley, Dr. Rhiannon Tudor Edwards, Ms Pat Linck, Prof Ian Russell The IY BASIC Parenting Programme: Focuses.
Exploring the evidence for early interventions Helen McConachie.
Key features of effective parenting programmes: what works and why Claire Halsey, Consultant Clinical Psychologist North Staffordshire Combined Healthcare.
EVALUATING THE IY TODDLER PROGRAMME WITH NURSERY STAFF BACKGROUND: Parenting programmes are an effective early intervention for the prevention and treatment.
Lessons from the IY Toddler trial in Flying Start services across Wales Nia Griffith March 2012.
Chapter 5: Improving and Assessing the Quality of Behavioral Measurement Cooper, Heron, and Heward Applied Behavior Analysis, Second Edition.
Visit our websites: PhD Study: Evaluation of the Efficacy of the Incredible.
1 Incredible Years in Wales: Research update Professor Judy Hutchings Centre for Evidence Based Early Intervention, Bangor University March 2011.
The Archways basis.point programme Delivering Incredible Years ® in Dublin’s Inner City 24 February, 2015.
Objectives for Session Nine Observation Techniques Participatory Methods in Tanzania Hand back memos.
1 Update on progress to deliver the IY programmes as part of the Parenting Action Plan for Wales IY Annual Conference 2008 Metropole Llandrindod Professor.
The Importance of Fidelity as a Means for Enhancing and Assessing Program Quality A Case Study of the RESPECT Program.
Evaluating the Cost-Effectiveness of the Incredible Years Toddler Programme Joanna Charles Bangor University, Wales Dr. Rhiannon Tudor-Edwards, Prof. Judy.
Parent Tutoring (PT) An Individualized Tier 3 Intervention for Students with Reading Problems Study 1 Duvall, Delquadri, Elliott & Hall (1992) Study 2.
Dr Amanda Perry Centre for Criminal Justice Economics and Psychology, University of York.
The Incredible Years Programs Preventing and Treating Conduct Problems in Young Children (ages 2-8 years)
11 Professor Judy Hutchings Centre for Evidence Based Early Intervention Bangor University Results.
Moving from Development to Efficacy & Intervention Fidelity Topics National Center for Special Education Research Grantee Meeting: June 28, 2010.
Evaluating the Enhancing Parenting Skills (EPaS) 2014 programme Margiad Elen Williams CEBEI, Bangor University Supervisors: Professor Judy Hutchings Dr.
Kirstie Pye, Research Officer NWORTH Clinical Trials Unit Bangor University.
Incredible Years Programmes in Powys Dr. Sue Evans, Consultant Child Psychologist, Powys tHB.
Dr. Tracey Bywater Dr. Judy Hutchings The Incredible Years (IY) Programmes: Programmes for children, teachers & parents were developed by Professor Webster-Stratton,
Program Fidelity Influencing Training Program Functioning and Effectiveness Cheryl J. Woods, CSW.
Evaluating the Incredible Years School Readiness Parenting Programme Kirstie Cooper.
Researching the Incredible Years Infant & Toddler Programmes Nia Griffith Bangor University Nia Griffith Bangor University.
Evaluation of the Incredible Years SCHOOL READINESS Parenting Programme in North Wales 25 th January 2013 Kirstie Pye, PhD Student.
Figure 1. Mean scores for Child Deviance controlled Figure 2. Mean scores for Child Affect controlled by pre-intervention scores. by pre-intervention scores.
Programme Information Incredible Years (IY)Triple P (TP) – Level 4 GroupPromoting Alternative Thinking Strategies (PATHS) IY consists of 12 weekly (2-hour)
Researching and rolling out the IY parent programme in Birmingham presentation for the CEBEI Annual Conference April 10 th 2014 Bangor University Bridget.
11 10 years on: developing and researching the Incredible Years programmes in Wales: Wednesday 17 th March 2010 Village Hotel, Cardiff Professor Judy Hutchings.
Laying the Foundation for Scaling Up During Development.
Lessons from the IY Toddler trial in Flying Start services across Wales Nia Griffith January 2012.
Background Treatment fidelity in group based parent training: Predicting change in parent and child behaviour Dr. Catrin Eames, Bangor University, UK
Evaluating the EPaS 2014 programme and an e-version of the Little Parent Handbook Margiad Elen Williams CEBEI, Bangor University.
USING MUSIC TO SUPPORT LEARNING How Can the Use of Music as a Teaching Tool Support and Enhance Learning and Improve Learning Outcomes ? Millicent Howard.
Project KEEP: San Diego 1. Evidenced Based Practice  Best Research Evidence  Best Clinical Experience  Consistent with Family/Client Values  “The.
Piloting the Incredible Years (IY) Therapeutic Social and Emotional skills programme alongside the IY classroom curriculum with young high-risk children.
Evaluation of the Incredible Years Teacher Classroom Management (TCM) Programme In North Wales Primary Schools Pam Martin, Judy Hutchings, David Daley,
SUBJECTS 12 parents (10 mothers and 2 fathers – 2 couples) of preschool children in the 3-5 years of age range; 1 mono-parental family; middle class /
Measuring Fidelity in Early Childhood Scaling-Up Initiatives: A Framework and Examples Carl J. Dunst, Ph.D. Orelena Hawks Puckett Institute Asheville,
Parent-Child Language in Wales Presented by: Nicole Gridley Supervisors: Judy Hutchings and Helen Baker- Henningham.
Participants Observational data collected from participants who took part in the large scale randomised controlled evaluation of the Incredible Years Toddler.
Datum: The influence of therapist factors on the effectiveness of the IY parent program Mentor Meeting 2016 Maartje Raaijmakers The Netherlands.
Supporting the Incredible Years® Trained Workforce
Evaluating the Incredible Years Baby Programme Catrin Hedd Jones Supervised by Dr David Daley, Dr Tracey Bywater School of Psychology Bangor University,
An evaluation of the online universal COPING parent programme:
SKILLS (Support for Kids in Learning and Language Strategies) Online Programme for School Support Staff Anwen Rhys Jones: School of Psychology, Bangor.
Implementing and Evaluating KiVa in the UK
Incredible Years© Evaluation
Incredible Years® Evaluation
Social Validity and Treatment Integrity
Presentation transcript:

The Importance of Treatment Fidelity in Delivering Parent Programmes Presentation for the Incredible Years Wales Conference Llandrindod Wells, 7th February 2008 Catrin Eames Bangor University Catrin Eames Bangor University Supervisory committee: Dr Judy Hutchings, Dr Carl Hughes, Dr David Daley Funded in collaboration by ESF/Objective 1 fund and Incredible Years Wales

Overview of presentation The importance of treatment fidelity History of treatment fidelity research Current Ph.D. research The importance of treatment fidelity History of treatment fidelity research Current Ph.D. research

What is fidelity? Fidelity may be referred to as the degree to which intervention delivery adheres to the original intervention protocol (IOM, 2001).

Fidelity research terminology Adherence Treatment delivery Treatment differentiation Treatment exposure Treatment integrity Participant responsiveness Adherence Treatment delivery Treatment differentiation Treatment exposure Treatment integrity Participant responsiveness

Effective delivery Adequate evidence base Methods used Means of assessing fidelity Adequate evidence base Methods used Means of assessing fidelity

History of treatment fidelity research Poor methodologies Surge of treatment manuals Manuals alone do not guarantee fidelity Few studies reported any fidelity issues Of those that did, no reference to the development of valid fidelity measures Poor methodologies Surge of treatment manuals Manuals alone do not guarantee fidelity Few studies reported any fidelity issues Of those that did, no reference to the development of valid fidelity measures

The importance of fidelity It is necessary to implement all components of a programme fully in order to preserve the behaviour change mechanisms that made the original programme model a successful one (Arthur & Blitz, 2000; Mihalic et al., 2002) Can offer a template of service delivery Further resources/supervision/training Ensure adherence to delivery of intervention models Programme drift most common reason for poor treatment outcomes (Mills & Ragan, 2000; Mowbray et al, 2003) It is necessary to implement all components of a programme fully in order to preserve the behaviour change mechanisms that made the original programme model a successful one (Arthur & Blitz, 2000; Mihalic et al., 2002) Can offer a template of service delivery Further resources/supervision/training Ensure adherence to delivery of intervention models Programme drift most common reason for poor treatment outcomes (Mills & Ragan, 2000; Mowbray et al, 2003)

Current fidelity measures Typically, manuals or checklist format Measure programme content Threats to validity Typically, manuals or checklist format Measure programme content Threats to validity

Benefits of observation Precise account of behaviour as it unfolds Bias reduced with independent observation Continuous sampling, quantitative Precise account of behaviour as it unfolds Bias reduced with independent observation Continuous sampling, quantitative

Rationale for current research IY : Internationally delivered and evaluated Over 10,000 trained in the BASIC Numbers of independent replications increasing Quantitative observational measure of process skills Ensure measure is reliable and valid Relate leader fidelity scores to changes in parent and child behaviour from the Sure Start Study IY : Internationally delivered and evaluated Over 10,000 trained in the BASIC Numbers of independent replications increasing Quantitative observational measure of process skills Ensure measure is reliable and valid Relate leader fidelity scores to changes in parent and child behaviour from the Sure Start Study

Leader Observation Tool (LOT) Objective frequency count Continuous time sampling at 10-minute intervals throughout the 2-hour session Both leaders coded 18 behaviour categories forming five skills subgroups: Listening Empathy Physical Encouragement Positive Behaviour Other/Negative Objective frequency count Continuous time sampling at 10-minute intervals throughout the 2-hour session Both leaders coded 18 behaviour categories forming five skills subgroups: Listening Empathy Physical Encouragement Positive Behaviour Other/Negative

Data set 12 groups, two leaders per group, 2 hour sessions Course split up into : Play and relationship building Praise and reward Effective limit setting Handling misbehaviour Four sessions selected for coding for each group Of these, 30% randomly selected for second coding Coders blind to behaviour change scores to avoid bias 12 groups, two leaders per group, 2 hour sessions Course split up into : Play and relationship building Praise and reward Effective limit setting Handling misbehaviour Four sessions selected for coding for each group Of these, 30% randomly selected for second coding Coders blind to behaviour change scores to avoid bias

LOT reliability and validity Internal reliability at both p<.05 and p<.01 84% average inter-rater agreement Intra-class correlations of each category: mean of.92 and.95 for code-recode and inter-rater reliability respectively Concurrent validity with parent and leader reports of the group intervention process LOT identified variability between groups, highlighting the heterogeneity in process skills and treatment fidelity Internal reliability at both p<.05 and p<.01 84% average inter-rater agreement Intra-class correlations of each category: mean of.92 and.95 for code-recode and inter-rater reliability respectively Concurrent validity with parent and leader reports of the group intervention process LOT identified variability between groups, highlighting the heterogeneity in process skills and treatment fidelity

LOT scores and behaviour change Leaders attributed total score per subgroup Parent data = 86 completers Each parent attributed a leader fidelity exposure score Predict that probability of parent behaviour change greater when attending a group delivered with high fidelity Greater change in parent behaviour in turn predict greater change in child behaviour Leaders attributed total score per subgroup Parent data = 86 completers Each parent attributed a leader fidelity exposure score Predict that probability of parent behaviour change greater when attending a group delivered with high fidelity Greater change in parent behaviour in turn predict greater change in child behaviour

Measures Leader Parent Child Leader: LOT Parent: Parenting Scale, Positive Parenting Child: ECBI, Child positives, Compliance Leader Parent Child Leader: LOT Parent: Parenting Scale, Positive Parenting Child: ECBI, Child positives, Compliance

Results Leader ParentParent Child Leader Positive Behaviour Positive Parenting (ß =.28, p<.01; R 2 =.08) Change in Positive Parenting Child Positive (ß =.60, p<.01; R 2 =.36) Change in Positive Parenting Compliance to Direct Commands (ß =.23, p<.05; R 2 =.05) Leader Physical Encouragement Parenting Scale (ß =.26, p<.01; R 2 =.07). Change in Parenting Scale ECBI-I (ß =.23, p<.05; R 2 =.05) Change in Parenting Scale ECBI-P ( ß =.34, p<.01; R 2 =.12)

Therefore…. In order to change child behaviour, need to change parenting behaviour Leader treatment fidelity of process skills significantly predict both observed change in parenting behaviour as well as parent self reported behaviour In turn, predict the change in child behaviour In order to change child behaviour, need to change parenting behaviour Leader treatment fidelity of process skills significantly predict both observed change in parenting behaviour as well as parent self reported behaviour In turn, predict the change in child behaviour

Conclusion Need to report fidelity criteria in treatment outcome studies, especially considering the number of manual based interventions that are replicated in the community today Poor implementation fidelity is the most common factor for poor treatment outcome (Mills & Ragan, 2000) Valid fidelity measurement can predict treatment outcome, as well as document treatment differentiation in multi-centre research Need to report fidelity criteria in treatment outcome studies, especially considering the number of manual based interventions that are replicated in the community today Poor implementation fidelity is the most common factor for poor treatment outcome (Mills & Ragan, 2000) Valid fidelity measurement can predict treatment outcome, as well as document treatment differentiation in multi-centre research

Thank you for listening For further information please contact: Catrin Eames School of Psychology College of Health and Behavioural Sciences Bangor University Gwynedd LL57 2DG