Appendix A. The Incredible Years: Parent, Child and Teacher Training Series

The Incredible Years  Award winning “exemplary best practices” program with demonstrated effectiveness. [Office of Juvenile Justice and Delinquency Prevention, Substance Abuse and Mental Health Services Administration]  Comprehensive—targets home, school and child.  Designed to prevent, reduce and treat conduct problems and to increase children’s social competence.  Parent skills training uses group discussion, videotape modeling, and homework exercises.  Child series (Dina Dinosaur) uses puppets and videotapes to teach social skills, conflict management, and successful classroom behavior.

Research indicates that children’s emotional, social and behavioral adjustment is as important for school success as cognitive and academic preparation.  Child program promotes children’s cognitive problem- solving strategies, increases use of pro-social skills and reduces conduct problems at home and school.  Parent program helps strengthen parenting skills, reduces parenting stress and increases parent’s involvement in school activities.  Teacher program strengthens classroom management skills, reduces classroom aggression and improves teachers’ ability to focus on student’s social, emotional and academic competence. The Incredible Years Outcomes

CMHRC commitment to excellence, demonstrated through:  Training and certification of facilitators, dedicated FT coordinator for Incredible Years programming.  Purchase of all required curriculum and program materials.  Use of treatment fidelity groups, peer review, videotape feedback.  Continued use of ongoing training and technical support from the Incredible Years developers  Ongoing evaluation of parent and child groups. Incredible Years Adherence to Model Fidelity

1.Implement a comprehensive model that employs evidence-based multiple strategies for providing supportive environments and that accommodates ethnic, cultural and economic diversity. 2.Expand the target population to universal and earlier prevention—reaching children with certain family characteristics that put them at particular risk for developing conduct problems, delinquency and substance abuse. 3.Strengthen partnerships with local agencies that serve at-risk populations and, with the community at large, promoting a holistic view of health that includes physical and mental well-being. CMHRC Goals for Incredible Years

The Incredible Years and Ben Franklin School  Partnership sponsored through a two-year grant from the Rochester Area Foundation First Steps Initiative  Implements the comprehensive model-teacher, child and parent.  First year, Franklin Kindergarten teachers and para- professionals receive extensive training and mentoring from certified Incredible Years trainers.  Second year, Franklin Kindergarten teachers implement independently with guidance and support from CMHRC IY staff.  Second year, Franklin 1 st Grade teachers will receive comprehensive training and mentoring from CMHRC IY staff (like Kindergarten teachers this year).

Why offer Dina Dinosaur in kindergarten classes at Ben Franklin school?  Social and emotional competencies are important predictors of school readiness and academic success.  Prevalence of aggressive behavior in early childhood 10% and may be as high as 25% for socio-economically deprived children.  Dina program cost effectively reaches high number of at-risk youth early in life when behavior is most malleable and before severe behavior problems arise. Incredible Years Dina Dinosaur Prevention

Goals for program at Ben Franklin [Kindergarten]:  Prevent early behavior problems from escalating.  Integrate children into peer group –classroom as “family”.  Improve social competence for entire class, provide common vocabulary and problem solving steps to use in everyday conflict.  Magnify dosage of IY intervention through teacher reinforcement of key concepts throughout the school day. Incredible Years Dina Dinosaur Prevention

Implementation at Ben Franklin during School Year :  Offered 2 times a 6-day cycle in 5 kindergarten classrooms  Lessons over entire school year, seven units in sequential order [rules/behavior, feelings, problem solving, anger management, peer relations etc.]  Led by Children’s Mental Health Incredible Years staff with Ben Franklin teachers participating in planning and delivery of lessons Incredible Years Dina Dinosaur Prevention - Children

Incredible Years Teacher Training at Ben Franklin Teacher training at Ben Franklin during School Year included:  Five full day interactive classroom management workshops offered throughout the school year, and 3 days of training from Seattle IY trainer on Dina Dinosaur curriculum.  Sessions on promoting pro-social behavior and school readiness, reducing non-cooperation and aggressive behaviors, and ways to collaborate with parents to support school involvement and school- home consistency.  Teachers implementing a classroom action plan with ongoing support from CMH IY staff.

Incredible Years Parent Training at Ben Franklin Parent training at Ben Franklin during School Year included:  A 12-week parent group with two CMH IY facilitators meeting once a week. Child care, supper and transportation offered to program participants.  Used a collaborative process to promote positive parent-child relationships through play, praise and attention.  Worked on effective limit-setting, use of incentives, building social competence and handling misbehavior.  All Kindergarten parents received detailed handouts on the Dina Dinosaur curriculum throughout the year and received tips on how to use the Dina techniques at home.

Evaluation of Incredible Years at Ben Franklin Evaluation on all three components of the model, data collection includes:  Classroom Environment Scale [pre/post]  Social Competence Scale on Student Behavior [pre/post]  Teacher Classroom Strategies [pre/post]  Teacher Workshop Satisfaction Surveys  Parent Dina Dinosaur Satisfaction Surveys  Parent Stress Index [parent group, pre/post]  Eyberg Child Behavior Inventory [parent group, pre/post]  Parent Group Satisfaction Surveys

Classroom Environment Scale Teachers filled out the scale in October 2007 and again in May Instructions were to: Think about the general atmosphere of your classroom during the last month. Use the scale below where ‘10’ stands for always, ‘0’ stands for never and ‘5’ stands for half the time. 0.… ….3.…4….5.…6….7.…8.…9….10 Rarely Sometimes Almost always

Classroom Environment Scale October 2007 versus May 2008 Rarely Some- times Almost Always Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms) Q1. Students follow classroom rules Q2. Students work cooperatively (helping sharing, teamwork). Q3. Students attempt to problem solve when conflicts arise (talk about problem, discuss feelings, suggest solutions). Q4. Students manage anger (uses words not fists, takes deep breaths, remains calm). p≤.002 p≤.001 p≤.000 p≤001 Statistically significant = Source: Classroom Environment Scale database

Rarely Some- times Almost Always Teachers rate their classrooms on scale of 1 through 10 for behaviors exhibited in past month (N=8 Classrooms) Q5. Students express feelings appropriately (positive or negative) Q6. Students handle transitions well. Q7. Students stay on task during class activities. Q8. Students show awareness and concern for individual differences p≤.000 p≤.002 p≤.003 p≤000 Statistically significant = Source: Classroom Environment Scale database Classroom Environment Scale October 2007 versus May 2008

How confident are you in managing current behavior problems in your classroom? How confident are you in managing future behavior problems in your classroom? Teachers rate their confidence on six point scale: very unconfident, unconfident, somewhat unconfident, somewhat confident, confident, very confident (N=8 Classrooms) Source: Classroom Environment Scale database Classroom Environment Scale October 2007 versus May 2008

Average TOTAL score on Classroom Environment Scale by Program (maximum =80 points) Source: Classroom Environment Scale database Classroom Environment Scale By Franklin Program (AM, PM, Extended Day) October 2007 versus May 2008

Social Competence Scale – Teacher* Teachers rate individual student behavior on 25 questions using scale: ‘not at all’, ‘a little’, ‘moderately well’, ‘well’ ‘very well’. Ratings occurred in October 2007 and again in May Questions look at:  Emotional regulation  Problem solving skills  Attentiveness and initiation N= 89 students (17 students moved before post data was collected and are not included in the analysis.) * Tool used by Incredible Years Programs – Denver Colorado School District

Social Competence Scale Teacher Emotional Regulation Source: Social Competence Scale Teacher database N=89 students Percentage who do “well” or “very well” October 2007 versus May 2008

Social Competence Scale Teacher Social Skills/Peer Interaction Source: Social Competence Scale Teacher database N=89 students Percentage who do “well” or “very well” October 2007 versus May 2008

Social Competence Scale Teacher Attentiveness/ Initiative Source: Social Competence Scale Teacher database N=89 students Percentage who do “well” or “very well” October 2007 versus May 2008

Social Competence Total Score by Franklin Program (AM, PM, Extended Day) October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) All statistically significant at p≤000 N=89 students Source: Social Competence Scale Teacher database

Social Competence Total Score by Ethnicity October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) All statistically significant at p≤01 Source: Social Competence Scale Teacher database N=42 N=10 N=6 N=20 N=8 1 Native American pre 75, post 125

Social Competence Total Score by Gender / Ethnicity/ Disability: October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) All statistically significant at p≤01 Source: Social Competence Scale Teacher database N=6 N=83 N=51 N=38 N=42 N=47

Social Competence Total Score by Teacher October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) Source: Social Competence Scale Teacher database N=18 N=20 N=33N=18

Social Competence Total Score by Gender / Ethnicity/ Disability: October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) All statistically significant at p≤01 Source: Social Competence Scale Teacher database N=6 N=83 N=51 N=38 N=42 N=47

Social Competence Total Score by Ethnicity October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) All statistically significant at p≤01 Source: Social Competence Scale Teacher database N=42 N=10 N=6 N=20 N=8 1 Native American pre 75, post 125

Social Competence Total Score by Franklin Program (AM, PM, Extended Day) October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) All statistically significant at p≤000 N=89 students Source: Social Competence Scale Teacher database

Social Competence Total Score by Teacher October 2007 versus May 2008 Average TOTAL score by Program (maximum =125 points) Source: Social Competence Scale Teacher database N=18 N=20 N=33N=18

Teacher Strategies Scale* Teachers rate how often they use various teaching strategies on 34 questions using scale: ‘rarely/never’, ‘sometimes’, ‘half the time’, ‘often’ ‘very often’. Ratings occurred in October 2007 and again in May Questions divided into five scales**:  Positive Approaches with Parents  Praise and Incentives  Proactive Strategies  Limit-Setting Strategies  Inappropriate Strategies (lower score better) N= 5 teachers, parent scale =4 teachers *Tool used by Incredible Years Programs – Denver Colorado School District ** Incredible Years in process of completing psychometric analyses, currently using these scales

Average total score on each subscale, n=5 (parent n=4) Source: Teacher Strategies database Teacher Strategies Scale Fall versus Spring Increase in proactive strategies and decrease in inappropriate strategies significant at p≤05, increase in praise marginally significant p≤06

Teacher Training: Satisfaction with Workshops 1-5 There were no “not helpful” answers Source: Teacher workshop satisfaction database Ben Franklin Staff Only

Teacher Training: Satisfaction with Workshops 1-5 There were no “neutral” or “not helpful” answers Source: Teacher workshop satisfaction database Ben Franklin Staff Only

Teacher Training: Satisfaction by Workshop Average score Scale: 1=not helpful, 2=neutral, 3=helpful, 4=very helpful WORK SHOP CONTENTVIDEOSLEADER’S TEACHING GROUP DISCUSSION ROLE PLAYS 1 N= N= N= N= N= Source: Teacher workshop satisfaction database Ben Franklin Staff Only

Parent Feedback Survey Dinosaur School Parents were asked to read a description of the Dinosaur School (puppets named Dina and Wally and the lessons being taught) They were then asked to think about the last three months and rate the following questions on a 4-point scale “very often”, “somewhat often”, “once in awhile”, “not at all”. 1.How often does your child talk about the things he or she is learning in Dinosaur School? 2. How often have you noticed your child doing the following: a.Talks about his/her feelings b.Identifies / recognizes when there is a problem c.Tries to solve problems when they come up (uses solutions/ideas from Dinosaur School) d.Shares or helps friends/siblings

Tools Used to Measure Outcome: 1. Pre and Post Parenting Stress Index [PSI] Paper/pencil assessment parent fills out to assess parent- child relationship (three subscales and total stress score) 2. Pre and Post Eyberg Child Behavior Inventory [ECBI] Paper/pencil behavioral rating scale parent fills out to measure their perception of child conduct problems and the extent to which it is a problem for the parent. Parent BASIC Group Weekly parent group using Incredible Years curriculum to strengthen parenting skills and reduce stress. Nine parents completed [3 fathers, 4 mothers, 1 stepparent, 1 grandparent]

Significant Drop in Percentage of Parents at a “Clinical Level” of Parental Stress (TSS*) Average stress level for the Ben Franklin parent group [beginning and end] is less than for the parent “treatment groups” held at the CMHRC. Average Entry Score – 81.2 (SD=8) Average Exit Score (SD=13) t{5}=6.5, significant at p<.001 *Total Stress Score (TSS) is an indication of the stress level experienced within the role as parent. Clinical level is at or above the 90 th percentile on PSI scale. N=6 with pre and post PSI* Parenting Stress Index Pre to Post Program Source: PSI / ECBI Database

Significant Improvement in Behavior Intensity and Child Behavior Problems (as perceived by parents) INTENSITY Average Entry Score (SD=19) Average Exit Score - 89 (SD=8) t{8}=8, significant at p<.001 PROBLEM Average Entry Score – 12 (SD=10) Average Exit Score - 1 (SD=3) T{8}=3, significant at p<.01 Eyberg Child Behavior Inventory [ECBI] Pre to Post Program N=9 with pre and post ECBI Average ECBI levels for the Ben Franklin parent group [beginning and end] is lower than for the parent “treatment groups” held at the CMHRC. Source: PSI / ECBI Database

Parent Satisfaction with BASIC IY Parent Program, n=9 parents 1.Problem(s) that originally prompted me to take this program for my child…. 2.Use of methods to try and change child’s problems… 3.My feelings about my child’s progress… 4.Has helped with other personal/family problems… Source: Tally sheet for parent BASIC satisfaction surveys

Parent Satisfaction with BASIC Parent Program Source: Tally sheet for parent BASIC satisfaction surveys Of 9 respondents:  All would “strongly recommend” the program to a friend or relative.  All felt “positive” [3] or “very positive” [6] about achieving their goal in the program for child and family.  All were “confident” [5] or “very confident” [4] they could manage current behavior problems.  All were “optimistic” [1] or “very optimistic” [8] about good results from the program.  All felt the parenting approach used to change child behavior was “appropriate” [1] or “greatly appropriate” [8].

Parent Satisfaction with BASIC Parent Program Source: Tally sheet for parent BASIC satisfaction surveys Two facilitators: 1.I feel the leader’s teaching was… 2.The leader’s preparation was… N=9 responses  All participants felt “very supported” [6] or “supported” [3] by the group.  Eight of nine program participants would like to keep meeting as a group. (The one participant who did not want to keep meeting was a grandparent).  Seven felt it was “likely” or “somewhat likely” they would continue meeting. The other two were “neutral” Response to FacilitatorsResponse to Group

Parent Satisfaction with BASIC Parent Program What did you see as the main benefit of the Incredible Years program?  I have become so much better with my children in all the ways that a father should be. I feel this will help me stay that #1 dad that I’ve always wanted. Thank you very much.  Positive talk was the most beneficial for me. Also the skills taught – how and when to ignore, using calming down techniques.  How easy it is to make small changes and have such wonderful results. When the program is broken down the way it was, everything makes so much sense.  Learning skills to improve communication, bringing parents into the children’s setting really helped my kids. They have been so proud and excited to have me here every Monday.  Consistency and reinforcing positive/expected behaviors.