Boone County Schools Extended Day Program RA Jones Middle School and Collins Elementary School

Extended Day Program Video

The purpose of this program was to narrow achievement gaps for all students, especially targeting students who are transient and live in poverty. PURPOSE IMPETUS Through work with Dr. Joe Murphy (Vanderbilt University) both inside and outside of the district, we realized that students with 2+ year achievement gaps need additional time beyond the 6 hour instructional day to achieve at the same level as their peers that do not start with this gap; Dr. Murphy described a “pancake approach” to meeting the needs of students with obstacles such as poverty and transience; this program seeks to be a crucial layer or “pancake” in a systemic system of supports for these students.

Quantitative Data Examination A summative sample of performance was obtained and compiled from Boone County extended and non-extended school students for a time period during the 2011 – 2012 school year. Assessment Data All students completed the Scantron Performance Series test before the program began and took the Scantron Performance Series test as a post test at the conclusion of the program.

Comparisons Examined in Summary Results Entry level reading and math scores were collected to compare reading and math scores of extended day program students to non-extended school programs. End of program scores (20 weeks) comparing extended day students and non-extended day students. 5 th grade specific analysis of reading and math scores comparing extended day students and non-extended day students.

Hypothesis Test Results extended day/non-extended day students comparison reading and math gains Questions included: Is there a difference in program effectiveness for the extended school program for the students attending the program 80% of the time to students who did not attend the extended school program? What is the difference in gains, if any, between the students who attended the program 80% of the time to students who did not attend the program? What is the likelihood of obtaining the observed difference in sample means, assuming there is no difference in the overall means between the two groups?

2 nd Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

3rd Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

4th Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

5th Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

6th Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

7th Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

8th Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

General Linear Model: RD Gain versus Grade, PRG80 Conclusion: There is marginal statistical evidence of program effectiveness, in addition the program possibly resulted in improved average scores by up to 42 points. If that is important, larger sample size needed to show effectiveness statistically. However, LOW R 2 value for the model suggests there are other things much more important.

General Linear Model: Math Gain versus Grade, PRG80 Conclusion: There is marginal statistical program effectiveness, in addition the program may have resulted in improved average scores by up to 34 points. Note R 2 <10%.

5th Grade Reading and Math Gains The overall trend in 2 nd – 8 th grade data shows greater gains for Extended Day Program students compared to non-program students.

Hypothesis Test Results μ 1 : mean of 5 th Reading Gain/Loss Program μ 2 : mean of 5 th Reading Gain/Loss NO Program μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 H A : μ 1 - μ 2 > 0 (without pooled variances) 1) The data indicated a statistical significant difference between the 5 th grade Extended Day students and the non-extended day 5 th students enrolled in the reading program. 2) The difference indicated that Extended Day students were performing scale score points better than the non-extended day students in the 5 th grade. 3) The p-value is the probability of obtaining the observed difference in the sample means and we would only have a 4% chance of finding a sample mean of Therefore we reject the Ho that there is not difference in the sample mean. We except the alternative HA that the extended day reading program produced a difference in scale scores gains of 5 th grade students attending the program versus the students not attending the program.

Hypothesis Test Results μ 1 : mean of 5 th grade Math Gain/Loss Program μ 2 : mean of 5 th Math Gain/Loss NO Program μ 1 - μ 2 : mean difference H 0 : μ 1 - μ 2 = 0 H A : μ 1 - μ 2 > 0 (without pooled variances) 1) The data indicated a marginal statistical difference between the 5 th grade Extended Day students and the non-extended day 5 th grade students enrolled in the math program. 2) The marginal difference indicated that Extended Day 5 th grade students were performing scale score points better than the non-extended day 5 th grade math students. 3) The p-value is the probability of obtaining the observed difference in the sample means and we would only have a 5.9% chance of finding a sample mean of Therefore we reject the Ho that there is not difference in the sample mean. We except alternative HA that the extended day Math program produced a difference in scale scores gains of 5 th grade students attending the program versus the students not attending the program.

Teacher Survey Teachers were asked to do the following: 1.Please rate the effectiveness and provide specific comments on the following curriculum: Time Warp, Successful Reader, Do the Math, Accelerated Math 2.Please rate the progress and provide specific comments of the students. 3.Please comment on the overall after school program. 4.What suggestions do you have to make the program more effective?

Teacher Survey Results

Teacher Survey Comments Specific comments from teachers include: 1.More PD needed on programs 2.Successful Reader was hard to implement because of large amounts of absences of students 3.Classes needed to be smaller 4.Do the Math program was beneficial to lower level students that have not yet mastered basic skills 5.Program was more effective when teachers had fewer groups of students 6.Behavior of students impeded progress for students 7.Program length of 3 hours needed to be reduced to 2 hours 8.Program needed to be more organized 9.More help for disciplinary issues 10.Four days a week instead of five

Parent Survey Parents were asked to do the following: 1.Did your child participate in the extended day program? If yes, did they participate every day and for the full length of time? If not, why not? 2.If your student was not in the program, why? 3.What is your level of satisfaction of the program? 4.What suggestions do you have to make this program stronger for your child’s success?

Parent Survey Results Parent responses: responses 2.58% had child participate but 69% did not require children to stay every day and for the full time. 3.47% reported other obligations got in the way 4.26% reported students didn’t stay because they couldn’t get homework done 5.11% reported children were too tired 6.11% reported children didn’t like the program 7.5% reported their children didn’t like the snacks 8.42% didn’t require their children to attend the program 9.46% reported the program was to long of a day 10.52% reported it conflicted with parent schedule 11.25% reported other

Parent Survey Results Parent responses on level of satisfaction: 56% would enroll their children if program was offered again and offered the following reasons why: Very good for students and helped bring up grades Save family money Children had fun Keep kids out of trouble being home alone Met new friends

Parent Survey Results Parent responses on level of satisfaction: 38% would not enroll their children if program was offered again and offered the following reasons why: Depends upon what the other children were doing Conflicts with other activities Need more family time Doesn’t need extra help Prefer 2 days a week

Parent Survey Results Parent suggestions for improving program: Make program shorter (length) – less hours per day Reduce number of days per week Offer more homework help Have less students Offer larger snacks

Student Survey Students were asked the following: 1.Why did you come to the after school program? 2.What did you like the most about the program? 3.What did you like the least about the program? 4.What would you change about the program to make it better? 5.What else do you have to add about the after school program?

Student Survey Results Students on why they came: 1.Most students reported because parents made them 2.Second most common response was because of bad grades 3.Several reported coming because they had fun 4.Came for fun, activities, and their friends came

Student Survey Results Students on what they liked most: 1.Gym activities 2.Got to be with friends 3.Snacks and drinks

Student Survey Results Students on what they liked least: 1.Math class 2.Classes in general 3.Some teachers 4.Staying so late 5.The work

Student Survey Results Students on what they would change to make better: 1.Nothing 2.More fun in class 3.Pick their classes 4.More time in gym 5.Play more games 6.Make it shorter 7.Offer better snacks

Student Survey Results Students on what else they had to add about the program: 1.It was really fun. 2.It’s awesome. 3.I really liked the program and teachers. 4.It stinks. 5.I hate the program.

Conclusions There was evidence of marginal statistical significance of program effectiveness for math students gain scores. Math gain scores improved up to an average of 34 points more for those in the program 80% of the time vs. those not in the program. There was evidence of marginal statistical significance of program effectiveness for reading students gain scores. Reading gain scores improved up to an average of 42 points more for those in the program 80% of the time vs. those in the program. At the end of the 20 week program, there was a difference in gains when comparing 5 th grade math extended day students to the non-extended day students in math which warranted a separate analysis for 5 th grade math students.

Conclusions At the end of the 20 week program, there was a difference in gains when comparing the 5 th grade reading extended day students to the non-extended day students in reading which warranted a separate analysis for 5 th grade reading students. Implementation of programs were inconsistent Time spent on the programs varied Longitudinal data needs to be collected over time

Additional Notes Extended school students represent lower starting test scores than non-extended day students. Total sample represents 2011 – 2012 Scantron data. Extended day students raw gap score difference in reading mean started 68 points lower than non –extended school students. Extended day students raw score difference in math mean was 152 points lower than non –extended school students. Sample of extended and non-extended day students contain a distribution of ability levels from high to low.

Additional Notes Only 50% of teachers in the program at RAJMS were content specialists. Student pre-selected physical activity courses were not implemented resulting in a decrease in student engagement in the program. Inconsistent teacher attendance resulted in a lack of fidelity in instructional implementation. 51% teacher absence rate at CES 30% absence rate at RAJMS

Next Steps Data Collect and review longitudinal data of student performance Collect and review comparative data from multiple demographic groups Collect and review data from 21 st Century Skills (creative thinking/innovation; critical thinking; communication; collaboration) Implementation Explore innovation through a reconfiguration of the school day at CES and RAJMS. Screen applicants for appropriate dispositions during the hiring process. Restructuring the program to address student / teacher attendance by decreasing the number of days per week and extending the length of the program. Increase parent/school/community communication with regard to the importance of following program protocols.

Longitudinal Mean Gain Projection Based on Average Program Mean Gain

Literature Sited 1. Murphy, J (2011). Closing Achievement Gaps: Research- Based Lessons for Educators. Vanderbilt University 2. Harris, E (Nov 2010). 21 st CCLS-Funded After School Programs. Harvard Research Project. Article available online at Harris, E. and Wimer, C (May 2011). Out of School Time Programs for Older Youth. Harvard Family Research Project. Article available online at