1. OUT-OF-SCHOOL TIME AS A DEVELOPMENTAL CONTEXT Lessons from Successful (and Unsuccessful Afterschool Programs Deborah Lowe Vandell SRHD Biennial Conference San Antonio TX March 2010

2. The Policy Context Disconnect between the length of the school day and children’s academic, social, behavioral, and physical needs High rates of maternal employment –74% of the mothers of children 6-17 yrs Concerns about –Low test scores –Unmet needs of English Language Learners –Negative effects of low supervision –Youth as victims & perpetrators of crime –Childhood obesity Some evidence of beneficial effects of afterschool programs but not always An element in Race to the Top and other school reform efforts

3. What is an after-school program? Narrow definition – programs that are offered by schools or other organizations on a daily basis throughout the school year Broad definition – includes extracurricular activities, sports, clubs, and sports offered by schools, libraries, and youth organizations

4. After-School Programs Sometimes Offer Academic enrichment & remediation Homework help Science, computer & math clubs Book clubs Non-academic enrichment Organized sports & recreational games Music, drama Arts and crafts Scouts, 4-H, YMCA

5. Public and Private Investments in Programs After-school programs –Serve 7 million+ children –21% of 6 to 9 yr olds & 14% of 10-12 yr olds –CA After-School Education and Safety Program –21 st Century Community Learning Centers –National Network of State Afterschool Networks Participation in at least 1 organized activity in a yr –81% of 6- to 11-yr-olds –83% of 12- to 17-yr-olds –90% of non-poor vs 60% of poor children

6. Do afterschool programs have positive effects on child developmental outcomes?

7. Effect Size a statistic that measures the magnitude of a program’s impact on a particular outcome One common metric – Cohen (1988) “small” - d =.2 "medium” - d =.5 “large” - d =.8 Effect sizes also can be benchmarked against those reported in other studies. 1. Aspirin on heart disease d =.03 2. School-based substance abuse prevention programs on drug & alcohol use d =.09 3. Class size reductions on math achievement d =.23

8. A RECENT META-ANALYSIS Durlak, J. A., & Weissberg, R. P. (2007) 49 reports of 73 programs Meta-analyses of 1. All programs 2. Programs with Sequential & Active activities and Focused & Explicit content (SAFE).

9. Durlak & Weissberg Meta-Analysis Outcomes# of Studies Overall Effect Size Met SAFE Criteria Did not meet SAFE criteria Self- perceptions 22.34.35.14 School bonding 28.14.26.03 + social behaviors 35.19.30.06 Misconduct42.18.26.07 Drug use27.11.22.03 School Achievement 20.16.31.03 Grades25.11.24.05 School Attendance 20.16.31.03

10. ADDITIONAL COMPONENTS OF PROGRAM QUALITY (NRC, Eccles & Gootman, 2002) Positive relationships between students and staff Positive relationships between students Mix of academic and non-academic skill- building activities High levels of student engagement Mastery orientation Appropriate levels of structure Opportunities for autonomy and choice

11. Study of Promising Afterschool Programs (Vandell, Reisner, Pierce et al. 2007) A longitudinal study of almost 3,000 low-income, ethnically diverse elementary and middle school student Eight states: CA, CO, CT, MI, MT, NY, OR, RI 14 communities: rural, small towns, mid-size cities, large cities All located in high-poverty communities First year devoted to identifying & describing promising programs Studied student academic & social outcomes over two years

12. Programs received consistently high ratings on all of the program quality dimensions Positive staff-student relationships Positive relationships between students Mix of skill-building activities High levels of student engagement Mastery orientation Appropriate levels of structure Opportunities for autonomy and choice

13. Study Participants Elementary School Sample 1,796 students in Grades 3 & 4 from 19 schools 89% free- or reduced-price school lunch 88% students of color Middle School Sample 1,118 students in Grades 6 & 7 from 16 schools 63% free- or reduced-price lunch 69% students of color

14. Organized activities such as team sports, school- based activities and lessons Home alone and home with a sibling “Hanging out” with peers without adult supervision Each reported on 4-point scales –1 = not at all/ once or twice –2 = about once a week –3 = 2 – 3 times a week –4 = 4 or more days a week Other Afterschool Experiences

15. – Programs only (PO) – Programs plus other activities (PP) – Low supervision with activities – Other FOUR CLUSTERS WERE IDENTIFIED

16. Participation in Promising Programs and Other Activities Over 2 Years Elementary Sample 54% regular program participants (about 90 day/yr) 2/3 Program Only; 1/3 Program Plus 15% low supervision Middle School Sample 49% regular program participants (about 55 days/yr) 2/3 Program Only; 1/3 Program Plus 16% low supervision

17. Child Outcomes Measured at Baseline, at the end of Year 1, and at the end of Year 2 Test ScoresSelf-ReportsTeacher Reports Math Reading Work habits Misconduct Substance use Work habits Task persistence Academic performance Prosocial behavior Aggression

18. HLM analyses Level 1 - child Level 2 – school/program Difference scores 10 multiple imputations Analytic Plan

19. Covariates Gender Race-ethnicity Grade level Household income (per $1000) Maternal (or guardian) education (in yrs) Mother (or guardian) works full time Household structure

20. Elementary Sample: Child Report Program Only vs. Low Supervision Effect Sizes

21. Elementary Sample: Child Report Program Plus vs. Low Supervision Effect Sizes

22. Elementary Sample: Teacher Report Program Only vs. Low Supervision Effect Sizes

23. Elementary Sample: Teacher Report Program Plus vs. Low Supervision Effect Sizes

24. Elementary Sample Math Achievement Test Scores Effect Sizes

25. Middle School Sample: Youth Report Program Only vs. Low Supervision Significant Effect Sizes

26. Middle School Sample: Youth Report Program Plus vs. Low Supervision Effect Sizes

27. Middle School Sample Math Achievement Test Scores Effect Sizes

28. Conclusions: Elementary School Sample Standardized Tests Gains in math for PO after one year; gains for both PO and PP groups after 2 years Child Reports Gains in work habits & reductions in misconduct for both PO & PP groups; larger effects after 2 years Teacher Reports Gains in work habits, task persistence, social skills, prosocial behaviors & reductions in aggression for PO after first year; improvements for both PO and PP after 2 years

29. Conclusions: Middle School Sample Standardized Tests Gains in math achievement for PO & PP groups after 2 years (not 1 year) Youth Self-Reports Both PO and PP youth reports reported gains in work habits after 2 years (not 1 year) and larger reductions in misconduct and substance use after two years Teacher Reports Little evidence of differences between Program and low supervised youth

30. Why are after-school programs beneficial?

31. Study 2: Experience Sampling - Out-of-School Time Vandell, Shernoff, et al. (2007, 2009) 191 8 th grade students, primarily low-income students of color 8 middle schools in 3 communities Students wore watches that beeped them at random times during non-school hours – afterschool, evenings, weekends 35 signals during a one week period in the fall and 35 signals during a one week period in the spring

32. Students Filled Out Logbooks After each signal, students recorded –Who they were with –Where they were –What they were doing –How they were feeling –And their levels of effort, concentration, motivation

33. Very Little Missing Data !! On average, students responded to 33 of the 35 signals during the week. 12,143 after-school, evening, and weekend experiences were reported. 5, 136 of the experiences occurred after school.

34. Students’ Activities at Programs and Elsewhere

35. Differences in Supervision (% of time) No program Not at program At Program Unsuper’d peers 16.721.0 7.2*** Sup. Peers 26.426.0 91.1*** No adults 38.438.4 7.5*** Alone12.911.70.0 Sib care 5.83.50.1

36. Differences in Motivation, Effort, and Feelings (4-pt ratings) No program Not at program At program Motivation2.72.9 3.0*** Effort1.91.9 2.5*** Importance2.42.5 2.9*** Positive Emotions 2.22.3 2.5*** NegativeEmotions1.31.21.3

37. Choice, Concentration, and Effort

38. Study 3: Are Particular Program Features Associated with Child Developmental Outcomes? (Pierce, Bolt, Vandell, 2010) 3-year longitudinal study of children who attended 30 programs of varying quality n = 150 in 1 st grade n = 120 in 2 nd grade n = 91 in 3 rd grade 49% male 89% white 60% parents have college degrees

39. Observations Of Program Quality 4 program observations in 2 nd grade and 3 observations in 3 rd grade staff-child relationships availability of age-appropriate activities programming flexibility Analyses control for family background and child prior functioning

40. Child Developmental Outcomes Teacher ratings –Mock Report Card Reading Math –Work habits (6 items, 5 pt ratings) –Social skills with peers (7 items, 5 pt ratings Collected from classroom teachers in G1, G2, and G3.

41. Analytic Strategy HLM analyses in Grade 2 & Grade 3 Level 1 – child Level 2 – program Covariates: gender, minority (yes/no), maternal education, family income, 2-parent household (yes/no), firm/responsive parenting, prior child functioning,

42. Effects Associated With Program Quality Features in G2 and G3

43. Five Take-Home Messages Regular participation in high quality afterschool programs is linked to positive social and academic outcomes Gains are more evident after two years than after one year, suggesting that duration is important Youth reports of experiences reveal differences in activities, social partners, motivation, effort, and affect while attending afterschool programs There is evidence that particular aspects of children’s experiences are related to child developmental outcomes Out of school time IS an important developm ental context

44. http://childcare.gse.uci.edu/ for more information