Implementation and Evaluation of the Rural Early Adolescent Learning Project (REAL): Effects on Students in K-12 Northern Plains Schools

Project REAL Jill V. Hamm, University of North Carolina at Chapel Hill Thomas W. Farmer, University of Pennsylvania Kimberly Dadisman, University of North Carolina at Chapel Hill Dylan Robertson, University of North Carolina at Chapel Hill NREA Research Symposium October 25, 2008

Key Personnel for Project Real PIs: Thomas Farmer, Ph.D.; Jill Hamm, Ph.D. Investigators –Patrick Akos, Ph.D. Kimberly Dadisman, Ph.D. –Diane Gut, Ph.D. Matthew Irvin, Ph.D. –Carol Malloy, Ph.D.Linda Mason, Ph.D. –Judith Meece, Ph.DDylan Robertson, Ph.D. Intervention Staff –Kirsten Feil Allen Murray, Intervention Director –Chrissy Nehrenberg Project Support Staff –Jonathan BanksAmity CrowtherCourtney MannKerrilyn Lambert –Bryan HutchinsKelli O’BrienMallory VinsonMaggie Gravelle –Jana ThompsonJenny WestrickMatt SettlemeyerRachel Goolsby Graduate Assistants, Intervention –Beverly GlienkeVictoria SchaefferAbby Hoffman

General Aims of Project REAL Professional development for rural teachers who serve early adolescents –Responsive to local resources, needs, and school configurations –Promote strategies that provide universal support for all students during early adolescence –Promote strategies that help teachers advance the learning of low-achieving students Creation of a portable, research-based intervention package to be used across diverse regions in rural areas of the United States. Evaluation of the efficacy and impact of the intervention on rural early adolescents’ adjustment

Overview of Project REAL Schools Common Challenges : –Poverty and changing employment base; –Increasing diversity of student population; –Financial impact of large geographic distances; –Shrinking population of communities and schools; –Rapidly changing economic foundation of communities. –Teacher recruitment, retention, and licensure issues; Common Strengths : –Faculty are a part of the community; –Schools are focal point for community; –Generational affiliation with local school and/or school system; –Participation in extra-curricular activities at higher than average rates –Home ownership rates above the national average –Social indicators (church, civic clubs) show strong community ties,

Pilot Sites: –2 Appalachian states, 4 middle school transition schools Full Implementation Sites: –Northern Plains: 4 k-12 schools –Appalachia: 4 middle school transition schools –Midwest: 2 k-8, 2 middle school transition schools –Southwest: 4 middle school transition schools –Deep South: 4 k-12 schools –Southeast: 2 k-8, 2 middle school transition schools –Far West, 4 middle school transition schools –Pacific Northwest, 4 k-8 schools Across Rural Areas of the United States: Total Number of Districts/Schools: 28 districts/56 schools Total Number of Teachers: 392 Total Number of Students: approximately 4000 Research Sites

Academics Behavioral Engagement Social Relations

Positive Behavior Enhancement Academic Engagement Enhancement Social Dynamics Training Academic Engagement Enhancement – -General strategies that promote an instructional context that is responsive to the need of a broad and diverse range of students Positive Behavior Enhancement – - Strategies to create structure and consistency across classes - Encouraging self-directed behavior - Proactive approaches to prevent behavioral difficulties Social Dynamics Training – - Promoting teachers’ awareness of the impact of peers on motivation & achievement. - Recognizing peer groups and social roles - Identifying youth with social difficulties that interfere with their own or others’ learning - Strategies to use peer group dynamics to foster classroom engagement - Strategies to help students with social difficulties develop positive, supportive relationships

Project REAL Intervention: Directed Consultation Model Training and consultation are provided to both regular education teachers and specialists who have “at-risk” students in their class. Focus is on the issues in the transition to early adolescence that impact on academic performance. Focus in on class-wide issues and “at-risk” students (as defined by teacher).

Universal Components Site Visits, Needs Assessments, and Summer Institutes Web-based training modules on universal topics including: –Early Adolescent Development –Motivation and – Academic Engagement –Instruction for low-achieving students –School and classroom social dynamics –Information processing –Literacy support Project REAL Intervention Targeted Components Video-conference consultation with Project REAL staff Provides a forum for intervention specialists to facilitate discussions of: –Web-based training modules –Strategies for at-risk students –Strategies for classroom issues

Research Design for Project REAL Intervention schools matched with control schools –1/2 with middle school transition configuration –1/2 alternative configuration (e.g., k-8, k-12) –Schools are matched closely on demographic data Baseline student data collected in spring of 5 th grade; Process/transition data collected in fall and spring of 6 th grade Student outcome data on school adjustment and academic achievement collected in spring of 6 th grade A follow-up year of data is collected (7 th grade, fall & spring)

Findings for Intervention Effects: Northern Plains Site Site Description Recruited from all 4 th, 5 th, & 6 th grade classrooms of four public k-12 schools in a state in the Northern Plains –72% agreed to participate, N=165 student participants –45% Native American, 55% White –92 boys, 73 girls Schools were eligible for U.S. Department of Education’s Rural and Low-Income School Program (RLISP) –locale code 7 or 8 and at least 20% of students are from families living below the federal poverty level –In our sample, 64.6% of students were eligible for subsidized meals through the National School Lunch Act

Equivalence of Intervention and Comparison Groups on Selected Child and School Attributes Total ComparisonInterventionp-value Female child, % Native American child, % White child, % Other race/ethnicity, % Child in 4 th grade at baseline, % Child in 5 th grade at baseline, % Child in 5 th grade at baseline, % School involvement at baseline, mean End of course grade at baseline, mean School size, mean Eligible for Free lunch, % Minority students, % Pupil/teacher ratio Note. There were 165 participants (105 intervention and 60 control students) and 4 schools (2 intervention and 2 control schools).

Perceptions of Social-Academic Context –Peer norms for effort and achievement (Hamm, 2001) –Emotional risk of participation (Hamm & Faircloth, 2005) –Bullying Context (Song, 2008) Affective Relationships with Schooling –Sense of Belonging (Hagborg, 1994; 1998) –School Valuation (Voelkl, 1996) Achievement –End-of-year state-level standardized test (school records) –Grades (school records) Three Key Outcome Domains

Is participation in project REAL associated with more favorable outcomes on measures of (a) student’s perceptions of the social- academic context, (b) affective relationships with schooling, and (c) student achievement? Do estimated program effects differ by subgroup (i.e., gender or race/ethnicity)? Key Questions

Plan of Analysis Hierarchical Linear Regression Models Control variables : –Baseline, fall scores as controls –Ethnicity (Native American, White), Gender Intervention Main Effect Intervention X Ethnic Group, Intervention X Gender effects Tests for improvement to fit of model for block of variables; Test for significance of beta weight within significant block of variables

Hierarchical Regression Analysis of Indicators of Students’ Academic Achievement Note: Standardized Betas reported: +p <.06, *p <.05, **p <.01, and ***p <.001 (one-tailed). Standardized State Test End of Course Grades Native American -0.13** *-0.16*-0.29** Female ***0.26***0.36** Prior year score (spring) 0.71*** 0.32*** 0.30*** Intervention ***0.25** Intervention * Native American 0.23*0.18 Intervention * Female R2R F for change in R ***0.12.2*15.9***15.2***1.5

Hierarchical Regression Analysis of Indicators of Students’ Affective Relationship with Schooling School Valuing/ Involvement School Belonging Native American0.17*0.14* Female Current year score (fall)0.38***0.36***0.35***0.55***0.53***0.50*** Prior year score (spring)0.20* Intervention0.15* *0.07 Intervention * Native American ** Intervention * Female R2R F for change in R ***4.4* ***2.9* Note: Standardized Betas reported: *p <.05, **p <.01, and ***p <.001 (one-tailed).

Hierarchical Regression Analysis of Indicators of Students’ Perceptions of the Social Academic Context of Classrooms and School Peer Norms Emotional Risk Bullying Protection Variable Native American 0.19**.17** * * Female * Current year score (fall) 0.47***0.46***0.44***0.42***0.39***0.38***0.50***0.51***0.47*** Prior year score (spring) 0.22**0.18**0.21**0.16**0.15* * ** Intervention 0.13* ***0.11 Intervention * Native American 0.36**-0.33*0.21 Intervention * Female *0.07 R2R F for change in R ***3.2*3.6*11.4***2.23.6*30.1***10.0***1.5 Note: Standardized Betas reported: *p <.05, **p <.01, and ***p <.001 (one-tailed).

After controlling for previous year scores on each outcome, and for student demographic characteristics, students in Project REAL intervention schools, compared to their peers in matched control schools reported … More supportive peer norms for effort and achievement; More peer protection from bullying; A greater sense of school belonging; Greater valuing of school; Higher year-end grades. Results

Intervention effects were particularly pronounced for Native American (versus White) students for: State-level standardized test scores Sense of belonging Perceived peer norms for effort and achievement

Conclusions Findings from Project REAL indicate that universal interventions can be effective for improving adjustment among youth in general, but particularly among ethnic minority youth. Teachers in the intervention schools were able to create and sustain classrooms that undermined peer socialization against student effort and achievement.

Future Directions Examine intervention effects in middle school transition models Examine intervention effects in k- 8, k-12 versus middle school transition models Examine rural early adolescent adjustment patterns across time, diverse locales, diverse schooling configurations, and poverty levels