OCTOBER 13, 2010 SCHOOL OF EDUCATION RESEARCH SEMINAR SERIES Natasha K. Bowen UNC School of Social Work Too Good to Be True: Can a Brief Writing Intervention Really Close the Achievement Gap?

Acknowledgments Kate M. Wegmann and Kristina C. Webber orchestrated the intervention at the middle school. Funding for teacher incentives was provided by a Pfouts grant from the School of Social Work.

Previous Research Cohen, Garcia & Apfel, 2006, Science, 313, Report on two studies involving 119 African American and 124 European American 7 th graders, over 1 school term Random assignment; 2 conditions  treatment students were asked to indicate their most important value  control students their least important value. Regression analyses  2-way interactions  Dummy codes representing 3 teachers Repeated measures, & means tests also presented

Previous Research Cohen, Garcia & Apfel, 2006, Science, 313, Showed race by condition effects . 26 and.34 in GPA metric in the 2 studies; claimed 40% reduction in grade gap between African and European American students  Bigger effect for lower performing students  No effect for European American students  Conducted additional tests to demonstrate that psychological threat was the mechanism

Previous Research Cohen, Garcia, Apfel, & Master. (2009). Science, 324,  Follow-up of 2006 study (but now 3 cohorts and different sample sizes from those reported earlier)  Original study took place at beginning of 7 th grade  Follow-up examined effects on performance at end of 8 th grade  Booster treatments provided in 7 th grade  385 total subjects  Regression  3 way interaction among race, condition, prior performance

Previous Research Cohen, Garcia & Apfel, Purdi-Vaughs, Apfel, & Brzustoski. (2009). Science, 324,  Race by condition effects continued through 8 th grade  Better performance for African American students in TX  No change for European American students In sum, long-term effects of a brief writing intervention on performance, with biggest effect on low achieving African American students

Theory of Psychological Threat Worry about reinforcing a negative stereotype about a group one belongs to represents a threat that may cause stress and interfere with performance. Strong empirical evidence of (a) the existence of stigma-based threat, (b) the ease with which threat can be manipulated, and (c) the magnitude of the potential impact on performance Across groups, ages, situations

Transactional Developmental Processes Long term effects due to changing trajectories of performance.

Durham Study SAMPLE : Population of students at the middle school—622 students, -- Quarterly grade data on 585 students (absences and moves) African American 70% (407) Hispanic/Latino20% (117) White 5% (30) Multi-racial 3% (17) Asian 2% (14) Other

Durham Study DESIGN: 3X2 Experimental Random assignment of students to 3 writing prompts Neutral Self-affirming Group-affirming Random assignment of teachers to 2 reading conditions Read essays of all students Don’t read essays of any students

Durham Study MEASURES: Gender: 0=female; 1=male Race/ethnicity: Examined AA and HL subsets Teacher condition: 0=did not read essays; 1=did read essays Child condition: self=1 0; group=0 1; neutral=0 0 Quarterly grades in Math, Language Arts, and Social Studies (100 point scale) (pre- and post-EOG scores in math and reading)

PROCEDURES: Random assignment of students and teachers Preparation of envelopes for students Distribution of envelopes and writing exercise Teachers in reading condition, read essays End of year preparation of grade data linked to study codes (anonymous data provided to researchers) Durham Study

HYPOTHESES: 1. Students whose teachers read their essays will have better grades. 2. Students who receive the self-affirming prompt will have better grades than those who receive the neutral prompt. 3. Students who receive the group-affirming prompt will have better grades than those who receive the neutral prompt. 4. Students who receive an affirmation prompt and whose teachers read their essays will have the best grades.

ANALYSIS: 3-Level Hierarchical Linear Modeling Quarters at level 1 (nested in individual students) Individual students at level 2 Classrooms at level 3 Random intercept and slope (quarters) Tested all 3-way interactions and lower terms To avoid 4-way interactions and to identify within- group effects, analyzed AA and HL separately Durham Study

Results Overview Four significant treatment effects were found  All involved 3-way interactions  2 for African American students  2 for Hispanic Latino students  Generally in hypothesized direction  Language arts and social studies  No effects found on math

Interaction for African American Students Among Time, Teacher Condition, and Writing Prompt: Social Studies Grades ……………….

Interaction for African American Students Among Time, Gender, and Writing Prompt : Language Arts Grades girls

Interaction for Hispanic/Latino Students Among Time, Gender, and Writing Prompt: Social Studies Grades

Interaction for Hispanic Latino Students Among Time, Teacher Condition, and Writing Prompt: Language Arts Grades

Hypotheses 1.Students whose teachers read their essays will have better grades. African American students whose teachers read their essays had better Social Studies grades than students with teachers who did not read their essays, even when they wrote on the neutral prompt. Hispanic/Latino students who wrote on the group prompt AND whose teachers read their essays had the best language arts grades. Otherwise, teacher reading was not associated with better grades.

Hypotheses 2. Students who receive the self-affirming prompt will have better grades than those who receive the neutral prompt. Hispanic/Latino boys who wrote on the self-affirming prompt had higher grades in Social Studies than did H/L boys who received the neutral prompt. H/L girls with the self-affirming prompt had higher SS grades after Quarter 2 than boys and all those with the neutral prompt, and a positive grade trajectory. African American students who wrote on the self-affirming prompt AND whose teachers read their essays had better grades than students who had other prompts and whose teachers did not read their essays.

3. Students who receive the group-affirming prompt will have better grades than those who receive the neutral prompt. Hypotheses AA students with the group prompt had higher Social Studies grades than those with the neutral prompt at all time points IF their teachers also read their essays. The same was true for HL students with the group prompt for Language Arts grades. African American Girls with the group prompt had Language Arts grades BELOW those with the neutral prompt.

4. Students who receive an affirmation prompt and whose teachers read their essays will have the best grades. Hypotheses Significant interactions between child and teacher conditions occurred for Hispanic/Latino students and for African American students. --For African American students, Social Studies grades were the highest when the child and teacher conditions were combined (teacher read essay, child wrote on self-affirming prompt). --For Hispanic/Latino students, Language Arts grades were the highest when the child and teacher conditions were combined (teacher read essays, child wrote on group-affirming prompt).

Implications Language Arts and Social Studies grades of African American and Hispanic/Latino students appear to be positively affected by a brief writing intervention across middle school grade levels. The intervention affected both intercepts and slopes of quarterly grade trajectories. For African American students, the combination of affirming prompt and teaching reading essays was most effective. For Hispanic/Latinos, the self-affirming prompt was associated with better grades even without teacher reading the essays. Group prompt most effective when combined with teacher reading essay. No effect on Math in current analyses.

Implications Classroom factors that can be easily manipulated may have a notable effect on performance. Consistent with stereotype literature Writing intervention per se probably less important that teacher learning more about students, seeing positive side of student, in combination with student experiencing positive sense of self and identity in the learning location

Still to Do Initial test with incomplete data shows strong effect of the self affirming prompt on Hispanic Latino EOG scores, controlling for prior year EOG scores. Compare grade levels Examine effects within subject area taught by teaching administering the intervention See if there were effects for Other racial/ethnic groups at the school (closing the gap or improving performance of all?)