1. Predictors of Class-to-Class Differences in Adjusted Math-Score Gains at one Middle School: a Preliminary Analysis Using Student Survey Responses Ronald F. Ferguson Wiener Center for Social Policy John F. Kennedy School of Government Harvard University December 2006 Presentation for Conference of the Society for Research on Educational Effectiveness (SREE) Lansdowne Conference Center, Lansdowne, VA

2. Fact: Past research has not produced clear answers regarding why some teachers are more effective than others at producing test score gains. Core questions for this project: Why do some middle school classrooms achieve higher average math score gains than others? Do student responses to survey questions reveal patterns in student- perceived attitudes and practices of teachers in classrooms that differ in gain patterns?

3. For four patterns of classroom-level achievement gains, the slides in this presentation show percentages answering “True” or “Mostly True,” instead of “Somewhat,” “Hardly at All,” or “Totally Untrue,” for survey items in the following categories:  Teacher Demands  Teacher Encouragement  Teacher Sensitivity to Students’ Feelings  Teacher Responsiveness to Questions  Teacher’s Use of Multiple Explanations  Quality of the Teacher’s Explanations Classroom Goal Orientations:  Real Comprehension  Getting Good Grades

4. 76% White Students, 14% Blacks, 10% Others Gains are for Spring 2005 to Spring 2006 Unadjusted gains at the student level are 2005-to-2006 changes in z-values, defined on the 2005 and 2006 distributions of math scores among classmates at the school. For each student, the adjusted gain is his or her unadjusted gain, minus the average unadjusted gain in the student’s decile of the spring 2005 score distribution. Predictors are from classroom-level student surveys conducted spring 2006. This presentation focuses on graphical analysis. However, the key findings are also statistically significant in multinomial logit analyses.

5. The analysis focuses on two groups of students, across four types of classrooms. “Low Achievers” had a baseline score in the school’s bottom two quintiles. “High Achievers” had a baseline score in the school’s second two quintiles.* Definitions of four classroom types In classroom types:Low achievers have:High achievers have: “Low, Low” (LL)Low Gains “High, High” (HH)High Gains “High, Low” (HL)High GainsLow Gains “Low, High” (LH)Low GainsHigh Gains *Note: Students in the top quintile of the baseline distribution or in advanced math classes are not consideration in what follows.

6. Class Type LL Class Type HH Class Type HL Class Type HH # Low Achievers N=42N=11N=38N=29 # High Achievers N=50N=13N=29N=15 # of Classes N=4N=1N=4N=3 The sample consists of 7 th and 8 th grade math students. Numbers of students and classes of each type are as follows:

7. Adjusted math gains

8. “The teacher in this class demands that the students work hard.” The teacher in this class encourages me to do my best.” “When I work hard in this class, an important reason is the teacher demands it. Teacher Demands for Hard Work “When I work hard, an important reason is that the teacher encourages me.” Teacher Encouragement

9. “My teacher in this class makes me feel that he/she truly cares about me.” “My teacher in this class treats the students with respect.” “My teacher really tries to understand how students feel about things.” Teacher Sensitivity “I feel close to my teacher in this class.

10. “My teacher has several good ways to explain each topic we cover.” “The teacher in this class welcomes questions if anyone gets confused.” “If you don’t understand something, my teacher explains it another way.” “My teacher likes it when we ask questions.” Teacher Responsiveness to Questions Teacher’s use of Multiple Explanations

11. “In this class, really understanding the material is the main goal.” “My teacher wants me to explain my answers – why I think what I think.” “This class makes me a better thinker.” “In this class, getting good grades is the main goal.” Classroom Goal Orientations: Real comprehension and/or getting good grades

12. “I could do much better in this class if I worked harder.” Hours/week math homework. “One of my goals is to show others that class work is easy for me. “If I didn’t understand something, my classmates would be happy to help me.” Possibilities for why Low Achievers Gain more in HH than in LH classrooms.

13. “I have pushed my self hard to completely understand my lessons in this class.” “I am satisfied with what I have achieved in this class.” “I have done my best quality work in this class all year long.” “Our class stays busy and doesn’t waste time.” Working Hard and Staying Busy Performance and Satisfaction

14. What seems most different about the HH classroom? Teacher Demands that students Work Hard  Teacher Encouragement to Do their Best  Teacher Sensitivity to Students’ Feelings [LOW]  Teacher Responsiveness to Questions [LOW]  Teacher’s Use of Multiple Effective Explanations Classroom Goal Orientations: Real Comprehension is the goal  Getting Good Grades is the goal  Possibilities for why Low Achievers Gain more in HH than in LH classrooms. Working Hard and Staying Busy High Self-Assessed Performance and Satisfaction