1. Classroom Network Technology as a Support for Systemic Mathematics Reform: Examining the Effects of Texas Instruments’ MathForward Program on Student Achievement in a Large, Diverse District Corinne Singleton, SRI International

2. The Reform Initiative: MathForward  What: Systemic reform initiative in which teachers integrate classroom network technologies into their mathematics instruction  Goal: improve student achievement in middle school mathematics and algebra  Components:  Double period of math for increased instructional time  Use of TI-Navigator Classroom Learning System, with TI-graphing calculators  Strong administrator support and involvement  Deep, ongoing PD for teachers  Common planning time for teachers  Higher expectations for all students

3. Classroom Network Technologies

4. The District: Richardson, ISD Location/populatio n Suburban community of ~100,000 people in Dallas metropolitan area DemographicsEthnically diverse About 1/3 of students eligible for FRPL AchievementDistrict math scores are above average, relative to other districts in TX Participation in MathForward Involved with MF from the program’s infancy Year 2007-08 was the district’s 3 rd year of participating in MF That year, 13 schools participated: 8MS, 4HS, and 1 center serving freshmen

5. Research Sample  All students in grades 7, 8, and 9 at participating schools in RISD (Treatment group N=817)  Includes 152 MathForward teachers  Some classes assigned to MF, others not  Students in MF tend to be lower-performing than the district average  A majority of students in MF did not meet proficiency standards the year prior to participation  Propensity score matching to create comparison groups that were statistically similar to the treatment group with respect to background and prior achievement was ultimately unsuccessful  Thus, an important limitation: the treatment and comparison groups are not similar on any measured characteristics, so differences in gains may be due to a priori differences in the two groups

6. Evaluation Methods  Pre-post nonequivalent comparison group design to analyze effects of MathForward on student achievement

7. Results: Program-level implementation  The program as implemented was largely congruent with the overall program model, both with respect to classroom-level implementation and the implementation of supporting conditions  All students had double periods of math  All teachers participated in PD related to TI-Navigator use in the classroom  About 2/3 of teachers attended the content-related PD provided by TI  School and district leaders were supportive of MF and of teachers’ participation in the program

8. Results: Classroom-level implementation  Use of Navigator Tools  Teachers made regular use of TI-Navigator tools with students  Higher use of most Navigator tools amongst Grade 7 teachers, as compared with Grades 8 and 9 (particularly for the most powerful Navigator tools)  Use of Interactive Pedagogies  Teachers “sometimes” used interactive pedagogies along with technology  Overall, 7th grade teachers implemented most interactive pedagogies more frequently than 8th and 9th grade teachers

9. Results: Student Achievement  Across all grades, model results indicate a significant positive main effect for MathForward  For 7 th grade, model results indicate a significant positive main effect for MathForward (p<.001)  For 8 th grade, model results indicate a positive association between MathForward and achievement gains, but not statistically significant (p.<.1)  For 9 th grade, model results indicate a significant negative main effect for treatment (p.<.01)

10. Results: the Achievement Gap  In grades 8 and 9, lower scores for African American and Hispanic students in both groups, as compared with White students  In 9 th grade, we found an interaction between treatment assignment and ethnicity: Hispanic students in MathForward lost less ground than Hispanics in the control group  No other significant interaction effects, thus no evidence that MF is closing achievement gaps, except possibly for 9 th grade Hispanics

11. Factors to Consider  Studies of programs in the real world often tell mixed stories  Some criticism of standardized tests, like TAKS, for being insensitive to the kinds of conceptual understanding and problem-solving skills that programs like MathForward can promote  Threats to validity:  Significant differences between the two groups  Potential for regression to the mean when using gains scores as the outcome measure

12. Discussion & Conclusions  Overall, results are suggestive of the promise of MathForward, but models tested here do not allow us to conclude that we have unbiased estimators of program impact  Gains made by 7 th grade MathForward students likely relate to the higher implementation rates amongst 7th grade teachers  Poor results in 9 th grade MathForward classes could relate to the fact that the program was less mature at that grade-level (first year of implementation, compared with third year for grades 7 and 8)*  The differences in findings across grades suggests the need for more research to understand what factors support strong implementation  This study provides an existence proof for integrating classroom network technology into a systemic effort to improve mathematics teaching and learning at the district level

13. Thank you!  Questions, comments welcome  For more information, contact Corinne Singleton at corinne.singleton@sri.com or Bill Penuel at william.penuel@sri.com corinne.singleton@sri.com william.penuel@sri.com