1 Milwaukee Mathematics Partnership Program Evaluation Year 6 Results Carl Hanssen Hanssen Consulting, LLC Cindy Walker University of Wisconsin-Milwaukee November 2009 MTL Meeting

2 Where are we?  MMP efforts have been ongoing since 2003  This is the 6 th year we have been conducting the evaluation  We anticipate continuing effort for several more years

3 MMP Support  Original funding from NSF Started in Currently in 7 th year  MMP Phase II funding from NSF Awarded last fall for 3 years Focus on research and evaluation  State of Wisconsin Provides funding for released MTL positions

4 Evaluation Goals  Help the MMP better serve its constituents and improve its effectiveness  Serve the broader mathematics education community through documentation and dissemination of MMP success factors

5 Key Evaluation Question What are the critical factors or conditions promoted by the MMP that are related to student achievement?

6 Agenda 1. MMP Involvement 2. Social Network Analysis 3. Learning Team and Math Meeting Observations 4. Mathematical Knowledge for Teaching 5. Conclusions 6. Next Steps

7 Thank You!  142 schools provided data for the MMP Online Survey producing a record response rate!  114 schools provided social network analysis data!  25 schools allowed us to observe meetings!  Many others have provided data for a variety of other measures!

8 1. MMP Involvement Metrics:  MMP Involvement Attendance at MTL Meetings MMP Courses  WKCE Student Achievement Growth from Achievement

9 MMP Involvement & Growth Schools that have been more involved over time demonstrated higher student achievement growth from 2005 to 2008: Average % Growth Number of Schools: high (n=34), medium (n=91), low (n=26), none (n=27)

10 MMP Involvement & 2008 Achievement Schools that are more involved demonstrated higher student achievement in 2008: Average % Proficient Number of Schools: high (n=34), medium (n=97), low (n=31), none (n=36)

11 2. Social Network Analysis We asked you to list individuals with whom you communicated about mathematics education in the past several months. For each individual: 1. Do they work at your school? 2. Their role 3. Frequency of communication 4. Extent of collaboration

12 Key  Maps identify MTL or MTS or Teachers Principal Literacy Coach Others in school Others outside school  Statistics Network density In-School density MTL In-Degree MTL Betweeness MTS IN-Degree Benefit—a graphical AND statistical description of school-based networks.

13 Hi Density

14 Lo Density

15 Hi MTL In-Degree

16 Lo MTL In-Degree

17 Hi MTL Betweeness

18 Hi MTS In-Degree

19 So what? A combination of factors…all are necessary but no single factor is sufficient Math Focus LT Quality Network Density MTL In-Degree MTL Betweeness Predicts Student Success Discriminant function analysis Indicates that this combination of factors predicts achievement quartiles

20 3. Meeting Observations  Overall, LT meetings tended to focus more on ‘administration’ than ‘learning’ though higher performing teams retain more of an emphasis on learning  Math meetings more focused, yet less team oriented  Common observations from math meetings: Real work is typically accomplished (e.g., scoring CR items) Substantive mathematical discussions take place Implications for classroom practice are rarely discussed as a basis for wrapping up

21 4. Mathematical Knowledge for Teaching  Two groups of teachers took the MKT assessment in Math Teacher Leaders Self-selecting Math Teachers

22 MTL MKT— Results Grade 8-9 MTLs consistently score higher than the K-7 MTLs on both pre-tests and post-tests…especially in Geometry and Prob & Stat.

23 MTL Compared with Classroom Teachers MTLs consistently score higher than classroom teachers in all areas of the MKT assessment.

24 5. Overall Conclusions  Schools that have more heavily participated in MMP-sponsored activities are demonstrating greater student achievement growth.  There appears to be a set of necessary conditions for improving student achievement—no single factor is sufficient.

25 Overall Conclusions  Productive work is taking place at math-focus meetings in the context of grade-level meetings, collaborative planning time, or other special sessions.  A shortcoming of these sessions is often not considering implications for classroom practice based on the good work being done.

26 Overall Conclusions  MTLs have regularly demonstrated math content knowledge gains during the academic year, and there is some evidence to suggest these gains are being sustained over time.  MPS Math Teachers MKT results are generally lower than that of the MTLs. Slight improvements over time have been observed.

27 6. Evaluation  MMP Online survey in May 2010  Continue to focus on 25 case study schools for the next 2 years Math meeting observations SNA School honorarium  Implement SNA in most schools across the district

28 Small Group Discussion 1. What implications does the information presented have for MTLs as they promote PD efforts in their building? 2. What insights have you gained about the work of the MTLs? How will that information impact your work in your building? 3. What are some areas you can enhance or strengthen in your work with teachers that will make an impact on student achievement? 4. According to M. Fullan, “The role of the leader is to ensure that the organization develops relationships that help produce desirable results.” As you reflect on Fullan’s work about building relationships, what connections can you make to your work and the information just shared?