The Impact of the MMP on Student Achievement Cindy M. Walker, PhD Jacqueline Gosz, MS University of Wisconsin - Milwaukee

Primary Goal Within a school it is ultimately the teacher that is responsible for teaching mathematics that has the greatest capacity to impact student achievement in mathematics. How is the MMP influencing what is going on in mathematics classrooms across the district?

Structure of MMP in MPS

Concerns Much of the effect of the MMP is not under the direct control of the MMP The effect of the MMP on student achievement depends on –School Climate –Quality of Learning Team –Quality of MTL –Support of other Teachers

Solutions Capture school level variability in variables that are not under the direct control of the MMP Compare and contrast schools to determine if under the best conditions there is an increase in student achievement in mathematics

Methodology On-line survey made available to all MPS participants in the MMP that may have an effect on increasing student achievement in mathematics including –Learning Team Members –Literacy Coaches –Math Teacher Leaders –Math Teachers

Participation Rate A total of 1,107 MPS employees from 129 schools in the district responded to the survey 93% of the elementary schools, 90% of the middle schools, and 9% of the high schools were represented in varying degrees.

Participation Rate by Role SchoolNLCLTMTLMT%MT Elementary1, % Middle % High130229

Instrumentation Six primary areas were evaluated 1.Professional Development 2.School/District Functioning 3.Learning Team Functioning 4.Classroom Practice 5.Role of the MTL 6.Role of the LC

Professional Development How involved were respondents in organized professional development activities geared toward mathematics this year? Role in the MMPNMeanSD Literacy Coach Learning Team Member LT Member & Math Teacher Math Teacher Leader Math Teacher Only

Professional Development How often did respondents participate in professional learning experiences related to the teaching and learning of mathematics this year? Role of Respondent NMeanSD Literacy Coach Learning Team Member LT Member & Math Teacher Math Teacher Leader Math Teacher Only

Professional Development How involved were respondents in informal professional development activities geared toward mathematics this year? Role of RespondentNMeanSD Literacy Coach Learning Team member only LT member & Math Teacher Math Teacher Leader Math Teacher Only

Professional Development How useful were professional development activities, in terms of helping teachers improve their classroom teaching of mathematics? Role of RespondentNMeanSD Learning Team Member Math Teacher Leader Math Teacher Only

School Functioning How strong is the focus on increasing student achievement in mathematics in MPS schools? There is a Strong Focus on Increasing Student Achievement in Mathematics at My School Role of RespondentSDDASA LC (n = 47).0%17.4%32.6%50.0% LT (n = 83)1.3%6.3%38.8%53.8% LT_math (n = 170)4.2% 34.3%57.2% MTL (n = 126)3.2%12.0%40.0%44.8% MT (n = 692)3.7%8.2%42.9%45.2%

School Functioning How cohesive are the mathematics curriculum and pedagogical approaches used in MPS schools? SDDASA Teachers have different expectations about what students can learn 7.6%42.0%37.9%12.5% Teachers often individually choose the content they teach 12.6%42.1%36.0%9.3% Teachers are encouraged to use similar instructional practices 3.2%13.8%58.1%24.9% Within each grade, teachers are expected to follow the same curriculum 3.1%8.4%51.9%36.6% Across grades, teachers are expected to follow the same curriculum 3.9%19.4%48.9%27.9%

School Functioning To what extent do respondents participate in school improvement efforts? Role of RespondentNMeanSD Literacy Coach Learning Team Member LT Member & Math Teacher Math Teacher Leader Math Teacher Only

School Functioning How aligned do respondents feel their mathematics program is with standards and assessments? DASA My school’s math program is aligned with MPS learning targets 9.2%57.5%28.6% My school’s math program is aligned with WI state standards 8.9%58.2%28.2% My school’s math program is aligned with the Comprehensive Math Framework 10.5%59.0%23.5% My school’s math program is aligned with state/district assessments (WKCE/Terra Nova) 14.0%56.6%22.4%

School Functioning What information is used to plan and evaluate school improvement activities?

School Functioning

District Functioning How much support do the district’s learning targets provide? –Regardless of role in the MMP, the majority of respondents felt that the district’s math learning targets 1.Provide clear information about what students are expected to know and be able to do and 2.Drive their school’s math improvement agenda

District Functioning How much support do the district’s math teaching specialists provide? –The majority of LC, MTL, and LT members that do not teach math felt that the district’s math teaching specialists provided ample support –Nearly 40% of respondents who teach math teachers did not feel that the district’s math teaching specialists provided ample support. Most likely because they have the least amount of contact with them

Learning Team Functioning Who sets the agenda for LT meetings? –76.5% of LT members indicated that the principal sets the agenda –61.6% indicated that the LC sets the agenda –37.4% reported that the MTL sets the agenda –28.9% reported that someone other than the LC, principal or MTL set the agenda

Learning Team Functioning Who provides input into setting the LT agenda? –86.3% of LT members respondents reported that the principal provides input –77.3% of respondents reported that the LC provides input –69.7% of respondents reported that the MTL provides input –55.5% reported that others provide input

Learning Team Functioning How often is mathematics discussed at LT meetings? –19.6% of LT members (n = 419) reported that math was discussed all of the time –16.1% reported three-fourths of the time –23.3% reported half of the time –37.3% reported one-fourth of the time –3.6% LT members reported never

Learning Team Functioning How often is mathematics a formal agenda item for LT meetings? Role of Respondent MTL (n = 123) Other LT Member (n = 286) Never2.4%2.1% Some of the time36.6%22.7% Most of the time36.6%32.5% Always24.4%42.7%

Learning Team Functioning Does the LT regularly meet to plan and problem solve about issues surrounding mathematics? Role of Respondent MTL (n = 123) Other LT Member (n = 292) Strongly Disagree6.5%3.4% Disagree32.5%18.8% Agree46.3%47.3% Strongly Agree14.6%30.5%

Learning Team Functioning What are the work priorities of LT across the district? –Much variability across the district –Supporting the implementation of improvement efforts in math most often reported as high or top priority –Monitoring the implementation of improvement efforts, developing and implementing staff development programs was least often reported as high or top priority

Learning Team Functioning How often are specific math issues discussed at LT meetings?

Learning Team Functioning

How much of an asset is the LC in helping schools focus improvement in math?

Learning Team Functioning How much of an asset is the MTL in helping schools focus improvement in math?

Learning Team Functioning

How collaborative is the LT? –In general, the majority of all LT members reported that they usually felt comfortable expressing their opinions and the atmosphere of LT meetings was typically respectful, open, and friendly. –To a lesser extent LT members reported a shared responsibility in decision making regarding the implementation of math programs and development in mathematics.

Classroom Practice What pedagogical approaches are being utilized in mathematics classrooms in MPS? –The majority of math teachers in the district reported utilizing all of the pedagogical approaches described which were designed to on at least a weekly basis –These approaches were designed to contrast didactic versus interactive approaches to mathematics instruction

Classroom Practice Compared to other math teachers, a significantly smaller percentage of MTLs reported regularly spending time –Having students solve problems using a method taught by the teacher –Having students complete textbook exercises for review

Classroom Practice A significantly greater percentage of MTLs reported regularly spending time having students –Analyze the similarities among representations, solutions, or methods –Justify why a particular solution strategy in mathematics worked –Work on math problems that had multiple answers or solution methods

Classroom Practice A significantly greater percentage of MTLs and LT members that teach math reported regularly spending time having students –discuss mathematical ideas, problems, or solutions in small groups –write explanations of math ideas, solutions, or methods –practice computational skills

Classroom Practice What content is being taught in mathematics classrooms in MPS? MTL (n = 113) LT_MT (n = 170) MT (n = 692) Number concepts and computation Less than 10 lessons9.9%7.2%12.8% More than 10 lessons90.1%92.8%87.2% Geometry Less than 10 lessons27.5%40.0%47.7% More than 10 lessons72.5%60.0%52.3% Measurement Less than 10 lessons37.6%55.1%63.0% More than 10 lessons62.4%44.9%37.0% Data analysis/statistics Less than 10 lessons33.0%41.4%49.2% More than 10 lessons67.0%58.6%50.8% Probability Less than 10 lessons66.4%72.6%79.0% More than 10 lessons33.6%27.4%21.0% Mathematical patterns/algebra Less than 10 lessons22.5%36.5%38.6% More than 10 lessons77.5%63.5%61.4%

MTL Functioning How often do MTLs interact with others to discuss teaching and learning in math?

MTL Functioning How supported do MTLs feel in their role?

MTL Functioning How closely are MTLs working with Math Teaching Specialists?

LC Functioning How involved is the LC in helping school improve in mathematics?

Discussion It is apparent that quite an impact has been made by the MMP in the majority of elementary and middle schools in MPS this year It is also apparent that there is quite a lot of school level variability in the level of impact that has been made in the district.

Discussion In some schools within the district, improving student achievement in mathematics appears to be a top priority In other schools in the district improving student achievement in mathematics does not appear to be a priority

Discussion This level of variability is to be expected with any educational improvement plan that is implemented on such a large scale Only by capturing such variability can the impact of the MMP on increasing student achievement in mathematics be evaluated

Discussion Is there an increase in student achievement in mathematics at schools in the district that are 1.Utilizing all the resources provided to them though the MMP? 2.Doing everything within their control to collaborate, plan and implement procedures designed to improve the teaching and learning of mathematics?

Discussion This is the direction in which the evaluation of the effect of the MMP on student achievement will proceed It is expected that the variability that was observed in school level implementation will help to predict increases in student achievement in mathematics