Michigan MSPs June 2007 Wendy Tackett, PhD, iEval

Shared Instrumentation Measures of: Teacher knowledge Michigan Test for Teacher Certification (MTTC) Learning Mathematics for Teaching (LMT) Instructional knowledge & curriculum implementation Survey of Enacted Curriculum (SEC) Science and Mathematics Program Improvement (SAMPI) lesson observations Principal knowledge Lenses on Learning (LOL) Student knowledge MEAP

Shared Comparison Group Designed not to deny services to teachers & students Share the costs among participants Logistics Non-participants from Macomb County ISD (round 1) served as 1st comparison group Separate unfunded district (round 2) served as comparison group Resulted in a rigorous evaluation design at a reduced cost

Collaboration is Essential At the administrative-level Focused efforts Reduction in costs Time savings At the teacher participant-level Create a community of learners Increased knowledge base Utilize technology to save time and increase learning Maximize the benefits for student learning

Common Results Sharing resources to make for more accurate analyses Must consider many variables Teacher participation levels in coaching Teacher participation in math institutes Principal participation in program Baseline data about teacher instruction Baseline data about teacher content & pedagogical content knowledge

Common Results: SAMPI Statistically significant improvements in: Overall rating for classroom instruction Subcategory ratings of lesson implementation, lesson content, & classroom culture Substantive student-student interaction Appropriate pacing for all student levels Adequate reflection time Growing use & comfort with teacher use of technology Differences between teachers using standards- based math curricula vs. traditional curricula

Common Results: SAMPI 2006 Spring MSP conference in Boston Iris Weiss discussed impact of building climate on student learning We developed survey that addressed five elements of building climate Completed one-way ANOVA with climate score and various observational elements from SAMPI observation Found statistically significant differences (p<.05) in behaviors observed in the classroom between teachers in buildings with high climate ratings (3.0 or higher) and the rest of classrooms

Common Results: SAMPI Example from MERC, Oakland Schools

Common Results: LMT & MTTC Significantly greater knowledge of mathematical content and pedagogy Attendance at three or more content- focused or curriculum-based courses had a statistically significant positive impact on the pedagogical content knowledge of teachers Teacher growth in math content knowledge highly related to student motivation in math

Common Results: LMT LMT ALMT B Attend 3 or more courses 56%69% Attend 1-2 courses 36%54% Comparison group 51%48% Example from MERC, Oakland Schools

Common Results: MTTC Example from Washtenaw County

Common Results: MEAP Round 2 projects still too early to determine impact on student academic achievement Round 1 projects confirmed increases on MEAP math in elementary and middle grades

Common Results: Coach Observations & SEC Teachers are exhibiting deeper understandings of math - leading to more math talk & student discourse Teachers strongly support the professional development sessions Teachers report greater instructional focus and alignment to state standards Teacher concerns moved from logistics to student learning

Keys to Success in Evaluation Collaborate with your colleagues Share tools Share comparison groups Share findings & suggestions for improvement Easier cross-program comparisons Maximizes resources in purchasing tools & analyzing results More robust comparisons (higher n)

For More Information Michigan MSP website MTTC Study Guide Survey of Enacted Curriculum (SEC) Learning Mathematics for Teaching (LMT) SAMPI