1. MSP Evaluation in the Far North Evaluator Collaboration Evaluation Design Action Research Studies

2. Evaluation A collaborative sport Not a competitive sport Not a spectator sport Adapted from Kenneth Gross

3. Benefits of MSP Collaboration Comparison groups and improved research design Instrumentation

4. The Evaluation Model Level 1: Response Level 2: Acquisition of Knowledge and Skills Level 3: Opportunity Level 4: Program Implementation Level 5: Impact on Students

5. Level 2: Knowledge and Skills Learning for Math Teaching (LMT) 932356+4 -356360+40 400+500 900+32 932576 Student A 932 932 -356 -300 632 - 50 582 - 6 576 Student B 932 936 976 -356 -360 -400 576 Student C For more information contact Heather Hill at hhill@umich.edu Which of these 3 students is using a method that could be used to subtract any two whole numbers?

6. Level 2: Acquisition of Knowledge & Skills Measure of Math Pedagogy: Learning for Mathematics Teaching Scale (LMT)

7. Level 2: Acquisition of Knowledge & Skills (cont) Measure of Math Content Knowledge: MI Test of Teacher Cert Study Items (MTTC)

8. Level 3: Opportunity Focus Groups: Do teachers have the time, resources and support to implement the program?  Initial concerns  Challenges

9. Level 4: Implementation Science and Math Program Improvement Scale

10. Level 4: Implementation (cont) Science and Math Program Improvement Scale

11. Level 4: Implementation (cont) Survey of Enacted Curriculum (SEC)

12. SEC Map: MI Standards for Grade 6

13. Evaluation A lot like Action Research A living process, never a finished plan Active, requiring twists, turns, new paths

14. When Your Results Don’t Quite Make Sense, Add Them Up Town of Altamont, Michigan: Altitude 500 Population 3,500 Total 4,000

15. Action Research in Three Michigan MSP Evaluation Projects Replication Study from Cognitively Guided Instruction Relationship of Student Motivation to Teacher Growth in Content Knowledge Relationship of Program Implementation to School Climate

16. Action Research 1: Washtenaw County’s Replication Study: Cognitively Guided Instruction (Carpenter et al)

17. Action Research 1: Washtenaw Replication Study (cont)

21. Action Research 2: Wayne County’s Study of Student Motivation and Teacher Content Knowledge

22. Action Research 3: Oakland County’s Climate Study 2006 Spring MSP conference in Boston  Iris Weiss discussed impact of building climate on student learning We asked ourselves where building climate fits within our theory of change Developed survey that addressed building climate

23. Climate Survey

24. Analysis Developed average climate score for each building - mean across five climate elements - based on surveys submitted from teachers, administrators, and coaches Completed one-way ANOVA with climate score and various observational elements from SAMPI observation Found statistically significant differences (p<.05) in behaviors observed in the classroom between teachers in buildings with high climate ratings (3.0 or higher) and the rest of classrooms

25. Analysis

26. MSP Evaluation in Summary Collaborative Ongoing Action Research Oriented