MSP Evaluation in the Far North Evaluator Collaboration Evaluation Design Action Research Studies

Evaluation A collaborative sport Not a competitive sport Not a spectator sport Adapted from Kenneth Gross

Benefits of MSP Collaboration Comparison groups and improved research design Instrumentation

The Evaluation Model Level 1: Response Level 2: Acquisition of Knowledge and Skills Level 3: Opportunity Level 4: Program Implementation Level 5: Impact on Students

Level 2: Knowledge and Skills Learning for Math Teaching (LMT) Student A Student B Student C For more information contact Heather Hill at Which of these 3 students is using a method that could be used to subtract any two whole numbers?

Level 2: Acquisition of Knowledge & Skills Measure of Math Pedagogy: Learning for Mathematics Teaching Scale (LMT)

Level 2: Acquisition of Knowledge & Skills (cont) Measure of Math Content Knowledge: MI Test of Teacher Cert Study Items (MTTC)

Level 3: Opportunity Focus Groups: Do teachers have the time, resources and support to implement the program?  Initial concerns  Challenges

Level 4: Implementation Science and Math Program Improvement Scale

Level 4: Implementation (cont) Science and Math Program Improvement Scale

Level 4: Implementation (cont) Survey of Enacted Curriculum (SEC)

SEC Map: MI Standards for Grade 6

Evaluation A lot like Action Research A living process, never a finished plan Active, requiring twists, turns, new paths

When Your Results Don’t Quite Make Sense, Add Them Up Town of Altamont, Michigan: Altitude 500 Population 3,500 Total 4,000

Action Research in Three Michigan MSP Evaluation Projects Replication Study from Cognitively Guided Instruction Relationship of Student Motivation to Teacher Growth in Content Knowledge Relationship of Program Implementation to School Climate

Action Research 1: Washtenaw County’s Replication Study: Cognitively Guided Instruction (Carpenter et al)

Action Research 1: Washtenaw Replication Study (cont)

Action Research 2: Wayne County’s Study of Student Motivation and Teacher Content Knowledge

Action Research 3: Oakland County’s Climate Study 2006 Spring MSP conference in Boston  Iris Weiss discussed impact of building climate on student learning We asked ourselves where building climate fits within our theory of change Developed survey that addressed building climate

Climate Survey

Analysis Developed average climate score for each building - mean across five climate elements - based on surveys submitted from teachers, administrators, and coaches Completed one-way ANOVA with climate score and various observational elements from SAMPI observation Found statistically significant differences (p<.05) in behaviors observed in the classroom between teachers in buildings with high climate ratings (3.0 or higher) and the rest of classrooms

Analysis

MSP Evaluation in Summary Collaborative Ongoing Action Research Oriented