1 Developing an Evaluation Plan _____________________ The Mathematically- Connected Communities MSP Developed for the February, MSP Conference Dr. Karin Wiburg Dr. Ted Stanford MC 2 – New Mexico

Evolution of a strong MSP In the early days: Summer Academies – often disconnected from what teachers are expected to teach, only some schools and some teachers Isolated well-meaning efforts- diverse stakeholders Decision to create a mathematically- connected community (8 research mathematicians, 8 math educators, a research & evaluation team)- 4 th year Decision to work with only those school districts who will be full partners in implementing math reform district-wide Decision to begin to systematically evaluate the effect of the MSP on teacher knowledge and student learning and achievement

2 MC 2 (Mathematically-Connected Communities) 3 MC2 partners provide professional development and district leadership support throughout the state of New Mexico. You can learn more about MC2 and get many mathematics teaching resources, information on lesson study, and a copy of the latest evaluation plan by going to the MC2 web site mc2.nmsu.edu

Need for Strong Research Models for Evaluation of Mathematics Reform Efforts 4 Evolution in New Mexico: 1.Lots of descriptive research, who did what, when, numbers served, pd provided, some student scores. (systemic change initiatives) 2.Evolution of Teacher Action Research through Lesson Study – 3 years, Takahashi, Lewis 3.NSF required student outcomes studies on another project, we learned to connect PD to student achievement, evolution of mixed-effects model (Remmenga & Wiburg ) 4.An evolving focus on systems research and change of whole districts.

Developing our Evaluation Plan The Overarching Question for MSP Professional Development Projects: What is the effect of our professional development on teacher learning, classroom teaching, and student learning? You need an evaluation team with qualitative and quantitative research abilities

Developing Research Questions 1. Does the MC 2 PD partnership program positively affect student achievement (including all sub-groups) as measured on our state Standards-Based Assessment? 2. Does the MC 2 PD partnership program result in growth in teacher pedagogical content knowledge? 3. Does the level of implementation of the Professional Development in the classroom affect student achievement ?

Turn the Questions into Objectives and Design Measurements/ Analysis 1. Does the MC 2 PD partnership program positively affect student achievement? Objective 1: Provide PD that supports teaching standards-based mathematics Measurements: demographic data, hours and types of PD, workshop evaluations, summer academy evaluations

Turn the Questions into Objectives and Design Measurements 1. Does the MC 2 PD partnership program positively affect student achievement? Obj. 4: Students will show growth in achievement on the NM SBA. Measurements: Use SBA data by district, school, and teacher. Analyze relationships between amount and time of PD and achievement. Analyze relationship between growth in teacher knowledge and achievement

Turn the Questions into Objectives and Design Measurements 3. Does the level of implementation of the Professional Development in the classroom affect student achievement ? Obj. 3: Districts will integrate PD into the school culture (classroom level) Measurements: Classroom snapshots (engagement, type of learning & teaching), Levels of Use (based on CBAM and SEDL guidelines), Horizon Classroom Observation by trained observers.

Turn the Questions into Objectives and Design Measurements 2. Does the MC 2 PD partnership program result in growth in teacher pedagogical content knowledge? Obj. 2: Teachers in partner districts will participate in PD to build pedagogical content knowledge Measurements: Survey for Teacher Knowledge for Teaching Mathematics (Ball et.al, University of Michigan). Mathematicians’ assessment of student growth in summer academy and classroom work.

A Mathematician talks about Pedagogical Content Knowledge 1. Multiple ways to present mathematical topics to meet diverse learning needs. 2. Sensitivity to common student errors and how to address them. 3. Recognizing valid student thinking even when it is not in the lesson plan.

A Mathematician talks about Pedagogical Content Knowledge 4. Choices about remediation: Re- teach, or review in the context of new material? 5. Helping students move on with their learning, even when their background is lacking. 6. Understanding vertical alignment – how does what I teach fit in to the big picture?

How we assess teacher knowledge for teaching mathematics Mathematicians, writing and the one and two week academies The Teacher Survey from Michigan

Time is Up, We will discuss To contact us or the NM MSP write Dr. Ted Stanford Or Dr. Karin Wiburg We are happy to share instruments and evaluation plans and strategies.

Disclaimer The instructional practices and assessments discussed or shown in this presentation are not intended as an endorsement by the U.S. Department of Education.