1. LMT Sharing Session: April 2011 Vicky Kukuruda Riverside County Office of Education Patty O'Driscoll Public Works, Inc. Susan Tucker Evaluation & Development Associates LLC

2. AGENDA Content of LMT Appropriate use and fidelity to research embedded in the LMT project Strengths and challenges of using LMT Overview of options Project examples

3. LMT Overview Developed by researchers led by Deborah Ball at the University of Michigan Builds off of research begun at the MPDI (Math Professional Development Institutes) Supported by NSF since 2002 Focuses on nature of the mathematical knowledge needed for teaching –Special focus on instructional practice that can intervene on significant patterns of educational inequality in mathematics education

4. Design of the LMT Designed to measure effectiveness of PD and its impact on the mathematical content knowledge that teachers need to teach mathematics well LMT Design team develops the test through writing, piloting, and analysis of problems that reflect real math tasks facing teachers LMT piloted with the help of over 2000 teachers –Mapping item pool against the NCTM and California standards Can be used to study: –content-focused PD – teacher learning from pre-service coursework –curriculum materials and projects exploring contribution of teacher knowledge to student achievement

5. What you learn in a training Training at UM and national conferences –no cost for training or for the measures themselves, but users must cover transportation and accommodation expenses To use the LMT in a research project, you must attend an LMT training Literature and research framework for the development of the assessment system How test design is embedded in mathematics professional development How it is different from past efforts to measure effectiveness of PD When to use it and when not

6. Training Topics LMT measures & their development Early statistical work and their measures Item Response Theory (IRT) basics –focus on its use in scoring the assessment Orientation to scales and forms Using technical reports to understand measures Terms of use of the LMT Administration & lessons learned from experience Teaching Knowledge Assessment System (TKAS) (new on-line version of LMT)

7. Content covered in LMT about teacher content knowledge Number and operations (K-6, 6-8) Patterns, functions, and algebra (K-6, 6-8) Geometry (3-8) Topic-specific modules (4-8) in: – Rational number – Proportional reasoning – Geometry – Data, probability, and statistics

8. Appropriate Use of the LMT LMT does NOT offer measures that can be used for hiring, promotion, pay, or tenure LMT is NOT designed to make highly accurate statements about individuals' math knowledge LMT does NOT participate in study design or data analysis or provide PF to teachers or schools LMT is for comparing groups of teachers' mathematical knowledge, or examine how a group of teachers' knowledge develops over time

9. LMT Assumptions To make progress in developing theory of knowledge for teaching To develop items that measured math that teachers use in teaching, not just what they teach To orient items around problems that all teachers might face in teaching math To produce useable measures: –With items that are not grade-specific –With items that do not represent any single view of teaching (e.g., “reform”) –Which can discriminate among teachers’ capability in this area

10. Fidelity of Use of the LMT Want to measure range of teacher ability reliably –Need easy, medium, difficult items –No evidence that teachers must answer any of these items to be effective Limits conclusions –50% average ≠ “teachers failing” –No conclusions about individuals’ competency

11. Using the LMT Making Selections Choose content domain Identify forms that match your professional development content Consider the ability of your participants –Consider short AND long term development Select forms –match to ability and content of PD –look for high reliability scales

12. Available Scale Measures Elementary number and operations (K-6, 6-8) Elementary patterns, functions, and algebra (K-6, 6-8) Geometry (3-8) Rational number (4-8) Proportional Reasoning (4-8) Data, probability, and statistics (4-8) –Each K-6 item has each been piloted with over 600 elementary teachers, –Middle school items have each been piloted with over 300 middle school teacher

13. Sample Item- Elementary

14. 5. Mrs. Johnson thinks it is important to vary the whole when she teaches fractions. For example, she might use five dollars to be the whole, or ten students, or a single rectangle. On one particular day, she uses as the whole a picture of two pizzas. What fraction of the two pizzas is she illustrating below? Mark ONE answer a)5/4 b) 5/3 c)5/8 d)1/4

15. Sample Item- Elementary 2. Ms. Chambreaux’s students are working on the following problem: Is 371 a prime number? As she walks around the room looking at their papers, she sees many different ways to solve this problem. Which solution method is correct? (Mark ONE answer.)

16. Sample Item-Middle School

17. Other considerations/Examples Pros and Cons –On-line system online using the Teacher Knowledge Assessment System (TKAS) available to those who have previously attended an item training workshop. contact Katherine Mikesell at kmikesel@umich.edu –Computer Adaptive testing –Paper-based assessments Examples to Share –Riverside –NSF MSP ACES grant at CSUSB –Downey Discussion and Questions

18. Augmenting the LMT at CSUSB NSF MSP ACES grant directed by Dr. Davida Fischman Working in Ontario-Montclair School District Demonstrating certain LMT items with think aloud protocol Video taping lesson study –Adapting MQI protocol to analyze videos –Mathematical Quality of Instruction (MQI) –Online training for MQI: Nina Cohodes at nina_cohodes@gse.harvard.edu or 617.496.4815.

19. Example of LMT use in Riverside: 110 Teachers grades 3 - Algebra 1 4 small feeder K-8 districts, High School district and County Special Education and Alternative Education teachers Objective 1: Teacher Content Knowledge 3 year Content Focus: –Numbers and Expressions –Proportional Reasoning –Functions and Equations

21. Cycles of Administration: First Administration – Pre A 3 surveys on first day of institute (first hour) Second Administration- Post B Saturday before we released them Third Administration- Post A 2 hour open session with a series of afterschool choices/locations

24. Objective 1: Teacher Content Knowledge DELTA teachers increased performance on the LMT MKT from pre to mid (ss) DELTA Regular Ed teachers performed better than Alt Ed/SPED (RCOE) teachers on the pre LMT MKT, however, Alt Ed/SPED teachers made greater gains and were performing at the same level by mid year.

25. Q & A about LMT What are questions you have about: –Content –Use –Challenges –Ways to leverage with other data sources

26. Citation for the work Copyright © 2006 The Regents of the University of Michigan. For information, questions, or permission requests please contact Merrie Blunk, Learning Mathematics for Teaching, 734-615-7632. Not for reproduction or use without written consent of LMT. Measures development supported by NSF grants REC-9979873, REC- 0207649, EHR-0233456 & EHR 0335411, and by a subcontract to CPRE on Department of Education (DOE), Office of Educational Research and Improvement (OERI) award #R308A960003.

27. Contacts for more info: Vicky Kukuruda: –VLKUKURUDA@rcoe.us Patty O'Driscoll: –patty@publicworksinc.org Susan Tucker: –sutucker1@mac.com