1. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt1 Developing Measures of Mathematical Knowledge for Teaching Geoffrey Phelps, Heather Hill, Deborah Loewenberg Ball, Hyman Bass Study of Instructional Improvement Learning Mathematics for Teaching Consortium for Policy Research in Education University of Michigan NSF/MSP State Coordinators Meeting October 20, 2005

2. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt2 Overview of today’s session 1. LMT/SII Measures Development 2. Some Sample Results 3. LMT/SII Measures and Dissemination

3. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt3 Subtract: What is “Content Knowledge for Teaching”? An Example From Subtraction

4. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt4 Analyzing Student Errors

5. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt5 Analyzing Unusual Student Solutions

6. LMT/SII Measures Development

7. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt7 Why Would We Want to “Measure” Teachers’ Content Knowledge for Teaching? To understand “what” constitutes mathematical knowledge for teaching To understand role of teachers’ content knowledge in students’ performance To study and compare outcomes of professional development and teacher education To inform design of teachers’ opportunities to learn content knowledge

8. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt8 Measuring Teachers’ Mathematics Knowledge: Background and History Research on teacher behavior Early research on student achievement –Proxy measures for teacher knowledge –Tests of basic skills 1985 on: “the missing paradigm” pedagogical content knowledge 1990s: interview studies of teachers’ mathematics knowledge (MSU -- NCRTE)

9. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt9 Study of Instructional Improvement Study of three Comprehensive School Reforms; teacher knowledge a key variable Instrument development goals: –Develop measures of content knowledge teachers use in teaching K-6 content for elementary school teachers Not just what they teach - specialized knowledge –Develop measures that discriminate among teachers (not criterion referenced) –Non-ideological But we faced significant problems….

10. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt10 Problems As We Began This Work No way to measure teachers’ content knowledge for teaching on a large scale –Small number of items, many written by Ball, Post, others appeared on every instrument –Nothing known about the statistical qualities of those items (difficulty, reliability) –Studies relied on single items, yet single items unlikely valid or reliable measures of teacher knowledge

11. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt11 Early Decisions and Activity Survey-based measure of CKT-M –3000 teachers participating in SII –Multiple choice Specified domain map 5 people + 5 lbs cheese + 5 weeks = 150 items (May 2001) Large-scale piloting, summer 2001

12. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt12 Early Decisions and Activity Types of knowledge Mathematical content Content knowledgeKnowledge of content and students Number Operations Patterns, functions, and algebra

13. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt13 Early Analyses and Validity Checks Results from piloting –We can measure teachers’ CKT –CK reliabilities.70-.90 –Factor analysis shows distinct types of knowledge Knowledge of content and students (KCS) separate from CK Specialized content knowledge (SCK) vs. common content knowledge (CCK)

14. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt14 Overarching Findings: Range of Piloted Reliabilities (IRT) Knowledge of contentKnowledge of content and students Number and operations (K-6).71-.89.51-.78 Patterns, functions, and algebra (K-6).77-.87 Geometry (3-8).92

15. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt15 Content Knowledge : Number and Operations Common knowledge –Number halfway between 1.1 and 1.11 Specialized knowledge –Representing mathematical ideas and operations –Providing explanations for mathematical ideas and procedures –Appraising unusual student methods, claims, or solutions

16. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt16 Representing Number Concepts Mrs. Johnson thinks it is important to vary the whole when she teaches fractions. For example, she might use five dollars to be the whole, or ten students, or a single rectangle. On one particular day, she uses as the whole a picture of two pizzas. What fraction of the two pizzas is she illustrating below? (Mark ONE answer.) a) 5/4 b) 5/3 c) 5/8 d) 1/4

17. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt17 Providing Mathematical Explanations: Divisibility Rules Ms. Harris was working with her class on divisibility rules. She told her class that a number is divisible by 4 if and only if the last two digits of the number are divisible by 4. One of her students asked her why the rule for 4 worked. She asked the other students if they could come up with a reason, and several possible reasons were proposed. Which of the following statements comes closest to explaining the reason for the divisibility rule for 4? (Mark ONE answer.) a) Four is an even number, and odd numbers are not divisible by even numbers. b) The number 100 is divisible by 4 (and also 1000, 10,000, etc.). c) Every other even number is divisible by 4, for example, 24 and 28 but not 26. d) It only works when the sum of the last two digits is an even number.

18. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt18 Which of these students is using a method that could be used to multiply any two whole numbers? Appraising Unusual Student Solutions

19. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt19 Common vs. Specialized CK Appears in exploratory factor analyses on 2/7 forms; confirmatory on 3/7 Individuals can be strong in common but not specialized; vice versa Suggests there is professional knowledge for teaching

20. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt20 Ongoing Work Item and measures development –Middle school national probability study (KCS, KCT) –Data analysis and probability –PFA equating Validation efforts –“Videotape” study –Cognitive tracing studies –Content validity checks

21. Some Sample Results

22. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt22 An Example : Establishing a Relationship to Student Growth

23. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt23 Links to Study of Instructional Improvement Student Achievement Analysis SII CKT-M measure – 38 items –SII:.89 IRT reliability Model: Student Terra Nova gains predicted by: –Student descriptors (family SES, absence rate) –Teacher characteristics (math methods/content, content knowledge) Teacher content knowledge significant –Small effect (LT 1/10 standard deviation) –But student SES is also on same order of magnitude

24. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt24 A Second Example : Evaluating Teacher Professional Development

25. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt25 Tracking Teacher Growth Items piloted in California’s Mathematics Professional Development Institutes (MPDI) –Instructors: Mathematicians and mathematics educators –40-120 hours of professional development –Focus is squarely on mathematics content –Summer 2001 –Pre/post assessment format (parallel forms)

26. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt26 MPDI Teacher Growth (Year 1) For all institutes for which we have data, teachers gained.48 logits, or roughly ½ standard deviation Translates to 2-3 item increase on assessment Considered substantial gain

27. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt27 Results from Sample Institutes

28. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt28 MPDI Evaluation: Other Findings Length of institute predicts teacher gains –120-hour institutes most effective, on average –But some 40-hour institutes very effective Focus on mathematical analysis, proof, and communication leads to higher gains Many questions remain –Effects of content (e.g., mathematics vs. student thinking) –Treatment of content: common vs. specialized –Effects of teacher motivation

29. LMT/SII Measures and Dissemination

30. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt30 Current Item Pool Elementary School (K-6) –Number and operations / Knowledge of content –Number and operations/ Knowledge of content and students –Patterns functions and Algebra/ Knowledge of content

31. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt31 Current Item Pool Middle School –Number and operations / Knowledge of content –Patterns functions and Algebra/ Knowledge of content Geometry (3-8)

32. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt32 Item Workshops and Dissemination Interested users attend a one-day workshop in Ann Arbor We cover –History of item development –Analytic methods and validation studies –How to use technical materials Users get –Access to measures –Support materials

33. www.sii.soe.umich.edu http://sitemaker.umich.edu/lmt33 Dates and Contact Information Learning Mathematics for Teaching –http://sitemaker.umich.edu/lmt Dates for LMT Workshops –November 10, 2005 –January 13, 2006 –Brenda Ely (elyb@umich.edu) Geoffrey Phelps –gphelps@umich.edu –734-615-6076