1. Iris Weiss Barbara Miller Horizon Research, Inc. Education Development Center Professional Development for Teachers of Mathematics: What do we know and how well do we know it?

2. Goal: To synthesize knowledge generated through the Math and Science Partnerships and integrate it into the broader knowledge base for education reform –Deepening Teacher Content Knowledge –Teachers as Intellectual Leaders –Involvement of STEM faculty –Professional Learning Communities MSP Knowledge Management and Dissemination Project

3. A new mathematics district leader, with responsibility for mathematics professional development, comes to you for advice: What should I do to support a solid and effective professional development program across my district?

4. There are many things you are now thinking about. Your task is to talk at your table with others about what specific pieces of advice you might offer. I’ll ask you to share out from your table a single idea for which there was reasonable consensus.

5. A new mathematics district leader, with responsibility for mathematics professional development, comes to you for advice: What should I do to support a solid and effective professional development program across my district?

6. Why to engage in professional development? What are the goals? What to do in professional development? What are the strategies? Who will lead professional development? What is the delivery? What are the conditions for professional development? What are the resources?

7. Professional development design and implementation is complex, with many decisions to make. Decisions are interrelated. Our eyes are always bigger than our stomachs – we always want to do more in professional development than we are able to do. What do we know?

8. What do we want to know? 1. Is professional development important? 2. What are effective professional development strategies? 3. For which teachers are those strategies effective, and under what conditions?

9. Professional Development Teacher Knowledge and Skills Teaching Practice Student Outcomes Simplified Logic Model for Professional Development

10. Professional Development Teacher Content Knowledge Teaching Practice Student Outcomes Teacher content knowledge matters… For teaching practice: Selecting content to emphasize Selecting instructional strategies and sequence Selecting assessment tasks Implementing curriculum materials

11. Professional Development Teacher Content Knowledge Teaching Practice Student Outcomes Teacher content knowledge matters… For student learning: Students of teachers with stronger content knowledge have higher achievement scores than other students, in particular on measures of conceptual understanding.

12. So if TCK matters… What do we know about deepening teacher content knowledge?

13. Facets of Teacher Content Knowledge Disciplinary content knowledge Pedagogical content knowledge Ways of knowing content

14. What facets of content to choose in content-focused PD Most people seem to agree that all of these facets are important for teaching With unlimited time and resources, you would likely address them all But we don’t have unlimited time and resources, so choices have to be made

15. Plan a PD program Middle grades mathematics teachers Focus on algebraic thinking 30 hour PD workshop in the summer There will also be an academic-year component, but we’re not concerned with that right now.

16. What % of workshop time would you devote to each of the following, and in what order? Having teachers: 1.Analyze the algebraic content addressed in the student instructional materials 2.Analyze student work on algebraic tasks to understand their thinking 3.Work on algebraic tasks from the student instructional materials 4.Work on challenging algebra problems (beyond what the students are expected to know)

17. Sequencing PD Goals One line of reasoning: Teachers can’t learn to teach what they don’t know. Therefore, it is important to start with mathematics content, and only after teachers themselves have a sufficiently deep understanding of the content, move to considering classroom application.

18. Sequencing PD Goals Another line of reasoning: Teachers are by their very nature practitioners. Starting with classroom applications, e.g., trying to analyze student work, provides a purpose and context for engaging the teachers in learning mathematics content.

19. Sequencing PD Goals Available research does not indicate which approach is “better” under a particular set of conditions.

20. MSP KMD Review of Research Identified more than 1000 “studies” on PD to deepen teacher mathematics content-related knowledge

21. MSP KMD Review of Research However, approximately 90% of the studies were screened out because: –They were advocacy or opinion pieces, not research, and/or –They were studies of pre-service teachers only, and/or –They did not include a measure (quantitative or qualitative) of teacher content knowledge.

22. MSP KMD Review of Research Applied standards of evidence to 28 studies of mathematics PD, those that were not simply opinion or advocacy pieces and actually measured teacher content knowledge

23. What research tells us Available research points to some elements of effective PD, but provides very little guidance about how to design and implement PD for particular purposes in particular kinds of situations.

24. Example 1 Opportunities to learn about student mathematics curricula were positively related to… reports of classroom practices advocated in the California mathematics frameworks; and student performance. Cohen & Hill, 2000

25. Example 2 Longer duration/more contact hours; and Opportunity to engage in mathematical analysis, reasoning, and communication … were positively related to teacher learning of mathematics content knowledge for teaching. Hill & Ball, 2004

26. Example 3 Focus on a specific practice/set of practices Coherence with other PD Active learning opportunities Collective participation of teachers “Reform types” of PD focused on higher order instructional/assessment practices …were positively related to changes in teachers’ instruction, assessment, and/or technology practices. Desimone, Porter, Garet, Yoon, & Birman, 2002

27. An Emerging Consensus Effective PD: Focuses on content knowledge and how students learn content Involves a substantial number of hours Sustains focus over time Models effective practice, including active learning experiences Engages teachers in communities of learning Involves active participation of school leaders

28. Study in progress A randomized controlled trial is testing the consensus view with a PD program that includes… Substantial number of contact hours over a full-year duration, including summer institutes, academic year seminars and in- school coaching Focus on developing teachers’ content and pedagogical content knowledge Collective participation of teachers in a school

29. Preliminary findings After one year, the study found impact on … Teachers’ use of instructional practices to elicit student thinking But no statistically significant impact on … Teacher content knowledge, Teachers’ use of representations in instruction, Teachers’ focus on mathematics reasoning in instruction, Student achievement. Garet et al., 2010

30. To explain these results, one might consider whether The content was appropriate for these teachers The instruments were sensitive to impacts that were occurring There was sufficient time for impacts to occur

31. An alternative explanation The current understanding of PD effectiveness is incorrect or underspecified.

32. Why don’t we know more from the empirical research? In applying standards of evidence, we often found vague or incomplete documentation of programs or interventions. Consequently, we know something worked, but we don’t know a lot about what “it” was.

33. Why don’t we know more? Studies tended to be more like program evaluations rather than research on particular strategies. Consequently, we know the overall experience worked, but we don’t know how much particular interventions contributed to the gains.

34. Why don’t we know more? We often found serious limitations with study research designs, including: –Selection bias in samples and contexts –Lack of comparison groups or criteria –Idiosyncratic instrumentation, without evidence of validity/reliability/credibility

35. Why don’t we know more? There are too few studies of any one phenomenon to be able to have confidence in the robustness of the findings in any case. High quality research is expensive, which may explain why so many in-depth studies in the literature involved fewer than 5 teachers.

36. Why don’t we know more? There is a tension between design for change and design for learning. From a system change perspective, if you plan on having two cohorts, it makes sense to start with the schools that are “ready.” But doing so makes research on program effectiveness problematic; it will not be possible to disentangle the effects of differences in readiness from the impact of the interventions.

37. Why don’t we know more? Even when individual studies are well- designed and well-implemented, it is difficult to look across them and figure out the extent to which the findings might generalize.

38. There is a difference between… What has worked What will work in my context

39. Filling the gap “Sensible propositions” can provide guidance and serve as hypotheses for research MSP-KMD project developed a methodology for collecting and vetting practice-based insights.

40. Comparing Empirical Research and Practice-based Insights We were surprised at how little guidance the available research provides and how much guidance expert practice provides, although without the backing that empirical research would provide. **

41. Comparing Empirical Research and Practice-based Insights Empirical findings tend to be larger grain size; practice-based insights tend to be more contextualized and nuanced.

42. Research + Practice = Guidance in designing and implementing professional development Focus on teacher leaders as one part of a professional development program

43. Teacher leaders matter in terms of impacting teachers’ classroom practice, with limited evidence that TL work impacts students. Teacher leader work takes different forms (e.g., leading pd, modeling lesson, planning) What do we know from research on teacher leaders?

44. Little guidance in terms of choosing particular practice, how to implement it, or under what conditions. What do we know from research on teacher leaders?

45. Articulate the purpose and nature of TL practice(s) –Each TL shouldn’t figure this out individually –Not the time to “let 1000 flowers bloom” because impossible to support –Impact is diluted when focus is dispersed –Trade-off between clear, shared articulation and local response What do we know from practice- based insights on teacher leaders?

46. Make decisions based on available resources and adjust program accordingly –TL practice should fit available time for working with teachers –TL practice should be calibrated to available support from school, district, beyond –Trade-off between “playing with the cards you’re dealt” and “pushing the envelope” What do we know from practice- based insights on teacher leaders?

47. Develop alignment among TL selection, TL preparation and TL practice What do we know from practice- based insights on teacher leaders?

48. 1.You want teacher leaders to be leading school-based professional development in department and faculty meetings and district-wide professional development in after-school and summer workshops. Given this, who do you select as TLs? Task 1

49. 2. Your selection process yields TLs who are enthusiastic, have credibility with their colleagues, have limited prior experience leading professional development, and display a wide range of content knowledge. Given this, how do you think about preparing TLs for the practice of leading professional development? Task 1

50. 3. There is limited time and even fewer resources to devote to preparing these TLs to lead professional development. Given this, how do you think about the intended TL practice? Task 1

51. In your experience with teacher leaders as part of a professional development program, what was the alignment between TL practice, TL selection, and TL preparation? Where did the alignment break apart (and wishful thinking take its place)? Construct your own triangle and identify how/whether each is aligned. Task 2

52. TL practice TL preparation TL selection

53. Develop alignment among TL selection, preparation and practice –Intended TL practice is an important first consideration –Selecting fewer, highly-qualified TLs may be preferable to selecting larger numbers of less-qualified candidates –Be clear about content knowledge needed for TL practice and whether it is a realistic selection criteria or preparation expectation –Trade-off between TL practice, selection and preparation What do we know from practice- based insights on teacher leaders?

54. KMD Resources Knowledge reviews on TCK, TL and STEM Measures of teacher content knowledge Sustainability cases on TCK and TL www.mspkmd.net

55. Sample Knowledge Reviews Teacher knowledge: Engaging with challenging mathematics/science content Selecting teacher leaders Involving STEM disciplinary faculty in deepening teacher/teacher leader content knowledge

56. How you can contribute Use the best evidence available; Document your treatment, including types of teacher content knowledge addressed, in what ways, in what sequence, etc.; Describe your participants and your context; Monitor how well it worked and how you know; Share results/lessons learned with the field.

57. REFERENCES Cohen, D. K. & Hill, H. C. (2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102(2), 294–343. Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis, 24(2), 81–112. Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Walters, K., Song, M., Brown, S., Hurlburt, S., Zhu, P., Sepanik, S., Doolittle, F., & Warner, E. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE 2010-4009). Alexandria, VA: U.S. Department of Education. Hill, H. C. & Ball, D. L. (2004). Learning mathematics for teaching: Results from California's mathematics professional development Institutes. Journal for Research in Mathematics Education, 35(5), 330–351.