Effective Continuing Teacher Education Heather C. Hill University of Michigan School of Education MSP Professional Development Institute

a.k.a. Tough Love For State Officials and Professional Developers What works, what doesn’t work –Formats –Content Things people think work but for which there is no evidence Unsolicited advice

What Works

Background on “What Works” Analysis: Studies of CTE focused on enhancing teacher knowledge and practice Links to student learning Method: –Examined PD “standards” set by AFT, NSDC, NCTM, scholars

Professional Development Standards Active learning Examples from classroom work Collaborative Model effective/relevant strategies Reflect, feedback, practice Content focus Focused on student learning Coherent

Background on “What Works” Analysis: Studies of CTE focused on enhancing teacher knowledge and practice Links to student learning Method: –Examined “standards” set by AFT, NSDC, NCTM, scholars –Examine published/reviewed studies that link professional development to student achievement –Allow in studies where measure of teacher learning is linked to student achievement –Descriptive studies of what’s “in” the professional development widely available to teachers

The Good News: Continuing Teacher Education Can Work Characteristics of graduate education have been linked to student achievement Number of studies over past 30 years demonstrate professional development can –Improve quality of teaching –Improve student learning (e.g., Goldhaber & Brewer, 1998; Cohen & Hill, 2001; Garet et al., 2001; Good, Grouws, & Ebmeier, 1983; Carpenter et al., 1989; Cobb et al., 1992; Saxe, Gearhardt & Nasir, 2001; many reviewed in Kennedy, 2001)

What Works? Formats What works:

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching What doesn’t work:

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching What doesn’t work: –Graduate degrees in any other subject, including education What doesn’t work: –Graduate degrees in any other subject, including education

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school –Not aligned with standards and assessments What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school –Not aligned with standards and assessments

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school –Not aligned with standards and assessments –Time – longer What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school –Not aligned with standards and assessments –Time – longer

What Works? Formats What works: –Graduate degrees in subject matter h.s. teacher teaches –Aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school –Not aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching (and yes, one- shot workshops) What doesn’t work: –Graduate degrees in any other subject, including education –??? Middle and elementary school –Not aligned with standards and assessments –Time – longer –Format: Summer institutes, extended-year follow-up, coaching (and yes, one- shot workshops)

What Works? Content What works –Focus on the content –Focus on student learning –Active learning –Examples from classroom practice What doesn’t work –Focus on the content –Focus on student learning –Active learning –Examples from classroom practice What doesn’t work –Focus on the content –Focus on student learning –Active learning –Examples from classroom practice Success Depends Upon Enactment

“Motion Math” K-6 workshop Meets several “standards” for best practice in PD –Teachers explored activities they could later do with students (practice-focused) –Worked collaboratively –Focus on mathematics and student learning

Motion Math: Space Alien Math “Motion Math believes in a hands-on approach to operations. Our leader recently became interested in proportionality, and discovered a new thing using it: every division problem has four numbers. Every division question is the same as saying “how much in one?” It becomes a set of proportional fractions when you think about it this way. One can cover any portion of the square and it becomes alternately a division or multiplication problem.” We didn’t slow down to explain which would be which. Several teachers seemed confused. Later, she clarified: “Division is really a ratio. All we are doing is fair sharing.” 1

Motion Math: Space Alien Math Eighteen aliens are on the moon. They use two spaceships to come back to earth. How many will be in each spaceship?

Motion Math: Meeting the “Focus on Mathematics” Standard? DOES connect division and proportionality, potentially rich However: Flawed presentation, mathematically –Superficial, disconnected treatment: Only a brief introduction – room for confusion ( “ Every division question is the same as saying ‘how much in one?’”) Not clear how this relates to more standard interpretations of division (“How many groups of 2 in 18?”) Not clear how the aliens are related to the symbolic division; where is the “1”? Confusing follow-up: “Division is really a ratio. All we are doing is fair sharing.” Flawed problem!!!! 1

Example II: Writing Mathematics Focused on helping teachers help students build mathematical explanations for state test Met many standards, in name –Collaborative –Teachers did mathematics problems –Used artifacts of student work –Follow-up to initial PD session

Writing Mathematics: Meeting the “Focus on Mathematics” Standard “Density” of mathematical problem-solving low –In six hours, teachers solve two problems, inspect state test, write two “open-ended” problems No elaboration on the mathematics Mathematical “explanation” really procedural description: –“first I subtracted 45 from 75, then I added 10…”

Writing Mathematics: Meeting the “Student Learning” Standard Graded student work on “open-ended” problems –Correctly identified student problem with regrouping –But remedy is purely procedural: [The leader] had a little colloquy with a teacher about learning place value and the idea of trading. He said when he goes into a classroom, he gets kids to make a list of “sad” (7,17,5,6) and “happy” (13,2,22,10) numbers in reference to the number 23. The teacher at my table immediately recognized the sad numbers as those you have to regroup, happy numbers as the ones which you didn’t. Leader said that every student should have a sense for that, if you’re working with 5, for instance, you have to regroup numbers that end in 4,3,2,1 and 0, that you don’t have to with 6,7,8,9.

Example III: Student Mathematics Did NOT meet many standards –40 minutes long –Large group: limited collaboration between other teachers DID meet key criteria: high-quality mathematical and student-related content

Student Mathematics (Teachers discuss how to solve one problem, , using multiple methods). Then the provider showed another video, this one with a student solving an addition problem in which neither number was close to 100. The student narrated the problem (24+37) like this: “Twenty- four plus thirty is fifty-four. Fifty four plus six is seventy, plus one is seventy one.” The provider asked teachers to talk to others around them, explaining what this student had to know to do this problem in this way. After members of the audience did so, the provider explained this student’s thought process, saying that he “holds that seven out,” then breaks it into two pieces to add. She challenged teachers to determine whether this student followed the rules and whether the method this student used worked only in a few cases or generally for all two-digit addition problems.

Take-Home It’s enactment that matters –Is there “academic press” – in terms of covering challenging mathematical content? –Is the professional development “mathematically dense”? –Is there rich, meaningful mathematics? –Are substantial connections made to how students will use/learn material?

Things that People Think Work But For Which There’s No Rigorous Evidence

What Do People Think Works But in Reality There’s No Empirical Evidence for It? Collaboration between teachers Modeling relevant pedagogy Opportunities for reflection, feedback, practice Teacher choice Involving teachers in planning and design (e.g., Lesson Study) Teaching elementary and middle-school teachers college-level mathematics

Unsolicited Advice

Unsolicited Advice Learn, learn, learn –Morally obligated to find out whether your professional development works –With your help, we might know something in 10 years –Rigorous evaluations needed Not only in sense of good design, but in careful implementation (i.e., no lost teacher IDs), careful instrumentation (i.e., no homegrown measures with ?? reliability and validity), and thoughtful links to student achievement

Unsolicited Advice Rigorous evaluations –Mechanism for making causal inference: What would have happened to these teachers’ teaching and learning in the absence of this program? –Measurement instruments that are objective and credible –Longitudinal measurement –No logistics problems to sink ships –Strong analyses, with plausible story about student achievement change (value-added estimates; controls for student SES, ELL, HLM)

Bargain-Basement Evaluations (Or, why I will be hunted by the IES police) Use teacher as unit of analysis N=50 (25 treatment, 25 control) Pre/post test teachers as condition of participation in the learning experience Pre/post test students in those teachers’ classes as condition of participation –Use established test close to your own curriculum –Administer in fall/spring at time point that makes sense Use HLM as analytic method (kids nested within teachers) –Treatment variable is at teacher level –Now available in “mixed models” in SPSS

Unsolicited Advice Don’t offer PD around topics teachers know well –Example from middle school algebra

What Teachers Know Well (Middle Algebra)

Unsolicited Advice Don’t offer PD around topics teachers know well Tailor PD to teacher needs –Middle school sample: high-school- credentialled-teachers may not need content coursework

Middle school teacher MKT, predicted by credentials and experiences Former high school teacher.24** High school credential.66**** Mathematics credential.39**** Years experience.008* Mathematics classes.17*** Math methods classes.08* Adjusted r-squared.32 Regression MKT as dependent “Omitted” category is middle school credential only Strong evidence that high school credential/ experience associated with high MKT

Conclusion: Enactment Matters Format Content Quality Matters –Difficult to determine quality without observing it –“Scaffolds” in terms of curriculum for PD (e.g., Videocase, DMI, CGI, LessonLab) Quality matters in your evaluation, too