The Genie is Out of the Bottle Implications of classroom level value-added analysis

Background Who is Battelle for Kids? Who is Mike Thomas Non-profit educational improvement organization since 2002 Work built around using VA and other data for school improvement Who is Mike Thomas BFK since 2003 OSU from 1992-2002 Teacher from 1977-1992 Focus on leadership, school improvement and adult development

The Genie is out of the bottle: Implications of classroom-level value-added analysis “Conventional Wisdom” Highly Effective Teacher Research What we did What we found What we think it means Connecting Conventional Wisdom to the Research

Common Educational Wisdom? “Based on the current evaluation process, everyone is effective.” “The best teachers know who they are; I don’t need to tell them they are good.” “Teachers are born not made, talent trumps craft.” “There’s not a thing wrong with my teaching; it’s those damn kids.” “Don’t smile until Christmas”. “It’s my job to teach and your job to learn; I’m doing my job, you’re not doing your yours.”

Highly Effective Teacher Research Selection of Participants What we did—Research Design What we found—Research Findings What it means—Theorizing about the findings

HET Selection Process Teacher-Level Value-Added Reports were used for the selection of teachers Teacher had to produce value-added scores of at least 3 S.E.s above expected in math and nearly 3 S.E.s above expected in reading. Teachers had to show positive gains with all student subgroups. Report from previous year, if available, had to be solid.

What we did: Appreciative Inquiry Appreciative Inquiry methods were used to induce reflection, to trigger conversation and to frame the inquiry perspective we were taking. Teachers interviewed each other to get at the factors that account for large student gains. Teachers shared what they learned in small groups to find commonalities in their experiences. Each teacher group produced a list of critical factors from which common themes were identified. Teachers each produced 2 hours of writing about how two themes played out in their classrooms.

Large-Scale Study Findings Teachers, regardless of the subject area or grade level they taught, cited these four themes as reasons for their effectiveness: Instruction that works for every student (Productivity/Competitiveness) Classroom environment (Structures that support learning/Control) Student-centered focus (Relationships/Collaboration) Professional growth and flexibility (Instructional improvement/Creativity)

Theorizing about the findings A list of themes, practices, factors or behaviors is, by itself, only minimally useful. Lists are not very interesting; they tend to oversimplify things that are inherently complicated. Lists lend themselves to shallow one-size-fits- all “solutions.” Lists leave out as much they illuminate. Lists tend to ignore the relationships that exist between elements on the list.

Competing Values Framework (Quinn) Flexibility & Openness Child-Centered Focus Relationships Support Collaboration Responsive teaching Student ownership of learning Relevance Professional Growth and Leadership Passion Adaptability Flexibility Creativity Parental communication Instructional improvement Human Relations Model: Human commitment (Collaboration) Cohesion Morale Value of human resources training Open Systems Model: Expansion, adaptation (Creativity) Adaptability Readiness Resource Acquisition External Support Internal Focus External Focus Internal Process Model: Consolidation, continuity (Control) Stability Control Information mgmt. Communication Rational Goal Model: Maximization of output (Competitiveness) Productivity Efficiency Planning Goal Setting Classroom Environment Rules Structures Control Routines Classroom management Instruction that works for every student High Expectations Productivity Persistence High quality student work Rigor Differentiation Competitiveness Structure & Control

Positive and Negative Zones

Other Findings Most HET’s reported that this was the first time they were recognized as being good teachers. Most HET’s tell us that they had to hide the fact that they were identified as highly effective. Most HET’s tell us that they have never seen their grade-level value-added information or had a focused data-based conversation about improvement. One size-fits-all PD is a waste of time.

Reconnecting with the Common Wisdom “Based on the current evaluation process, everyone is effective.” “The best teachers know who they are; I don’t need to tell them they are good.” “Teachers are born not made, talent trumps craft.” “There’s not a thing wrong with my teaching; it’s those damn kids.” “Don’t smile until Christmas.” “It’s my job to teach and your job to learn; I’m doing my job, you’re not doing yours.”

Next Steps Test out the framework Develop a survey to test out these ideas. Does it fit HETs? How do less effective teachers map out? Do deeper dives with HETs to get at the high leverage behaviors that connect competing values

www.BattelleforKids.org