1. Observing Classrooms Using Technology Heather C. Hill Harvard Graduate School of Education

2. Why use technology to capture classroom instruction? Define “technology”… 1970s, Cooley and Leinhardt videotaped classrooms for the Instructional Dimensions Study Since then, many uses of recorded classroom observations to enrich / enlighten Deborah Loewenberg Ball / Magdalene Lampert’s self- study of mathematics teaching TIMSS video study (Stigler and Hiebert) Video clubs (Sherin and colleagues); direct feedback on teaching (CLASS My Teaching Partner)

3. Advantages of technology (2010) To understand instruction: International comparisons “Local” comparisons Re-watching enables fine-grained analysis of what occurs in math classrooms

4. Unpacking instruction Graciano 8 th grade Students are supposed to find and interpret the slope of the line Going over homework Problem: Phone company charges $0.25 per minute. Graphthe cost per minute, find the slope of the line Question: What do you “see” upon watching the tape once? What would you want to know if you saw it again?

5. Graciano

6. What did you see? Do you want to see it again?

7. Advantages of technology (2010) To understand: International comparisons “Local” comparisons Re-watching enables fine-grained analysis of what occurs in math classrooms To improve: Take teachers “out of the moment” to reflect on their own teaching Provide direct feedback to teachers for improvement Major advantage: “Objective”

8. Advantages of technology: 2011? To monitor, evaluate and reward? Existing system of teacher evaluation is very, very broken Most evaluations not done by content experts Most teachers get positive evaluations (97% in Boston and LA) Anecdotal evidence that many principals do not even observe teachers, or observe for very long Yet we know from our studies and others that mathematics instruction in the U.S. is of variable quality Some very, very poor

9. Advantages of technology: 2011? Proposal: Use technology to monitor performance of beginning teachers of mathematics Use information to… Counsel some out of teaching, or teaching mathematics Funnel teachers to professional development specifically targeted to weaknesses Identify outstanding teachers for leadership/specialist positions

10. Challenges to this vision Need a cheap but high-quality way to capture teaching Graciano lesson: $800 To scale this proposal, need captures at about $150/lesson Need good-quality audio, also board capture Need capacity to handle/process video As it comes in As it is coded by objective raters (capacity) Need “wrap-around” data (teacher interviews, artifacts such as textbook materials)

11. Challenges to this vision Need studies that show what video cannot capture What does live observation afford that videotape does not? Are we penalizing some teachers unfairly? Instruments (!) MQI, CLASS, PLATO, need others…. Generalizability studies to determine teacher reliability, number of lessons required, best rater configurations… Implementation How do you use this with real teachers, real principals, real HR departments?

12. The Capacity to Support… (2010) Lots of research projects experimenting with “homemade” solutions (Flips, laptops, iPhones) Companies like thereNow, Teachscape have made inroads into cheap, remote

13. Trying this at home…issues Operator vs. no operator camera Former gives you much more flexibility, but is more expensive Level of video detail required for the type of information you wish to gather Do you need penstroke board capture? Do you need shots of student reactions? Just the teacher? One vs. two camera solutions The Make or Break: Audio

14. Conclusions Is this technology’s time? Certainly for small- & moderate-scale research projects, professional development For evaluation? Hard to tell History of reforms being “eaten by the system” But strong need for better identification and accountability