1. What Students Can Tell Us about the Quality of Teaching: Evidence from MET and Implications for Teacher Preparation Ronald F. Ferguson, PhD Harvard University and the Tripod Project for School Improvement Spring Regional CAEP Conference, The Mirage, Las Vegas, Nevada, March 16, 2012

2. Student perspectives can help guide: teachers as they work to improve; leaders as they set professional development priorities for teachers; coaches and mentors in supporting teachers; higher education institutions as they develop teacher training programs.

3. The Seven C’s What Teachers Do (What Students Experience) 1.Caring about students (Encouragement and Support) 2.Captivating students (Learning seems Interesting and Relevant 3.Conferring with students (Students Sense their Ideas are Respected) 4.Controlling behavior (Culture of Cooperation and Peer Support) 5.Clarifying lessons (Success Seems Feasible) 6.Challenging students (Press for Effort, Perseverance and Rigor) 7.Consolidating knowledge (Ideas get Connected and Integrated)

4. Student Engagement Targets (Both Individual and Collective) 1.Trust Safety (vs. Mistrust) 2.Cooperation (vs. Resistance) 3.Ambitiousness (vs. Ambivalence) 4.Diligence (vs. Discouragement or Disengagement) 5.Satisfaction and Efficacy (vs. Disappointment and Failure) The Seven C’s What Teachers Do (What Students Experience) 1.Caring about students (Encouragement and Support) 2.Captivating students (Learning seems Interesting and Relevant 3.Conferring with students (Students Sense their Ideas are Respected) 4.Controlling behavior (Culture of Cooperation and Peer Support) 5.Clarifying lessons (Success Seems Feasible) 6.Challenging students (Press for Effort, Perseverance and Rigor) 7.Consolidating knowledge (Ideas get Connected and Integrated) Teacher Professional Learning (PLCs) Content Knowledge Pedagogic Skill Relationship-Building Skills The Tripod STUDENT ACHIEVEMENT OUTCOMES

5. Key Patterns to Note  Teaching in some classrooms is much more effective than in others at fostering Seven C’s learning conditions.  There tends to be much more Seven C’s variation within schools—from one classroom to another—than between them, from one school to another.  Generally, students are more engaged (happier, more hard working and more satisfied with their achievements) in classrooms that rate higher on the Seven C’s.

6. 6 A composite 7C inter-classroom index can be formed by combining all seven of the indices— i.e., Cares, Controls, Clarifies, Challenges, Captivates, Confers and Consolidates—into one composite number. Then, the composite index can be used to rank classrooms into instructional quality deciles, quintiles or quartiles.

7. GRADE: K 1 ST Gr. 2 nd Gr. 3 rd Gr. 4 th Gr. 5 th Gr.

9. Developing healthy dispositions and experiencing a high quality of life in school: classrooms that rate higher on the Seven C’s also rate higher on:  Happiness in class  Good behavior in class  Healthy responses to social pressures (e.g., less holding back, hiding effort, help avoidance)  Academic engagement/effort  Satisfaction with performance and progress

10. Some findings from the Gates Foundation Measures of Effective Teaching Project

11. Predicted differences in months of learning for classrooms at the 25 th versus 75 th percentiles of the composite Seven C’s distribution for the MET sample. (Estimated using surveys from one class and gains from another class, taught by the same teacher.) Using student survey responses in one section to predict learning gains in another. Predicted difference per school year* On state math tests 4.8 months On the Balanced Assessment in Math 3.7 months On state English Language Arts (ELA) tests 2.3 months On the Stanford 9 Open Ended ELA 2.9 months *Based on Table 9, p. 26: Bill and Melinda Gates Foundation, “Learning about Teaching: Initial findings from the Measures of Effective Teaching Project.” December 2010.

12. The Point from MET: Student perceptions of classroom practice can help in predicting learning outcomes and should be taken seriously by policy makers and educators. But how can we use them?

13. Using the Seven C’s to develop profiles and identify priorities at multiple levels of drill down.

14. There are multiple embedded levels of possible “drill down” on any given issue, e.g., to address questions about: 1.The district 2.One or more schools or grade levels within the district 3.One or more grade levels within a school 4.One or more classrooms within a grade 5.One or more students within a classroom 6.One or more survey items from among an individual student’s responses

15. Use Case 1: Individual Teacher Analysis After being introduced to school-level results, teachers spend time reviewing their own results individually. Teachers identify and reflect on strengths and weaknesses and set goals for personal growth. Teachers include targets for improvement in professional growth plans. Teachers acquire knowledge, develop new skills, and implement new strategies.

16. Teacher A Teacher A Teacher B Teacher B Tripod Student Feedback Surveys and Assessments Tripod Student Feedback Surveys and Assessments Example: Actual Feedback for Two Very Different Classroom Teachers

17. Teacher A (Low) Teacher A (Low) Teacher B (High) Teacher B (High) Value student insight! Tripod Student Feedback Surveys and Assessments Tripod Student Feedback Surveys and Assessments

18. Value student insight! Teacher A Teacher A Teacher B Teacher B

19. Common Language and Frameworks for Evaluation and School Improvement

20. Teacher A Teacher A The benchmark in this example is the bottom classroom in the top third (i.e., the 67 th percentile) among all classrooms at this particular school.

21. Teacher A Teacher A

22. Teacher A Teacher A

23. Teacher B Teacher B

24. Teacher B Teacher B

25. Teacher B Teacher B

26. Teacher A Teacher A

27. Teacher A Teacher A

28. Teacher A Teacher A

29. Teacher B Teacher B

30. Teacher B Teacher B

31. Teacher B Teacher B

32. Teacher A Teacher A

33. Teacher A Teacher A

34. Teacher A Teacher A

35. Teacher A Teacher A

36. Teacher A Teacher A

37. Teacher B Teacher B

38. Teacher B Teacher B

39. Teacher B Teacher B

40. Teacher B Teacher B

41. School Norms – Based on all responses from the whole school. The benchmarks are averages from the whole district.

42. School Norms – Based on all responses from the whole school. The benchmarks are averages from the whole district.

43. Teacher A Teacher A Teacher B Teacher B Viewed side by side

44. Use Case 2: District and School Analysis District level leaders review results of the 7Cs analysis across all schools in the district School level leaders review results of the 7Cs analysis across classrooms in their schools District and school level leaders use these data to focus professional development strategies and to monitor the impact of school improvement efforts

45. CareClarifyCaptivateConsolidate ControlChallengeConferMean 13529475840354842 23626486042344842 33127545344375042 44031495842364743 53839485240434443 63932495342434744 73840485540435045 83342485749404845 93940475642434945 103642515748364745 113743505546444746 12374549544546 133934516247415147 143749505447444647 154339495644465047 163843525745435147 174332506245 5447 184529536446445348 194145525845434948 203930556550415648 214431526451415548 224332576550405449 234336546050495349 2440 596549425350 254148545947485350 264432576552455750 274632566652445650 2844 546248455651 294633556854455651 305332536651475551 314446536150495651 324534576653465851 334744556149485551 344835526358485651 354632567151486152 364339606656455852 374051586655455352 384736617055465853 395137586756506154 405138586863496155 415433577156546356 425042657353486156 434948607544546356 445142647256476256 455438627259516357 465730637660516958 475940657561567061 486341687970597164 >= 75% 60 to 74% 50-59% 40-49% 25-39% < 25% District: Percent favorable, by school, for each Seven C’s category. (Each line is one of 48 secondary schools in one urban school district.)

46. CareClarifyCaptivateConsolidate ControlChallengeConferMean 13529475840354842 23626486042344842 33127545344375042 44031495842364743 53839485240434443 63932495342434744 73840485540435045 83342485749404845 93940475642434945 103642515748364745 113743505546444746 12374549544546 133934516247415147 143749505447444647 154339495644465047 163843525745435147 174332506245 5447 184529536446445348 194145525845434948 203930556550415648 214431526451415548 224332576550405449 234336546050495349 2440 596549425350 254148545947485350 264432576552455750 274632566652445650 2844 546248455651 294633556854455651 305332536651475551 314446536150495651 324534576653465851 334744556149485551 344835526358485651 354632567151486152 364339606656455852 374051586655455352 384736617055465853 395137586756506154 405138586863496155 415433577156546356 425042657353486156 434948607544546356 445142647256476256 455438627259516357 465730637660516958 475940657561567061 486341687970597164 >= 75% 60 to 74% 50-59% 40-49% 25-39% < 25% Line 1 is for the lowest rated high school: See the next slide.

47. Each row shows the percent favorable responses for a classroom in the lowest rated high school. (Classes with at least 10 respondents) CareClarifyCaptivateConsolidate ControlChallengeConferMean 13231823 1720 1315282913212320 828222718261320 64022311983122 415212826333623 2523293928 3529 2419355016363831 2124354537263532 3326284235373934 3225374939454238 35 394244413639 3341384354373340 38324045 473641 32484740384041 4521425330445241 2550534732514043 3342575144403143 3329616443414345 4144596732465349 3829586747506250 4757 6033554851 45425865505652 5542586364496156 5761716848615961 6455657364 5763 6776687064595666 7173828364666772 >= 75% 60 to 74% 50-59% 40-49% 25-39% < 25%

48. CareClarifyCaptivateConsolidate ControlChallengeConferMean 13529475840354842 23626486042344842 33127545344375042 44031495842364743 53839485240434443 63932495342434744 73840485540435045 83342485749404845 93940475642434945 103642515748364745 113743505546444746 12374549544546 133934516247415147 143749505447444647 154339495644465047 163843525745435147 174332506245 5447 184529536446445348 194145525845434948 203930556550415648 214431526451415548 224332576550405449 234336546050495349 2440 596549425350 254148545947485350 264432576552455750 274632566652445650 2844 546248455651 294633556854455651 305332536651475551 314446536150495651 324534576653465851 334744556149485551 344835526358485651 354632567151486152 364339606656455852 374051586655455352 384736617055465853 395137586756506154 405138586863496155 415433577156546356 425042657353486156 434948607544546356 445142647256476256 455438627259516357 465730637660516958 475940657561567061 486341687970597164 >= 75% 60 to 74% 50-59% 40-49% 25-39% < 25% Line 48 shows the highest rated high school: See the next slide.

49. >= 75% 60 to 74% 50-59% 40-49% 25-39% < 25% Each row shows the percent favorable responses for a classroom in the highest rated high school. (Classes with at least 10 respondents) CareClarifyCaptivateConsolidate ControlChallengeConferMean 1830263223262726 1524223525462728 2225263537324131 3234235643314237 2934326031324237 3137345529424639 403240474039 40 3228404534535942 3223437034465844 3338475159525448 3830566356415649 3858597824405851 4665625160424853 63455868405455 7330767343655559 6460626661605561 3968667159537562 3960727871637766 4254658384647767 5674797077588571 6782738075686673 6776757765708373 716578 6980 75 7780788583708079

50. Revisiting Key Points  Teaching in some classrooms is much more effective than in others at fostering Seven C’s learning conditions.  Student perceptions of classroom practice on Seven C’s dimensions can help in predicting learning outcomes and should be taken seriously by policy makers and educators.  There tends to be much more Seven C’s variation within schools—from one classroom to another—than between them, from one school to another.  Generally, students are happier, more hard working and more satisfied with their achievements in classrooms that rate higher on the Seven C’s.  There are many ways that student perspectives can be used to provide useful information about patterns of teaching effectiveness, ideally in combination with other measures: we need multiple measures multiple times.

51. Ronald_Ferguson@Harvard.edu Assistant: Andrew Volkert 617-496-9154 For more information about Tripod Surveys, Contact: Rob Ramsdell Email: rob.ramsdell@camb-ed-us.comrob.ramsdell@camb-ed-us.com Web: www.tripodproject.org Tel: 781-915-0005