1. Improving Impacts of Classrooms: Professional Development and Classroom Observation Robert C. Pianta, Ph.D. Dean, Curry School of Education Director, Center for Advanced Study of Teaching and Learning Discover. Create. Change.

2. Questions and challenges for policy, research, and training in education  What experiences for social and cognitive development are offered to students in classroom settings?  Do interactions with teachers and experiences in classrooms matter for students?  Can observation leverage efforts to improve the richness, quality, and effectiveness of experiences in classrooms?  Can we use observation of teacher-student interactions to improve student learning?

3. Student-teacher interactions and schools Interactions with adults form “infrastructure” for school success:  Self-regulation, emotional self-control  Task orientation, persistence, motivation  Cognitive processes, language Instruction is, in part, a social process:  Interactions with teachers are a “medium” for engagement  Good instruction is embedded in interactions  Interactions operate across content/curriculum

4. Can observation improve teaching?  HR policy  Program / professional development  Scale and Standardization Scalable, reliable, valid, coarse distinctions – selection/retention Reliable, valid, articulated in experience of practitioners – tied to actual mechanisms of teacher impacts Consistent scoring across exceptional variation Technical properties support decisions and transparency Need for common lens/language Need for technical support and training infrastructure      

5. National-level observation studies  National-level studies  National Center for Early Development and Learning (NCEDL)  NICHD Study of Early Child Care  Up to 1,000 settings observed at preschool, K, 1, 3, 5 – more than 4,000 classrooms  Large set of systematic standardized classroom observation in U.S. schools  All teachers credentialed/certified

6. Observational/training procedures Detailed manual describing codes/procedures Master-coded video segments Central training of observers live or web Post-training practice and feedback Videotaped certification test Drift tests Designed for typical day/classroom setup Global ratings and time-sampled codes

7. Describing opportunities to learn: Counting behaviors, activities, practices  Vast majority of interaction/activity is whole group or individual seatwork  Few, if any, social or instructional interactions between teacher and individual child  Mostly literacy (50%-90% of instruction)  Exceptional variation within and across grades and classrooms  Consistent patterns from pre-k to 5 th grade  No association with child outcomes

8. How do students spend time?  High-levels (30%) of “business/routine” activity  Pk-5: managing materials, routines  High levels of “basic skills” focus  7:1 in pk-1; 14:1 in 3-5  Ratio of listening, sitting, watching: Doing  10:1

9. What is the CLASS?  CLASS is a tool for observing and assessing the quality of interactions between teachers and students  It measures the emotional, organizational, and instructional supports provided by teachers that have contribute to children’s social, developmental, and academic achievement.  CLASS is used to assess interactions among teachers and students for a variety of purposes:  Teacher Professional Development  Monitoring and Evaluation of Teacher Performance/ Effectiveness  Research

10. CLASS versions and development  Infant (CLASS-I) – presently in validation studies  Toddler (CLASS-T) – fully supported  Pre-Kindergarten (CLASS-Pre-K) – deployed at scale  Elementary (CLASS-K-3) – fully supported  Upper-Elementary (CLASS-4 to 6) – in validation  Secondary (CLASS-S) – in validation  Reliability/Validation/Training/Scale-Up

11. DOMAINS DIMENSIONS INDICATORS Instructional Support Emotional Support Classroom Organization Relationships, Affect, Respect, Communication Punitive, Sarcasm/ disrespect, Negativity Aware, Responsive, Address problem, Comfort Flexibility, Autonomy, Student expression Clear expectation, Proactive, Redirection Maximize time, Efficient routines and transitions Variety, Promote student interest, Clarity, Engaging Analysis/reasoning, Creativity, Integration Feedback loops, Encourage responses, Expand performance Conversation, Open- ended, Repeat/extend, Advanced language

12. Classroom ratings: CLASS PK-5  Positive climate  Negative climate  Teacher sensitivity  Regard for student perspectives  Effective behavior management  Learning formats/engagement  Productivity  Concept development  Evaluative feedback  Language modeling Emotional Support Instructional Support Organization/ Management

13. How is the CLASS Organized? Dimension Indicator Behavioral Markers Emotional Support Domain

14. Data on CLASS  CLASS has been used to observe over 8,000 classrooms across the United States.  CLASS is one of the most extensively used observational measures of teacher effectiveness for preschool through secondary classrooms.  Classrooms are generally passive settings – 25% score quite low on all; 25% fairly high. Instructional Quality generally quite low on average  Teachers with higher scores on CLASS have students who make greater academic and social progress during the school year.

16. Profiles of classroom quality: First grade Quality 31%23%29%17% 0 1 2 3 4 5 6 7 Emotional Instructional

17.  Exceptional variability within and across grades, generally passive instructional environments  Little to no association of observed behavior:  Teacher experience or training, curriculum, public/private  Teacher salary  Small associations (.10 -.20)  Class size: larger classes more structured; smaller classes more social and higher instructional quality  Family income/education related to more positive ratings  Students needing access to stable high-quality instruction do not typically receive it – 10% rate What observational studies tell us

18.  Designs that isolate effects for instructional and emotional inputs controlling for other influences  Family and demographics, prior performance, teacher/school  Consistent results, small-moderate effects (.10 -.20)  Instructional and emotional quality predict more positive achievement and social outcomes  Larger effects on more proximal outcomes (e.g., child engagement)  Stronger effects for groups of children (~.5 s.d.)  Low maternal education  Adjustment problems in K  Poor CLASS and student outcomes

19.  Where should we focus attention in policy, program development, and teacher preparation?  Predicting achievement growth during pre-k from:  Structural features (teacher ed., curriculum, etc.)  Observed interactions (ECERS, CLASS)  No association of structure with outcome, singly or in combination (e.g., NIEER index)  Instructional and Emotional Supports (CLASS predict positive changes in literacy, language, and math skills  Small effect sizes persist into kindergarten Pre-k quality and growth in child outcomes

20. Predicting student development in pre-k Changes in children’s development from beginning to end of preschool Mashburn, et al. (in press) Emotional Support Instructional Support ECERS-R Total Receptive Language  Expressive Language  Rhyming  Letter Naming  Math Skills  Social Competence  Behavior Problems 

21. Do associations persist into kindergarten? -  Yes, children in pre-k classrooms offering higher levels of Instructional Support displayed better language skills at the end of the kindergarten year.  Kindergarten Instructional Support scores made an independent contribution to gains in children’s language and math abilities.  One-point differences in observed instructional supports appear linked to shifts in child outcomes. Burchinal et al., (in press)

22. “Active range” for effects  Analysis of “thresholds” – points on distribution where impacts are evident  Emotional Support – “4.5” and above  Instructional Support – “2” and above  For IS, the active range appears linked to teachers’ increased support for cognitive skills  One-point shift appears meaningful

23. Dual-language learners  Two major studies  Does CLASS describe interactions in language-diverse settings?  Does CLASS predict outcomes similarly?  CLASS 3-factor structure and psychometric properties same across diverse classrooms  Predicts outcomes similarly for DLLs  Evidence supports use in diverse settings

24. Gains in grade 1 achievement in instructionally supportive classrooms Standardized tests of achievement adjusted LowModerateHigh 1 st Grade Instructional Support 98 99 100 101 102 103 104 105 106 107 High educ. Low educ.

25. Gains in grade 1 achievement in emotionally supportive classrooms Standardized tests of achievement adjusted LowModerate High 1 st Grade Emotional Support Kindergarten adjustment problems 98 99 100 101 102 103 104 105 106 107 No problems Multiple problems

26. CLASS-S results  Predicts state-standards test scores across all content areas (>2,000 students, >100 classrooms)  Moderate effect sizes (. 30) in all content domains  Also predicts observed student engagement  Accounts for gains in standards tests in the subsequent year  Now in studies with several thousand classrooms (MET, WTG study).

27. Support for high-quality interactions Professional development/ training Resource allocation Evaluation Curriculum Social and academic outcomes for children Improved teacher outcomes CLASS Instructional Organization Social Observational Assessment

28. Improving Caregiver-Child Interactions: MyTeachingPartner Coaching, Video Library, Course

29. Aligned professional development: MTP  Connecting outcomes and inputs - Alignment  CLASS – specific definitions of interactions  Video Library – analysis of others’ interactions  Coaching – ongoing analysis/feedback on own interactions  Course – knowledge and analytic skills  All tested in RCTs

31. CLASS examples: PK-3

32. Summer/Fall 2005 myteachingpartner.net Video library: Secondary

33. Summer/Fall 2005 myteachingpartner.net Detailed video examples www.mtpsecondary.net

34. Summer/Fall 2005 myteachingpartner.net MTP consultation cycle Classroom video recording at an established time 1 Teacher reviews clips and reflects on practice 3 Consultant reviews and edits video clips 2 Teacher and consultant meet and discuss teaching practices 4

35. Summer/Fall 2005 myteachingpartner.net MTP Prompts: Feedback for teachers

36. Summer/Fall 2005 myteachingpartner.net MTP Prompts: Feedback for teachers

37. MyTeachingPartner research  240 participating pre-K teachers split into groups based on three conditions:  Materials (traditional)  Website access, Materials  Website access, Materials, MTP  Support high-quality teacher- child interactions with the CLASS as the frame

38. MTP Coaching improves interactions 3.5 4 4.5 5 September October November December January February March April May June Teacher Sensitivity MTP Control

39. Classrooms with high poverty benefit most from MTP coaching for teachers 3.5 4 4.5 5 5.5 September October NovemberDecember January February March April May June Teacher Sensitivity Coaching--100% Poor Control--100% Poor

40. Findings: Effects of MTP support  Teachers with MTP coaches  Grew more sensitive in interactions with students  Increased students’ engagement in instruction  Improved language stimulation techniques  High-poverty classrooms benefit a great deal  Early career teachers benefit from coaching and video  Children with MTP teachers  Made greater gains in tests of early literacy  Experienced lower levels of problem behavior  Demonstrated higher levels of expressive language

41. NCRECE professional development study NCRECE evaluates two PD supports to teachers (N=450) : In-service course on effective support of language/literacy development Course improves CLASS Instr. Qual. In-service consultation using MyTeachingPartner

42. MTP in High School  Same approach – MTP coaching, video library  Randomized evaluation study – >100 classrooms  6 th -11 th grades, all content areas  Teachers improved instruction; kids more attentive, engaged  Average student with MTP teacher improved 35 th - 60 th percentile on state high-stakes standards tests  Replicating results in another VA city

43. Scale-up – At this moment  CLASS in Head Start monitoring nationally (mandated in Federal law); 50,000 teachers  4800 pre-k classrooms in GA  Statewide for Birth-5 programs in 2 states  10,000 preschool classrooms in CA  1600 preschool classrooms in Chicago  17,000 4 th -11 th grades classrooms in Gates-funded MET study  Trained > 3,000 observers, 150 coaches  MTP – training local coaches

44. Continuing innovation  Video Assessment of Interactions and Learning (VAIL): Predict teacher performance before they set foot in the classroom. Hiring new teachers?  Web-based feedback systems to districts/states  On-line version of the MTP course  Teacher preparation programs, credentialing  Continued development of standardization and decision-making supports

45. Standardized observation of interactions Feasible, reliable and valid at scale – A scalable language and lens for classroom settings p-3 Three domains: Emotional, Organizational, Instructional appear valid across grades Work in K-12 years confirming pk-3 work Small increments, particularly in IS, matter for child outcomes A focus for teacher professional development and preparation to increase quality and child outcomes

46. Accountability and program development  New policy frameworks (QRIS; HCMS; RTT) offer potential but also require careful evaluation  Accountability systems – wide-scale monitoring and feedback at correct level of aggregation. Link to aligned, proven-effective PD to target improvements in metrics.  Require capacity – # of observers/quality control  Live or video coding and stakes of the data  Issues to be resolved: cutpoints, combining with other metrics, implementation

47. Program development and improvement  Align measures with feedback to systems/individuals and resources to improve  Aligned PD can “move” interactions into “active range.”  Incentives and mechanisms to use effective PD  Degrees: Award credit for participation in effective PD. Certification: Award for competent performance.

48. Implications for P-12  Standardized, valid observations can be scaled and integrated into human capital management and PD  Focus performance measures and PD on teachers’ interactions with children – common language and aligned systems  Aligned PD improves interactions and student outcomes – experimental studies  Incentives, supports, and targets for teacher behavior/performance in classrooms  We can improve teachers’ impacts on kids