1. Improving Impacts in Diverse Classrooms: Professional Development and Classroom Observation
Robert C. Pianta, Ph.D.
Dean, Curry School of Education
Founding Director, Center for Advanced Study
of Teaching and Learning
Discover. Create. Change.

2. Realities: Effective teachers and teaching
Right now a good teacher is a matter of luck.
Tests show teachers/teaching are key, but cannot produce improvement – posting scores, firing teachers won’t fix this.
Teachers can narrow gap by half or more/year. $2-$16 return on $1. K-teacher can be worth $320,000 to a child she serves.
Districts waste PD money. Spend between $2500 and $9,000/teacher/year. No evidence of impacts. No evidence of impact of advanced degrees, pay, etc.
Move from luck to guarantee — the production of effective teaching.

3. Investing in effective teachers/teaching
Everyone agrees professional development is a critical pathway to achieving performance goals; no evidence that PD improves performance
A complex system – resources, certification/licensure, observed performance, student learning – too few metrics for key markers
Alignment of metrics/mechanisms that produce effective teaching and learning. Stakes aligned with measures: Incentives (merit pay, credit), licensure/certification, sanctions
Decisions supported by technical properties of reliability and validity

4. A focus on student-teacher interactions
Interactions with adults form “infrastructure” for school success:
Self-regulation, emotional self-control
Task orientation, persistence, motivation
Cognitive processes, language
Instruction is, in part, a social process/system:
Interactions with teachers are a “medium” for engagement
Good instruction is embedded in interactions
Properties of interactions are organized at level of relationships and operate across content, ethnicity, and culture, and can be focused on specific intended outcomes

5. Understanding the CLASS Framework: Overview
Teacher-Child Interactions
(these matter)
Emotional
Support
Classroom Organization
Instructional
Support

6. Measuring interactions: CLASS
CLASS is a tool for observing and assessing the quality of interactions between teachers and students
Ratings (1-7) of the emotional, organizational, and instructional supports provided by teachers that contribute to children’s social, developmental, and academic achievement.
CLASS is used to assess interactions among teachers and students for a variety of purposes:
Teacher Professional Development
Monitoring and Evaluation of Teacher Performance/ Effectiveness
Research

7. Dimensions of interaction: CLASS PK-5
Positive climate
Negative climate
Teacher sensitivity
Regard for student perspectives
Effective behavior management
Instructional learning formats
Productivity
Concept development
Quality of feedback
Language modeling
Emotional
Support
Organization/
Management
Instructional
Support

8. How is the CLASS organized?
Emotional Support
Domain
Dimension
Indicator
BehavioralMarkers
Refer to the structure in the manual – diagram on page 21.
Be sure to emphasize throughout training that we are looking for BOTH the presence or absence of behaviors specific to each of the dimensions.
In the manual, there is a face page for each dimension that provides an overview of the dimension; there are also brief descriptors that provide an overview of behaviors associated with low, mid, and high range interactions for each indicator.
The face page is ONLY a good first place to start/reference. When coding you should turn to the more descriptive pages to assign a code. Don’t rely solely on this page or the laminated tri-fold that comes with the manual.

9. Understanding the CLASS Framework: Overview
What to Know:
The next three slides are here again to allow the facilitators the opportunity to practice facilitating the research as this is often an area that most trainers describe as challenging. Those facilitating may use the notes below and those on the next two slides to guide their discussions:
First, what do we know about interactions in prek-3rd classrooms nationwide?
Based on CLASS data collected from several thousand pre-k-3rd classroom throughout the country, we know that:
Children tend to experience moderate to high levels of effective interactions for emotional support and classroom organization (red/green)
However, we also know that most children attend pre-k-3rd classrooms characterized by low levels of instructional support.
Observations at the population level in US early education – very similar results in upper grades

10. Interactions and children’s development
Emotional Support
Instructional Support
ECERS-R Total
Structural
Receptive Language 
Expressive Language
Rhyming
Letter Naming
Math Skills
Social Competence
Behavior Problems
Changes in children’s development from beginning to end of the year Mashburn, et al. (in press)
As you can see, what we found in these preschool classrooms was that the CLASS Instructional dimensions in particular were associated with children’s language learning (can go through more specifics).
Please note that this analysis used a previous grouping of CLASS dimensions so ES contained PC, NC-r, TS, OC-r, and BM, and IS contained PD, ILF, CD, and FDK

11. Do PK effects persist into kindergarten?
Yes, children in pre-k classrooms offering higher levels of Instructional Support displayed better language skills at the end of the kindergarten year.
Kindergarten Instructional Support scores made an independent contribution to gains in children’s language and math abilities.
One-point differences in observed instructional supports appear linked to shifts in child outcomes.

12. Similar findings for dual-language learners?
Two major studies
Does CLASS describe interactions in language- diverse settings?
Does CLASS predict outcomes similarly?
CLASS 3-factor structure and psychometric properties same across diverse classrooms
Predicts outcomes similarly for DLLs
Evidence supports use in diverse settings

13. Gains in grade 1 achievement in instructionally supportive classrooms
107
106
105
Standardized tests of achievement adjusted
104
103
High educ.
102
101
Low educ.
100 99 98
Low
Moderate
High
1st Grade Instructional Support

14. Gains in grade 1 achievement in emotionally supportive classrooms
107
106
Kindergarten
adjustment
problems
105
104
Standardized tests of achievement adjusted
103
No problems
102
101
Multiple problems
100 99 98
Low
Moderate
High
1st Grade Emotional Support

15. Results from Gates MET study
Students learn more in classrooms with higher CLASS scores
Students on average scored best in classrooms where the teacher had a Highly Effective Overall CLASS score and worst in classrooms where the teacher had an Ineffective Overall CLASS score. These scores are an illustration of the cut points based on achievement in Math classrooms so they are slightly different than the cut points above, which include all teachers.
Effect Size of 1 = 1 SD more in units of student achievement
Magnitude of Differences in Learning Based on CLASS Effectiveness Levels
(Effect Size)
.95
.23
.21

16. Interactions in 6th-12th grade: CLASS-S
Predicts state-standards test scores across all content areas (>2,000 students, >100 classrooms)
Moderate effect sizes (. 30) in all content domains
Also predicts observed student engagement
Accounts for gains in standards tests in the current and subsequent year – measuring stable attributes of teachers
Consistent results across ethnicity, income, language background of students

17. CLASS and language, culture
Sample are large and diverse; many tens of thousands of classrooms. Able to see trends
Consistent results for reliability – observers can use it accurately across diverse settings
Validity results are consistent – CLASS domains relate to student learning and development in similar ways in diverse groups
All analyses examine family and related background characteristics – common lens

18. PD - Improving Teacher-Student Interactions
The Goal: professional development needs to be:
Sustained – Focus on development of skills
Intensive – willing to push and stretch the teacher
Classroom-focused and job-embedded
Evidence-based
The Reality: 79% of teachers reported involvement in traditional PD workshops in comparison with only 5-16% who reported opportunities for more active learning (Birman et al., 2000).

19. Aligning observation with PD to improve
Observe teacher behavior that matters for student learning – Instructional Support, student engagement and motivation
“Engineer” and test models of PD that produce those behaviors
Key feature of alignment is focus on defined, observable, valid examples of effective teaching interactions as starting point

20. Aligned professional development: MTP
Connecting outcomes and inputs – Alignment
CLASS – specific definitions of interactions
Video Library – analyze others’ interactions
Coaching – ongoing analysis/feedback on own interactions
Course – knowledge and analytic skills
All tested in RCTs

21. 21 21

22. CLASS examples: PK-3

23. Video library: Secondary
myteachingpartner.net
myteachingpartner.net

24. Detailed video examples
myteachingpartner.net 24

25. MTP Coaching Description
Objectives
Structure
Detect and analyze own interactions - specifics
One-on-one coaching cycles every 2 weeks
Local coaches
Coaching manual
Weekly UVa support
Elements
Cycle
Video library
Prompts

26. MTP Prompts: Feedback for teachers
myteachingpartner.net

27. MTP Prompts: Feedback for teachers
myteachingpartner.net

28. Nice job Prompt Goals of Nice Job Prompt Attend to positive exemplars
Explicit focus on CLASS
Build teacher’s sense of efficacy
Hone teacher’s behaviorally grounded observational skills
Promote intentionality—reading and responding to children’s cues in the moment
Build teacher-coach relationship

29. Let’s look at your classroom
Goals of Let’s Look Prompt
Improve observation skills
Recognize impact on children
Identify opportunities for change
CLASS focused
Positive at beginning of year, but over time push to critically examine practice

30. Course description/aims
Objectives
Structure
Identify connections between interactions and learning
Describe elements of effective interactions
Accurately observe and label interactions
14 weeks (3-hour/week)
UVa-trained instructors
Instructor manual
Standard PPTs & videos
Weekly UVA support
Credit-bearing
Emphasis on video review

31. MyTeachingPartner research
240 participating pre-K teachers split into groups based on three conditions:
Materials (traditional)
Website access, materials
Website access, materials, MTP
Support high-quality teacher-child interactions with the CLASS as the frame
Key features – specificity and standardization
TALKING POINTS:
UVA’s Center for Advanced Study of Teaching and Learning undertook a two-year study on MyTeachingPartner.
This was the first study of MTP. This first study and the second study of MTP, which we will discuss in a few minutes, provide the research-based model for our implementation of MTP. 31 31 31 31

32. MTP Coaching improves interactions5
4.5
MTP
Teacher Sensitivity
Control 4
TALKING POINTS
Teachers in the MTP consultancy were compared to teachers with web-only access. On average, you can see teachers in MTP improved over the year in Teacher Sensitivity (TS). Teachers in the web group decreased a little in quality in TS.
3.5
March
April
May
June
October
November
January
September
December
February

33. Classrooms with high poverty benefit most from MTP coaching for teachers
5.5 5
4.5
Coaching--100% Poor
Teacher Sensitivity
Control--100% Poor 4
JEANNE PLEASE CUT OUT OF THIS SLIDE THE 50% POOR NOTES FROM THE LEGEND AND THE CORRESPONDING LINES IN THE GRAPH
3.5
April
May
June
September
October
March
November
December
January
February

34. Findings: Effects of MTP support in PK
Teachers with MTP coaches
Grew more sensitive in interactions with students
Increased students’ engagement in instruction
Improved language stimulation techniques
High-poverty classrooms benefit a great deal
Early career teachers benefit from coaching and video
Children with MTP teachers
Made greater gains in tests of early literacy
Experienced lower levels of problem behavior
Demonstrated higher levels of expressive language
TALKING POINTS:
Here are more of the findings of the MTP Professional Development Study. 34 34 34 34

35. NCRECE professional development study
NCRECE evaluates two PD supports:
In-service course
on effective interactions for language/literacy
In-service coaching using MyTeachingPartner

36. NCRECE PD study research design
Randomly selected into Course or No Course
Phase I
Summer
2008
Course
No Course
Randomly selected into Coaching or No Coaching
Also note
Two cohorts
Follow-up in classrooms during year directly following coaching intervention
Phase II
Fall 2008-
Spring 2009
Coaching No
Coaching
Coaching No
Coaching
During coaching phase, 4 children (2 boys, 2 girls) were randomly selected from each classroom for assessments.

37. Impacts on teaching practice*

38. The importance of “Seeing”
Standardize – coaching is expensive – want it to tightly align with program outcomes

39. What did we learn from course?
Teachers demonstrated changes in beliefs, knowledge, skills and practices
Effects on practice were strongest for instructional interactions – were sustained 1 year later
Course was effective for a range of teachers
Course promoted better interactions in the classroom through increasing teachers’ observation skills (VAIL) – can be trained
Standardize – coaching is expensive – want it to tightly align with program outcomes

40. Coaching impacts on instructional support
***
***
*p<.05; ** p<.01; ***p<.001

41. Effects on Instruction vary by coach
Coach*IS effect
Range from 4-14 teachers (most have 8-12)
Even when working in the same site, differences across coaches
Coach

42. Implementation results and implications
Effective PD interventions can be delivered locally with high fidelity and quality.
Quality of implementation matters for teacher engagement and for benefits of PD.
“Implementors” need focused support. Biweekly conference calls, check-ins, standardized protocols all essential to keeping focused.

43. Key impacts of Instructional Support prompts
Outcome
Predictor
Location of Maximum
Emotional Support
ES prompts
CO prompts
IS prompts*** --
12.26
Classroom Organization
CO prompts*
IS prompts*
5.15
10.93
Instructional Support
11.14
14.81
We hypothesized that of the two specific elements of dosage, the prompts represent a more intense, individualized experience of feedback to teachers.
Mean prompts: ES 5.43; CO 4.57; IS 7.77
Analytically, we examined the extent to which teacher-child interactions in the three CLASS domains could be predicted from the number of Classroom Organization, Emotional Support, or Instructional Support prompts.
Greater exposure to coach prompts in Instructional Support was associated with gains in all three CLASS-observed domains of teacher-child interaction. More specifically, the number of Instructional Support prompts had an inverted U-shaped (decelerating) relation with all three outcomes, with the point of maximum gain ranging from 11 to 15 prompts, suggesting diminishing gains in teacher-child interactions once teachers had been exposed to prompts in this range. It took more Instructional Support-focused prompts to reach the maximum gains in Instructional Support (approximately 15) than to reach maximum gains in Classroom Organization (approximately 11), perhaps suggesting the relative difficulty of changing instructionally supportive behavior. Furthermore, the number of Classroom Organization prompts demonstrated a similar inverted U-shaped relation with observed Classroom Organization and Instructional Support, with the point of maximum gain ranging from 5 to 11 prompts, with instructional behavior again requiring a larger dosage of prompts in order to reach the deflection point.
Focus on instructional support associated with all domain changes
Higher maximum for instructional support

44. MTP in secondary classrooms
Same approach – MTP coaching, video library
Randomized evaluation study – >100 classrooms
6th-11th grades, all content areas
Teachers improved instruction; kids more attentive, engaged
Average student with MTP teacher improved 10 percentile ranks on state high-stakes standards tests
Replicating results in another VA city
The levels of the CLASS presently available extend the instrument’s usability beyond assessment of teacher-child interactions in classrooms. Now, the tool can also be used to measure the presence of consistent types of effective interactions across levels and grades, accompanied by descriptions and examples specifically tailored to each age group. By providing a common metric and language for discussion of classroom quality across the early childhood years and into K-12, the CLASS addresses concerns with grade-to-grade transition and the need for coherence, while still providing a context-specific and developmentally-sensitive metric for each age group.

45. Online course: Mechanism to foster intentional teaching
Intentional teaching requires teachers to know what to do in each moment, see effective teaching in themselves and others, enact these strategies in the classroom, and reflect on, or analyze what works and what does not.

46. Active ingredients for changing practice
Enhanced ability to “see” effective practice in self and others
Video library
Review of own video
Course – focus on ineffective practice as well
Emotionally supportive context for change
Coach & Instructor
“Nice Work” prompt in MTP
Professional Development
Knowledge
Schemas
Relationships

47. Online course components
1. Online Content
2. Analyze Videos of Others
3. Analyze own Video
4. Talk with Instructor

48. Online content
Help teachers “chunk” complexities into 10 key elements of practice
Dimensions
Indicators
behaviors
Tightly couple knowledge of interactions with visuals of what it looks like in real classrooms

49. Analysis of others’ teaching

50. What about making observations?28 19 5 29 15
(throughout video)
(46 - 1:02)
27 learners
96 tags made: 76 correct, 20 incorrect
Of the 76 correct tags: 74 were tagged in the right time frame
Haven’t analyzed descriptions yet to see what they wrote.
The 15 Social conversation taggers mentioned the teacher talking about the smell of her hand, the remark about the girl’s long hair, or described a conversation related to the hamster

51. Accountability and program development
New policy frameworks (QRIS, HCMS, RTT) offer potential, but also require careful evaluation
Accountability systems – wide-scale monitoring and feedback at correct level of aggregation. Link to aligned, proven-effective PD to target improvements in metrics.
Require capacity – # of observers/quality control
Live or video coding and stakes of the data
Issues to be resolved: cutpoints, combining with other metrics, implementation

52. Improving impacts of classrooms
Standardized, valid observations can be scaled and integrated into human capital management and PD
Focus performance measures and PD on teachers’ interactions with children—common language and aligned systems
Aligned PD improves interactions and student outcomes—experimental studies
Incentives, supports, and targets for teacher behavior/performance in classrooms—implications
We can improve teachers’ impacts on kids

53. Appreciation and collaborators
Institute of Education Sciences support for the National Center for Research on Early Childhood Education
Eunice Kennedy Shriver National Institute for Child Health and Human Development
University of Virginia Center for Advanced Study of Teaching and Learning
Colleagues and collaborators at Frank Porter Graham Child Development Center at University of North Carolina Chapel Hill; UCLA, UNC-Greensboro.
Dozens of partner programs and districts and the hundreds of teachers who have worked with us