1. North Carolina Observation Calibration Process (OCT)

2. What is Observation Calibration Training?
Observation Calibration Training (OCT) is an online evaluator training platform combining BloomBoard’s PD resources with Empirical Education’s observer training and calibration tool, Observation Engine.
Goals of the OCT:
Increase observation skills resulting in stronger scoring calibration
Increase rater agreement
Provide a common experience for LEAs to host collaborative conversations to improve instructional leadership skills

3. Teachers deserve to be observed accurately and receive relevant feedback for improvement.
Teacher evaluators must have training to accurately rate teacher performance and provide feedback to promote improvement.

4. What is Observation Engine?
Online training tool that uses classroom videos to ensure accurate and reliable classroom observations
Videos from the MET (Measures of Effective Teaching) library have been master-scored on the NCEES teacher evaluation rubric to create learning opportunities for teacher evaluators in North Carolina. Master scoring including Educator Effectiveness staff working with Empirical Education staff.

5. Target Participants include:
practicing school administrators, teacher leaders, and central office administration
MSA and principal preparation students

6. Types of Participation:
Individual: self-directed
Participant in a facilitated group
Participant and manager of a facilitated group

7. Observation Engine NCEES Content
2 Observer Calibration Events – Scoring Studies
Rating video practice to increase rater agreement to target scores
Scoring Study 1 (video) is administered as an initial assessment of calibration
Scoring Study 2 (video) is administered at the end of the year as a post- assessment
19 Targeted Learning Exercises - Lessons
Targeted, self-paced online exercises with immediate on-screen feedback.
The OCT contains two types of lessons:
2 Full classroom observations
17 Element-specific classroom observations

9. Personalized, competency-based growth for educators

10. Thousands of curated collections of resources
Find collections of high quality learning resources, handpicked and vetted by experienced educators

11. Aligned to competency-based
micro-credentials
Explore over 200 micro-credentials, and earn recognition for the skills and competencies you’ve developed.

12. Virtual PLCs to enhance collaboration

13. Portal to Observation Engine
BloomBoard is accessible through NCEdCloud using IAM through the Educator Evaluation Professional Development System. 

14. Through BloomBoard, participants can easily access the OCT Home Page, Observation Engine, and supporting collections and micro-credentials.

15. Scoring Accuracy Improved
Key metrics:
Percent Target Agreement: the percent of scores that agree exactly with the target scores
Percent Target Discrepant: the percent of scores that disagree with the target scores by 2 or more performance levels (e.g. when the target score is 2 and the score provided is a 4)
T tests revealed that in Scoring Study 2, percent target agreement was higher (p<.0001) and percent discrepant was lower (p<.0001) than in Scoring Study 1

16. Completing OCT Lessons Improved Scoring
Regression analysis revealed that the number of lessons completed significantly increased target agreement and decreased target discrepancy
This relationship was particularly strong for observers that performed lower on Scoring Study 1, but all observers benefitted from taking lessons

17. Observation Calibration Training Participation

19. Questions? Kimberly Simmons Kimberly.simmons@dpi.nc.gov
Observation Calibration Training Wiki;