North Carolina Observation Calibration Process (OCT)

What is Observation Calibration Training? Observation Calibration Training (OCT) is an online evaluator training platform combining BloomBoard’s PD resources with Empirical Education’s observer training and calibration tool, Observation Engine. Goals of the OCT: Increase observation skills resulting in stronger scoring calibration Increase rater agreement Provide a common experience for LEAs to host collaborative conversations to improve instructional leadership skills

Teachers deserve to be observed accurately and receive relevant feedback for improvement. Teacher evaluators must have training to accurately rate teacher performance and provide feedback to promote improvement.

What is Observation Engine? Online training tool that uses classroom videos to ensure accurate and reliable classroom observations Videos from the MET (Measures of Effective Teaching) library have been master-scored on the NCEES teacher evaluation rubric to create learning opportunities for teacher evaluators in North Carolina. Master scoring including Educator Effectiveness staff working with Empirical Education staff.

Target Participants include: practicing school administrators, teacher leaders, and central office administration MSA and principal preparation students

Types of Participation: Individual: self-directed Participant in a facilitated group Participant and manager of a facilitated group

Observation Engine NCEES Content 2 Observer Calibration Events – Scoring Studies Rating video practice to increase rater agreement to target scores Scoring Study 1 (video) is administered as an initial assessment of calibration Scoring Study 2 (video) is administered at the end of the year as a post- assessment 19 Targeted Learning Exercises - Lessons Targeted, self-paced online exercises with immediate on-screen feedback. The OCT contains two types of lessons: 2 Full classroom observations 17 Element-specific classroom observations

Personalized, competency-based growth for educators http://bloomboard.com

Thousands of curated collections of resources Find collections of high quality learning resources, handpicked and vetted by experienced educators

Aligned to competency-based micro-credentials Explore over 200 micro-credentials, and earn recognition for the skills and competencies you’ve developed.

Virtual PLCs to enhance collaboration

Portal to Observation Engine BloomBoard is accessible through NCEdCloud using IAM through the Educator Evaluation Professional Development System. 

Through BloomBoard, participants can easily access the OCT Home Page, Observation Engine, and supporting collections and micro-credentials.

Scoring Accuracy Improved Key metrics: Percent Target Agreement: the percent of scores that agree exactly with the target scores Percent Target Discrepant: the percent of scores that disagree with the target scores by 2 or more performance levels (e.g. when the target score is 2 and the score provided is a 4) T tests revealed that in Scoring Study 2, percent target agreement was higher (p<.0001) and percent discrepant was lower (p<.0001) than in Scoring Study 1

Completing OCT Lessons Improved Scoring Regression analysis revealed that the number of lessons completed significantly increased target agreement and decreased target discrepancy This relationship was particularly strong for observers that performed lower on Scoring Study 1, but all observers benefitted from taking lessons

Observation Calibration Training Participation  

Questions? Kimberly Simmons Kimberly.simmons@dpi.nc.gov Observation Calibration Training Wiki; http://nc-oct.ncdpi.wikispaces.net/home