1. Updates to The SLO End-of-year process
Script:
Thank you for joining us today to discuss updates to the SLO End-of-Year process. We have additional resources and supports that I hope will be useful for you.
My name is Jonathon Haradon. I lead SLO implementation here at DPS.
This webinar is being recorded and will be posted on the ARE website and in the Schoology courses: ‘Returning SLO Leads’ and SLOs for evaluators’, in case you have others in your school who may benefit from hearing this information, or in case you need to return to this information in the future. The deck on it’s own will also be posted, to allow easier access to links referred to in the appendix
Finally, if you have questions along the way, please feel free to type them in. Alison Mueller is on hand for additional support in answering questions as we go.
Supports for Reaching Collaborative Consensus on SLO Data Jonathon Haradon, Assessment Specialist – SLOs

2. During this webinar, we will:
Goals and Agenda
During this webinar, we will:
Review what support throughout the year you can provide that sets up reaching collaborative consensus
Discuss new updates to SLO process and new resources for when teacher and evaluator are unable to reach collaborative consensus
See what it looks like in the SLO Application
By the end of the webinar, you will:
Know when to use new, additional resources that support the SLO Process
Understand new updates to the end-of-year SLO approval process
Script:
We have a couple of goals today. By the end of the webinar, we want you to know when to use new resources that can support the SLO Process. These are new resources. They primarily focus on supporting teachers or SSPs and their evaluators with collaboratively reaching consensus on end-of-course SLO data.
Also, we’ll want you to leave with an understanding of new updates to the end-of-year process. For most teachers, end-of-year approvals will not look any different. These new updates are meant to support you and your teachers in more difficult situations; where evaluators are unable to reach consensus with the teacher.
So for our time today, during the webinar we’ll talk about how supporting teachers with SLOs throughout the year makes the end of the year smoother. We’ll look at the updates to the SLO process and new resources to use when the teacher and evaluator can’t agree on end-of-course SLO data. And finally, we’ll see what these updates mean for evaluators in the SLO Application.
New
New

3. SLO Timeline and SLOs in LEAP
Teachers and leaders have multiple opportunities for collaborative consensus that are built into the SLO Process.
Aug
Sept
Oct
Nov-Mar
April
May
Choice of learning objectives
Determine baseline preparedness levels
Official approval point
Ongoing progress monitoring;
Feb: Finalize SLO inclusion in LEAP
Determine end of year expectation levels
Mar 1: Finalize SLO inclusion in LEAP
Script:
I want to call out that by March 1st, teachers are supposed to indicate which SLOs will be included in LEAP. This only applies to a teacher who has initially submitted two SLOs (and one has at least 10 students), and only wants one to be included in LEAP scoring. They must indicate which one will not be used in LEAP in the SLO Application. You likely saw this in Principal and Teacher weekly in early January, requesting that leaders and teachers discuss this at mid-year conversations. Teachers should not wait until the very end of the year to decide if they want an SLO included in LEAP evaluation. They must indicate that now. After March 1st, if an SLO is marked as to be included in LEAP evaluations, they are expected to submit end-of-course levels and finalize it.
If a teacher only submitted 1 SLO, they don’t need to do anything. If a teacher submitted two SLOs and wants both in LEAP evaluation (our recommendation, by the way), they don’t need to do anything. This is just if a teacher submitted two, but now only wants to include 1 SLO. Remember that 1 SLO must have at least 10 individual students with baseline levels. If you have teachers who might be in this situation, please have them review the January 10th Teacher Weekly, or go directly to SLO resources explaining this. Links to both are in the Appendix.
New
Don’t wait until the end of the year to be surprised by end- of-course levels!

4. Additional Supports Requested
We’ve heard from both school leaders and teachers that in some cases, reaching consensus on SLO data between a teacher and an evaluator can be difficult.
Want more support for steps teacher and leader can take to reach collaborative consensus
In extreme cases where evaluations went to redress, wanted clearer documentation in the SLO Application on differing understanding of student end-of-course levels
Script:
We’ve heard from both school leaders and teachers that in some cases, evaluators and teachers can’t reach consensus on end-of-year SLO expectation levels. And you, as evaluators, want more support when those disagreements happen.
And last year, we heard that some school leaders felt they had to approve SLOs, even though they disagreed with some SLO data, in order for the LEAP evaluation process to move forward. And they had no way to formally document those disagreements.
The resources we’ll be talking about today, are meant to address those issues, and support a process that feels more collaborative, not combative.

5. Resources for On-Going Collaborative Consensus
Aug
Sept
Oct
Nov-Mar
April
May
Choice of learning objectives
Determine baseline preparedness levels
Official approval point
Ongoing progress monitoring;
Feb: Finalize SLO inclusion in LEAP
Determine end of year expectation levels
Mar 1: Finalize SLO inclusion in LEAP
Model SLOs in the SLO Resource Bank
SLO Beginning-of-year conversation protocol
This slide contains animation.
Script:
Let’s review resources are already available to use throughout the year which support you and teachers, in having the appropriate level of rigor in classrooms and in progress monitoring student learning. Links to all of these resources are found in the appendix of this presentation.
[click]
At the beginning of the year, leaders and teachers or SSPs should be reviewing the model SLOs in the SLO resource bank and determining which SLO best aligns with the focus of their instruction for that year, and the most important standards of the course.
With an SLO chosen, teachers and leaders, in data-team meetings, should be determining which assessments to use for preparedness and the year-long body of evidence. You can be using the Beginning-of-year conversation protocol, or something similar, to guide level-setting on appropriate rigor and determining where students are starting out at. The Recommended Preparedness Level Reports in Illuminate are an excellent resource as a recommendation for where students are starting out.
Teachers or SSPs and evaluators should have 3-4 touchpoints on SLOs throughout October through April. When evaluators and data teams use a DDI protocol to evaluate student progress, they should occasionally connect that progress to a broader year-long perspective. The Data Team protocol with SLO Check-in is an example of what this could look like.
The mid-year conversation provides an excellent opportunity for one of those SLO touch points. Both the Data Team protocol and the SLO Mid-Year protocol ensure that you know the progress that teachers believe students are making. If rigor expectations are misaligned, these are opportunities to ensure that the rigor students must master is appropriate and high.
Moving into the end of the year, teachers should revisit the performance-based task connected with a model SLO. With a body of evidence collected throughout the year, teachers then determine end-of-course expectation levels. The SLO end-of-year conversation protocol, or something similar, can be a guide.
Illuminate Recommended Preparedness Level Reports
Data Team protocol w/ SLO Check-in
SLO Mid-year conversation protocol
End-of-year Performance-Based-Task linked to Model SLO (w SLOs in Resource Bank)
SLO End-of-year conversation protocol

6. SLO Supports and Engagement with Teachers
Three Adobe Connect polls will appear in a moment
Also, in the webinar general chat:
What SLO-related resource do you wish existed?
Show live data from poll as folks are answering. These polls are loaded via Adobe Connect and are not embedded in this slide. See below in the notes for the poll questions.
Script:
Ok, we’re going to pause and allow for some interaction from you. We’ve just talked about a lot of resources that are available. We want to hear what resources are useful for you. Three Adobe Connect polls should now appear on your screen. Please take a moment to consider and respond to those questions now.
Also, in the questions/comment box, please answer the question: What SLO-related resource do you wish existed?
As people answer, you will see a live refresh of the poll results.
[Review poll results]
Poll questions:
Select all that apply. I, or other leaders at my school, used the following resources:
Math or Literacy Recommended Baseline Preparedness Reports in Illuminate
Model SLOs and/or the Model SLO Resource Bank
SLO Beginning of year conversation protocol
Data Team protocol w/ SLO Check-in
Did you (and/or other evaluators) discuss SLOs with teachers at the mid-year conversation?
After setting SLO baseline levels, how often throughout the year do you discuss student progress on – or student work related to – SLOs with teachers?

7. Using Additional Supports If Cannot Reach Collaborative Consensus
If after requesting revisions, cannot come to quick consensus:
Reaching Collaborative Consensus Guide
If after requesting revisions, and after you’ve worked with the teacher and other in-building supports:
Reach out to a Neutral Content Expert
Guidance for Neutral Content Experts
Neutral Content Expert Feedback Template
New
This slide contains animation which provides for a slow reveal of text.
Script:
Lets talk about new resources for reaching collaborative consensus on end-of-course levels.
There will be times that a teacher or SSP submits an SLO, but you need to request revisions, because you have a few questions. And in most cases, these are resolved quickly, the teacher re-submits and the SLO is approved.
[Click for first animation]
In a small number of cases, you might not be able to reach consensus with the teacher or SSP right away. The ‘Reaching Collaborative Consensus Guide’ is for these cases.
The Collaborative Consensus guide initially walks through what you are most likely already doing when working on setting end-of-course expectation levels, that
Data teams meet their evaluators to discuss end-of-course levels, (potentially using a protocol like the end-of-year protocol)
If a teacher submits levels and the evaluator disagrees, the evaluator requests revisions.
If after requesting revisions from a teacher or SSP, and after working with the teacher again (potentially bringing in other in-building supports), if you and the teacher are still unable to reach a collaborative consensus, then we would recommend reaching out to a central office, neutral content expert.
We want to emphasize that this person should be viewed as a neutral voice by both the teacher or SSP and school leader. The content expert is not supposed to be a mediator or arbiter working towards a compromise. Their role is effectively a professional development one: supporting deeper understanding of the rigor of standards and the level of mastery student work represents.
To best support, the content expert will need some information from the teacher and school leader:
The SLO being used, with the learning progression
The assessments used as the Body of Evidence (including when they were administered)
Student work for 2-3 students where the teacher and evaluator disagree on the appropriate end-of-course levels. If the teacher and evaluator disagree on more students, they should still only send a selection of 2-3. The content expert feedback on the students submitted can be used as a guide for other students.
The feedback template linked here is guidance we provide to content experts on feedback they give, but they may choose some other format. And you are free to ask additional questions the feedback template does not address.
New

8. Adobe Connect Chat Boxes
Conversations with your peers
Box 1: What has your school done in the past when a teacher and evaluator disagreed on what the end-of-course SLO levels should be?
Box 2: What takeaways about SLOs do you have from mid-year conversations?
Box 3: Share strategies for coaching content areas outside your expertise (particularly BCC areas).
Chat rooms can be pre-built in Adobe Connect. All rooms are visible to every attendee, and attendees can move between rooms.
Script
We find that school leaders sharing each their experiences in their buildings can be particularly valuable. So we’d like to try and get you talking to each other now, using separate Adobe Connect Chat boxes.
In a moment, you will see on your screen 3 chat box options. You can choose anyone of them. They each have a different discussion prompts. The prompts are on this slide here, and are also the first line in each chat room.
Please choose a chat box that you can commit to either responding to the prompt, or entering your own question you have that relates to the prompt. This is an opportunity for you to share and learn from each other! So I hope that we can count on your engagement here. We’ll spend 6 minutes giving you time to respond to the prompts and each other’s questions.

9. Final Step – Documentation of Lack of Consensus
New
To allow for better documentation of lack of collaborative consensus, the school leader can use their best, informed professional judgement to submit end-of-course levels for students.
Only Principals or Assistant Principals can enter modified information
Entering modified SLO data should be considered a last resort only. All efforts should be made at reaching consensus.
If after:
A: the teacher or SSP and evaluator have attempted to find consensus on levels, and
B: after support of others in the school has been utilized, and
C: after a content expert from the district has provided guidance;
If after all of these supports, the teacher and school leader are still not able to arrive at consensus on student end-of-course levels, then we have updated the SLO Application to better allow for the teacher and school leader to document this lack of consensus.
A school leader can now, when all efforts at collaborative consensus have fallen short, enter modified end-of-course SLO data and levels based on their best, informed professional judgement. It’s really important to know that only Principals and Assistant Principals can add these modified levels. Senior Team Leads and Deans cannot. If a principal or AP is not the evaluator, they will need to be brought up to speed and involved in the lack of collaborative consensus.

10. LEAP Implications
The SLO Process should support a LEAP evaluation process moving forward appropriately
The leader’s end-of-course SLO data and expectation levels are used to calculate an SLO score
If the leader’s modified levels would have caused a change in the Student Growth rating, a teacher may initiate redress on the difference in SLO levels
New
This slide has animation for a slow reveal of text.
Script
We want to call out specifically what this means for LEAP evaluation for teachers (or SSP-GPS evaluation for SSPs).
The SLO Process must support finalizing evaluations. It’s state law, that a draft, of the complete LEAP evaluation, should be shared with teachers, by two weeks before the end of the school year. So SLO approvals need to be able to move forward even if the leader and teacher continue to disagree.
[click]
If a leader submits modified levels for some students, the levels the leader submits [] are used to calculate the SLO score. This SLO score from leader-entered data is then sent to LEAP and used in Student Growth.
We have created a method for a leader to submit modified levels so that there is better clarity on the lack of collaborative consensus, if redress occurs. Remember, a leader should only enter modified levels after multiple attempts at consensus, including contacting a neutral content expert.
A teacher can initiate redress if the modified levels would have caused a change in the LEAP Student Growth rating.
New

11. Functionality will go live end of day, March 5
SLO Application
Functionality will go live end of day, March 5
Finally, we’re going to show you what this looks like in the SLO Application.
This is not currently in place. It will go live approximately March 15.

12. Evaluator Steps in the SLO Application
New
Evaluator Steps in the SLO Application
Video assumes the teacher has submitted the SLO, revisions were requested, and the teacher has resubmitted.
[Narrate as video is running]
I’m going to play a video now of what this looks like in the SLO Application. I’ll talk through parts of it as the video is playing.
1. Evaluator must confirm the following:
Documented meeting multiple times to discuss student levels
Additional support has been sought to support reaching consensus
The teacher will be informed that evaluator plans to submit modified levels (the teacher should be informed before the leader submits the modified levels)
2. Check box in column: ‘Provide evaluator EOCEL’
3. Add modified information. The following may be modified:
SLO end-of-course expectation levels
Growth points for SLO Scoring Matrix decision boxes
Inclusion/exclusion in SLO scoring
4. Add comments, in pop-up dialog, on why modifying information
5. Submit
Not shown in this video:
An then goes to the teacher clearly showing the differences between teacher-entered and leader-entered SLO data.
Teacher- and leader-entered levels are visible to both the teacher and leader.

13. Evaluator Steps in the SLO Application
New
Evaluator Steps in the SLO Application
For the school leader to enter SLO levels:
Must have requested revisions at least once and teacher has resubmitted
Must confirm (in pop-up dialog screen) they have:
Documented meeting multiple times to discuss student levels
Sought out additional support for reaching consensus
Informed the teacher that evaluator plans to submit modified levels (before they are submitted)
Must add comments for each student for whom they enter a different level
After you submit:
Teacher receives confirmation
SLO Scoring matrix in SLO Application shows differences
To recap what we saw in the video:
For a school leader to enter SLO levels,
they must have requested revisions at least once and the teacher has resubmitted
The leader must confirm they have:
documented meeting multiple times to discuss student levels
Sought out additional support for reaching consensus
And you’ve informed the teacher, before you submit modified levels, that you plan to submit them
Must add comments for each student for whom you enter a modified level. These should be as detailed as possible.
What we didn’t see in the video, is that after you submit, the teacher will receive an automated clearly showing exactly what was modified. Also, in the SLO Application, the SLO Scoring matrix will show the differences between the teacher submitted and leader-modified levels. Both the teacher and leader can see this SLO scoring matrix and the differences between teacher and leader-entered levels.

14. These were our goals Questions or feedback?
Know when to use new, additional resources that support the SLO Process
Understand updates to the end-of-year SLO Process
Questions or feedback?

15. Appendix Finalizing SLOs to be included in LEAP
Resources for On-Going Collaborative Consensus

16. Finalizing SLOs to be included in LEAP
Teacher Weekly
ARE guidance
If a teacher only submitted 1 SLO, they don’t need to do anything.
If a teacher submitted 2 SLOs and wants both in LEAP, they don’t need to do anything.
If a teacher submitted 2 SLOs, but now only wants to include 1 SLO, they must indicate such in the SLO Application by March 1st. Remember that the 1 SLO they do include must have at least 10 individual students with baseline levels.

17. A B C Resources for On-Going Collaborative Consensus Aug Sept Oct
Nov-Mar
April
May
Choice of learning objectives
Determine baseline preparedness levels
Official approval point
Ongoing progress monitoring;
Feb: Finalize SLO inclusion in LEAP
Determine end of year expectation levels
Mar 1: Finalize SLO inclusion in LEAP
A. Model SLOs in the SLO Resource Bank
A. SLO Beginning-of-year conversation protocol
A. Illuminate Recommended Preparedness Level Reports
B. Data Team protocol w/ SLO Check-in
B. SLO Mid-year conversation protocol
C. End-of-year Performance-Based-Task linked to Model SLO (w SLOs in Resource Bank)
C. SLO End-of-year conversation protocol