1. Using Student Growth Percentiles in Educator Evaluations
Oregon Department of Education
Educator Effectiveness
Summer Assessment Institute
2015

2. Outcomes for Today
Common understanding of SLG goal requirements for
Explain the role of Student Growth Percentiles (SGPs) in relation to Category 1 Assessments
Answer questions
TANYA
Because there will be lots of new information…
Please jot down questions as they arise
Q & A opportunities will be provided throughout today’s presentation
PUT UP PARKING LOT

3. SLG Requirements for 2015-16 8 components including Rationale
REQUIRED
8 components including Rationale
Minimum of 2 SLG goals each year
*Statewide assessments (Category 1) required for teachers and principals in tested grades and subjects
Oregon Matrix used for summative evaluation
*Pending review of HB 2680
RECOMMENDED
Content is focused, not everything that is taught
Context can help ascertain instructional needs
Tier goals/targets where appropriate
Include the support needed to attain the goal
Even though ESEA waiver has been approved, we are now waiting on determinations from state level leadership on whether HB suspends the use of statewide assessments for the upcoming school year.

4. ESEA Waiver Requirement
Teachers in tested grades and subjects (Grades 4-8 ELA/Math) and principals must use state assessments as one measure of student learning and growth (SLG) in their evaluation
Oregon educators set two SLG goals annually; Category 1 goals are measured by state assessment; Category 2 goals are measured by other assessments
Teachers in tested grades and subjects & Principals
Category 1 SLG goal
Category 2 SLG goal
Teachers in non-tested grades and subjects
ELA/Math
Grades 4-8
As you know, the waiver requires the use of statewide assessments as a measure of SLG goal attainment for teachers and principals in tested grades and subjects for one of the two goals set.
In October of 2014, Oregon’s waiver was conditionally approved with the question remaining of how statewide assessments would be used consistently in evaluations across the state.

5. ESEA Waiver Update
Oregon’s waiver approved for 3 years with conditions removed
For evaluation purposes statewide assessments will only be used as a measure of SLG goal attainment for those grades that have baseline data
Tested grades and subjects are now ELA and Math 4-8
Oregon’s waiver was approved on July 23rd with all conditions removed. In that waiver Oregon indicated that it will only be using the statewide assessment for those grades in which baseline data is available. This means that tested grades and subjects are now grades 4-8. Grades 3 and 11 no longer required.

6. Two Options Using SGPs
ODE co-developed with the OEA, Chalkboard, and COSA two options that meet USED’s waiver requirements. Districts will choose one.
In both options, districts will use median Student Growth Percentiles (SGPs) based on the Oregon Growth Model
SGPs measure growth for an individual student
The median SGP represents the exact middle of students’ SGP scores
*Both options apply only to grade 4-8 ELA and math teachers and principals
The development of Oregon’s plan for incorporating statewide assessments was a collaborative effort and offers districts two choices.
Oregon’s approved waiver involves the use of median Student Growth Percentiles (SPGs) to fulfill the requirement of using state assessments in tested grades and subjects. SGPs measure growth for an individual student by comparing the change in his or her achievement on state assessments (Smarter Balanced) to that of his or her “academic peers” (those having similar historical assessment results)
The next section of the presentation will go into greater detail about how SGPs and median SGPs are determined.

7. Questions?
Field questions around SLG requirements and waiver. SGP info coming up next.

8. Student Growth Percentiles
(SGPs)
Normative growth measure
One year of growth
SARAH P
SGPs are a normative measure of student growth. SGPs quantify the growth an individual student made in one year relative to all other students in Oregon with similar score history in state assessments.
The slides we are going share in this next section are from a presentation created by OSPI in Washington
Growth is relative to students with a similar score history

9. What do we mean by normative growth?
What do we mean by normative growth? We are referring to the amount of growth a child makes in comparison to his or her peers. For instance, when a child goes to the doctor’s office, his weight and height are measured. The doctor can tell us if the child’s growth in the last year is “typical”, that is: is he growing the same, less, or more than other children his age?

10. Percentiles
35%
65%
35th
Often this is expressed in percentile terms. The child’s height may be at the 35th percentile which means that the child is taller than 35 percent of other children his age and shorter than 65 percent of children his age. Normative growth is the basis for student growth percentiles.

11. Anthony 3rd Grade 2262 Level 1 4th Grade 2398 Level 2
Let’s look at an example. Anthony was a 4th grade student last year. On his Smarter Balanced assessment in the spring, he scored a In the previous year, his 3rd grade score was There appears to be “growth” in that his score improved…but relative to what? How do we place Anthony’s growth in context?

12. Oregon Score Distribution 3rd Grade Reading Level 4 Level 3 Level 2
Let’s start by first examining the test score distribution for all students across the state in Anthony’s cohort.
We can see there is a spread of scores, from the “Level 1” range up to “Level 4”. Anthony’s score of 2262 is at the lower end of the distribution, in “Level 1”.
2432
2367
2490
ELA/Literacy Scale Score Thresholds

13. Grade 4 Grade 3 Level 1 Level 2 Level 3 Level 4 3rd Grade 4th Grade
2262
4th Grade
2398
Now, let’s look at Anthony’s 3rd and 4th grade reading scores in this context.
For simplicity, we’ll show a standard curve.
We can see that in 3rd grade he scored 2262 (or “Level 1”) and in the following year in 4th grade he scored 2398 (moving to the performance level of “Level 2”). 
Although Anthony did not meet standard in 4th grade, his score does show an improvement in performance. However, it remains unclear if this is an average or below average amount of growth.

14. 2398
- 2262
136 points
We want to understand how Anthony has improved. To do this we cannot simply subtract his 3rd grade score from his 4th grade score because Oregon state assessments is not vertically scaled, that is they do not combine to form one long yard stick from year to year so we cannot say he grew by XX points.

15. Anthony’s Comparison Group
BRIAN
Instead we attempt to define “above” and “below” average growth by placing Anthony’s growth in the context of a comparison group.

16. All 3rd grade test takers in Oregon
Level 1
Level 2
Level 3
Level 4
All 3rd grade test takers in
Oregon
When Anthony was in 3rd grade, he scored a Many other 3rd graders in Oregon took the same state test in reading. Anthony was just one student among them. Of all these 3rd graders in the state many others scored a 2262….
…with Anthony’s Prior Score ~2262
Anthony

17. Anthony’s Comparison Group
Level 1
Lev el 2
Level 3
Level 4
Let’s call this “Anthony’s Comparison Group” because these students had a similar level of academic proficiency as Anthony. Anthony’s comparison group is defined solely on their state assessment scores, not on any other characteristics. What we want to know is: relative to his comparison group, how well did Anthony perform in the next year - the 4th grade?
Anthony’s Comparison Group
Anthony’s prior
score ~2262
Anthony

18. 4th Grade Score Distribution
Level 1
Level 2
Level 3
Level 4
4th Grade Score Distribution
Middle score
High score
Here we can see the distribution of 4th grade test scores for Anthony’s comparison group. You can see that this is an overall lower performing subgroup of students relative to the statewide distribution
***
The highest scoring student met standard but
the middle score of Anthony’s Comparison Group is on the border of “Level 1” and “Level 2”.
Anthony’s Comparison Group
Anthony’s prior
score ~2262

19. 4th Grade Score Distribution
Level 1
Level 2
Level 3
Level 4
4th Grade Score Distribution
80% of students in Anthony’s comparison group scored below him
2398
Recall that this year Anthony scored Although Anthony is “Below Standard” in the state distribution, he is definitely above the middle score within his comparison group. Analyzing the data more closely, we can see that Anthony scored higher than 80 percent of students in his comparison group.
Anthony’s Comparison Group

20. 4th Grade Score Distribution
Level 1
Level 2
Level 3
Level 4
4th Grade Score Distribution
SGP = 80
This translates to a Student Growth Percentile (or SGP) of 80.
- 2 slow clicks –
Anthony’s Comparison Group

21. Processing Time With the colleagues at your table:
Talk about what you’ve heard so far
Identify a question your group still has about SGPs
(Up Next: Determining Median SGPs)

22. SARAH M
Next, let’s move from an individual child to the teacher level.
How can we calculate the median SGP?

23. Median Student Growth Percentiles
Students in Mr. Waters’ class
Student SGP Sheryl 3 Hector 22 Robert 36 Miranda 38 Tre 51 Anna 54 Eric 71 Anthony 80 Amina 93
To find the median, we first need a valid class roster. Using that roster, the growth percentiles for all the students in Mr. Waters’ class are listed in numerical order from lowest to highest. In this chart, assuming this was the total enrollment in Mr. Waters class, the median student growth percentile is 51. The median SGP shows that half of the students had growth above that level, and half had growth below.
This gives us a median number for that teacher, which we refer to as the “median student growth percentile”.
Median is used rather than the average, because an average can be influenced greatly by a few very high or very low scores. The median can give a better overall picture for a teacher.

24. Making a Determination of Student Growth
The Median Student Growth Percentile (SGP) determines the educator’s Category 1 SLG Rating
Much Less than Typical
Less than
Typical
More than
Category 1 Rating 1 2 3 4
Median
SGP
1 to 34
percentile
35 to 49
50 to 64
65 to 99
Interpretation
Majority of your students have “low” growth
Majority of your students have below “average” growth
Majority of your students have “above average” growth
Majority of your students have “high” growth
With a median SGP of 51, Mr. Waters would receive a score of 3 using the SGP rubric. This number is then used as one of the 2 SLG goal scores that determine his rating on the X axis of the Oregon Matrix. This means that Mr. Waters would only write and score one SLG goal.

25. X-Axis = SLG Rating Mr. Waters SGP was rated 3 Second SLG was rated 3
SLG performance level based on two goals
Two-year cycle select two of four goals
Score SLG goals
Get a rating between 1 and 4;
Use X-Axis thresholds to determine SLG level:
4 = both goals 4s
3 = both goals 3s; one goal 3 & one goal 4; one goal 2 & one 4
2 = both goals 2s; one goal 2 & one 3; one goal 1 & one 3; one goal 4 & one 1
1= both goals 1s; one goal 1 & one 2
Mr. Waters
SGP was rated 3
Second SLG was rated 3
X-Axis Rating = Level 3 SLG Rating
For the X axis, the determination of overall performance level is based on 2 goals. For those educators on a two year cycle, two of the four goals would be selected
With a score of 3 on the SGP rubric, Mr. Waters has one goal rated a 3. The goal he wrote that used a Category 2 measure was also rated a 3. Using the threshold this means his overall SLG rating is a 3.

26. Mr. Waters: Y-axis = 3 & X-axis = 2
The matrix is a decision-making tool. It is the instrument that bring two inputs (PP/PR and SLG) together to help you make a decision for the final performance level you need to report. Districts use the thresholds established by ODE to determine the rating for each axis. The Y axis represents overall performance related to Professional practice and responsibility and the X axis represents overall performance related to student learning and growth.
It is important to note that the matrix is not a uniform percent model, and that by design the Y axis has more influence on the overall score (summative rating) than the X axis. For example, if you score a 4 on the Y-axis (PP/PR) and a 1 on the X-axis (SLG), the matrix shows that you’re still a 3 overall. In a 50/50 percentage model, you would be a 2.5.
Going back to our example of Mr. Waters, recall that he had an SGP score of 3, and an SLG score of 3 which made his overall rating on the X axis a 3. Mr. Water’s professional practice and responsibility rating on the Y axis was a 3. You can see that with a 3 on the Y axis and a 3 on the X axis the overall rating is a 3 which would mean a Collegial path. This means that paths for professional growth are developed jointly by the educator and evaluator. Even if Mr. Waters had scored a 2 overall on the X axis he would still be a 3 overall and on the Collegial path.
*Inquiry Process

27. Two Options; Districts Choose
Option A:
State Assigned SGPs
Option B:
Verify SLG Goal
This option does not require educators to set a Category 1 SLG goal
Student Growth Percentiles (SGPs) are used exclusively to determine the Category 1 SLG rating
This option requires educators to set Category 1 SLG goals using Smarter Balanced and rate their goals using the SLG scoring rubric
The SLG goal rating is compared with the educator’s median SGP rating to determine the Category 1 SLG rating
In Option A educators are provided with a median SGP which is used as a the SLG score for that goal. Best practice would include discussion of context, strategies and professional supports needed to attain your Category 1 goal.
In Option B, educators set and score a goal using Smarter Balanced and the state scoring rubric and then compare that score to their median SGP to determine the final score for the SLG.

28. What can districts do to prepare?
Study options with collaborative team and develop district policies related to SGPs
Create rosters this fall to verify in the spring
Practice calculating Median SGPs using growth data from ODE
We know that SGPs will be the method for incorporating Category 1 measures, it is just a question of when. All districts will need to select either Option A or Option B and have policies in place around teacher of record, procedure for roster verification and attendance.
Because districts will get student growth data from ODE later this month they can begin the process of creating class rosters and calcuating median SGPs. This will be a valuable exercise even if SGPs are not used until

29. Remaining questions?

30. Feedback Please! On an index card:
Do you have any feedback that could improve this presentation?

31. ODE Contacts Educator Effectiveness Team:
Tanya Frisendahl
Sarah Martin
Sarah Phillips
Brian Putnam