2. Why Measure Performance with Scale Scores
Why Measure Performance with Scale Scores? Dominic Zarecki, Senior Data Analyst
Date:   Dec
Time:  4:30-5:00 PM Room: Carr 

3. Agenda “Distance from Met” is: Nuanced Publicly available
Fair to all grades
Fair to all schools

4. The Foundation: “Distance from Met”
Improved from 24 below to 22 above Met
Declined from 44 below to 78 below Met
Example student score report created by CDE, found here:
Distance from Met measures how many scale score points a student is above or below the Level 3 (i.e. Met) cut point. This allows us to standardize scores across grades in a way that is more nuanced than the four SBAC levels. On the far right, we see that Sophie is 78 points Met in Math. While this is Level 1, it is right near the top of Level 1. (If you look really closely, you’ll notice that the standard error indicates that Sophia could very well score in Level 2 if she took the same test again.
Change in Distance from Met = to what extent is a student outpacing the academic treadmill?
The example on the right is particularly helpful. We see that Sophia’s scale score improved slightly, from 2392 to But the academic treadmill moved up even more; if Sophia had made average growth, she would have grown by even more scale score points. We can see this clearly by looking at her change in distance from Met: she dropped from 44 below to 78 below.
Distance from Met also allows us to see improvement within levels. This is especially helpful with level 1, where a student can make a lot of progress but still not yet be at level 2.

5. How do we calculate “Distance from Met” for schools?
A three-step process tells us how far the average student in a school is from the Met cut point:
Calculate the Gap: school average minus Met cut point for each grade and subject
Weight by student: Gap multiplied by the percent of students in that grade and subject
Create one number: Add the values from all grades, then average the ELA and Math scores
For each subject, we calculate the weighted average of DFM scores across all grades in a school. We weight by the number of students with valid test scores. This produces the same average DFM for each subject that we would obtain with student-level data. We then take the average of the ELA and Math DFM scores to obtain one overall DFM score for each school. The average ranked school in had a DFM of 23 points below Met. The standard deviation was 46 points, and DFM ranged from 215 points below Met to 151 points above Met.
To make State Ranks, we divide schools into deciles based on their DFM.
For the ranges, see:
The research file is available here:

6. Surprisingly even across grades!
ELA DFM
Math DFM
Overall DFM 3
-18
-11
-15 4
-19
-25
-22 5 -6
-43 6
-12
-28 7
-10
-42
-26 8 -8
-45 11 17
-60
Given that the scale is over 600 points wide, a difference of up to 13 points across grades is extremely small (2% of range).
The primary assumption this makes is that DFM is an equally difficult standard across grades. The table below shows the average DFM of students statewide by subject and overall. We see that while DFM for each subject varies significantly by grade, overall DFM is extremely uniform across grade levels. With the exception of third grade, average DFM scores only range from -22 to This means that DFM measures schools fairly regardless of the grades they serve.

7. DFM and % Met are 0.97 correlated…
39% Met/Exceeded (57th percentile)
-26 DFM
(59th percentile)
Distribution of 2015 CAASPP Scores
State Mean
Met Cut Point
-33 DFM

8. … but can diverge dramatically
38% Met/Exceeded (47th percentile)
-17 DFM
(60th percentile)
Distribution of 2016 CAASPP Scores
State Mean
Met Cut Point
-31 DFM

9. Thank you
Dominic Zarecki, Senior Data Analyst