1. 2019 MCAS Reporting Call for Principals and Test Coordinators
September 20, 2019 Embargoed until 12:01 a.m., September 24

2. Logistics for This Session
Questions may be asked at any time using the Questions feature on the GoToWebinar screen.
Questions about a specific student should be sent by to
Some questions may be covered during the course of the presentation.
We will answer some questions at the end of this session.
After the session, we will questions and answers to participants.
This session is being recorded and will be available online in about one week at the MCAS Resource Center.

3. Michol Stapel, Associate Commissioner for Student Assessment
Presenters
Michol Stapel, Associate Commissioner for Student Assessment
Bob Lee, MCAS Chief Analyst
Scott Kelley, Data and Reporting Specialist
David Ragsdale, Test Security Specialist
Massachusetts Department of Elementary and Secondary Education

4. Today’s Call MCAS reporting highlights
Statewide results for all grades and subjects
Transition to next-generation grade 10 tests
Release schedule and additional resources

5. 2019 MCAS Reporting Highlights
Overall gains across grades 3–8 between 2017 and 2019
Grade 3 ELA: Meeting Expectations rose from 52% in 2018 to 56% in 2019
Essay scores rose by almost a point per essay
Grade 6 Mathematics: Meeting Expectations rose from 47% in 2018 to 51% in 2019
Open response scores up 7%
First math test in grades 3–8 to surpass an average scaled score of 500 and 50% Meeting Expectations

6. 2019 MCAS Reporting Highlights (continued)
Grade 10 ELA and Mathematics
First year of next-generation tests with new scaled scores & achievement levels
Graduation and scholarship eligibility standards and rules remain the same as the legacy standards for at least the classes of 2021 and 2022
87% of students passed the MCAS testing requirement in all three subjects on the first attempt (same as last year)
Transition to computer-based testing nearly complete
Only high school STE and retests were paper-based in 2019

7. 2019 MCAS Participation Rates by Subject and Mode
Grade
Participation Rates
ELA
Math
STE
Grade 3
99.4%
99.7% -
Grade 4
99.5%
99.6%
Grade 5
Grade 6
99.3%
Grade 7
99.2%
Grade 8
99.0%
99.1%
98.8%
Grades 3-8
Grade 10
98.4%
98.2%
Grade
Computer Based
Testing
Grade 3
97.3%
Grade 4
Grade 5
97.6%
Grade 6
96.7%
Grade 7
97.9%
Grade 8
97.8%
Grades 3-8
97.4%
Grade 10
(ELA & math only)

8. English Language Arts Achievement Results
2019 ELA by Grade
2017–2019 Change
Grade
Average Scaled Scores
Scaled Score Change, 2017 to 2019
2017
2018
2019
Grade 3
498.8
502.2
504.1
+5.3
Grade 4
499.2
501.8
+2.6
Grade 5
498.9
501.9
501.2
+2.3
Grade 6
499.4
501.0
501.3
+1.9
Grade 7
499.1
497.0
499.0
-0.1
Grade 8
499.8
+1.0
Grades 3-8
500.5
+2.2
Grade 10
506.2

9. Mathematics Achievement Results
2019 Mathematics by Grade
2017–2019 Change
Grade
Average Scaled Scores
Scaled Score Change, 2017 to 2019
2017
2018
2019
Grade 3
498.8
499.9
499.4
+0.6
Grade 4
498.0
497.9
499.2
+1.2
Grade 5
498.6
497.5
498.5
-0.1
Grade 6
500.8
+1.6
Grade 7
498.2
-0.4
Grade 8
499.6
499.0
-0.6
Grades 3-8
498.4
+0.4
Grade 10
505.1

10. Science and Technology/Engineering
May 2018
Science and Technology/Engineering
2019 Next-Generation Gr. 5 & 8
Legacy Grade 10
Grade 5
Grade 8
Exceeding Expectations 8
48%
46%
Meeting Expectations 40 38
Partially Meeting 39 41
Not Meeting Expectations 12 13
Avg. Scaled Score
498.9
498.2

11. Tips for reporting next-generation results at the group level
Use average scaled scores to compare groups
It is still appropriate to use percentages (such as percent Meeting or Exceeding Expectations), but average scaled scores are more precise when comparing groups or change over time.
Results can be averaged across grades 3-8, as shown on slides 7 and 8
Grade 10 results should not be averaged with lower grades.
For the state release, ESE defined improving as gaining 2.0 or more scaled score points, or more than 2% gains. Statistical significance varies depending on school size.
Student growth percentiles can be averaged across all grades

12. 2017 to 2019 School Level Scaled Score Changes in Grades 3-8
English Language Arts
Mathematics

13. Grades 3-8 Results by Race, 2017-2019

14. Changes in Scaled Scores by Race, 2017–2019

15. Grade 3 ELA Results – Essay Scores
Results by district
District results by poverty rate
99% of districts saw a reduction in zero-scored essays
The number of students receiving zeroes on one of their two essays dropped from 40,400 (58%) to 16,800 (25%)
Similar gains were seen in high poverty schools and low poverty schools

16. Transition to Next-Generation Grade 10 ELA and Math Tests
First administration of next-generation tests in 2019
Based on Massachusetts 2017 revised curriculum frameworks
Computer-based
Aligned to NAEP and evidence-based definitions of college & career readiness
Consistent with results established in 2017 for Grades 3-8
Legacy graduation and scholarship standards remain the same while students become familiar with the new tests
The Department used statistical techniques validated by expert educators to ensure that the knowledge, skills and abilities needed to pass and graduate with a Competency Determination remained similar for at least the next two classes.

17. Legacy Achievement Level Results (2015)

18. Grade 10 Transition: Legacy Compared to Grades 3–8 (2018)
May 2018
Grade 10 Transition: Legacy Compared to Grades 3–8 (2018)
ELA
Mathematics
Next-Generation
Legacy
Next-Generation
Legacy

19. % to College by ELA Grade 10 % to College by Math Grade 10
As most Proficient and many Needs Improvement students have attended college…
% to College by ELA Grade 10
% to College by Math Grade 10

20. % Graduating College by ELA % Graduating College by Math
. . . most of the Proficient and very few Needs Improvement students graduated from college within eight years.
% Graduating College by ELA
% Graduating College by Math
Meeting Expectations
Meeting Expectations
Proficient on Legacy
Proficient on Legacy

21. Standard Setting Needed anytime you introduce a new test
Establish standards for each achievement level; without standard setting, can’t say whether a student met expectations, only whether they answered an individual question right or wrong
Context for standard setting
Legacy MCAS needed to be updated to reflect expectations in revised learning standards, as well as for alignment with grades 3-8 tests
Legacy MCAS high school tests were designed to measure readiness to graduate, not readiness for college and career
Approximately 25% of MA public high school graduates who enrolled in public colleges or universities placed into remedial courses for Mathematics, 10% for reading and writing
In Mathematics 90% of students at 220 required remediation and 50% of students at 250

22. Standard Setting for Grades 5 & 8 STE and Grade 10 ELA & Math
August 5-7, 2019
75 educators (with an average of 18 years experience)
Four panels
Grade 10 ELA
Grade 10 Mathematics
Grade 5 Science and technology/engineering (STE)
Grade 8 STE
Massachusetts Department of Elementary and Secondary Education

23. Key Standard Setting Activities
Massachusetts educators worked in grade- and subject-specific groups, lead by facilitators, to make judgments that produced cut score recommendations by identifying the knowledge, skills, and abilities that students need to demonstrate to reach each achievement level
(Grade 10 only) Educators examined student results on the legacy and next-generation tests to verify the interim standard

24. Next-Generation MCAS Achievement Levels
Exceeding Expectations A student who performed at this level exceeded grade-level expectations by demonstrating mastery of the subject matter.
Meeting Expectations A student who performed at this level met grade-level expectations and is academically on track to succeed in the current grade in this subject.
Partially Meeting Expectations A student who performed at this level partially met grade-level expectations in this subject. The school, in consultation with the student's parent/guardian, should consider whether the student needs additional academic assistance to succeed in this subject.
Not Meeting Expectations A student who performed at this level did not meet grade-level expectations in this subject. The school, in consultation with the student's parent/guardian, should determine the coordinated academic assistance and/or additional instruction the student needs to succeed in this subject.

25. Achievement Levels and Cut Scores
Not Meeting
Expectations
Partially
Meeting Expectations
Meeting Expectations
Exceeding Expectations
Achievement
Partially Meeting Expectations
Cut Score
Meeting Expectations
Cut
Score
Exceeding Expectations
Cut
Score
Less
More

26. Educator Validation of Equipercentile Linking
For each cut score, educators compared:
Educators verified that the knowledge, skills, and abilities defined by each of these sets of results were equivalent for each cut score.
Student results on a set of items from the 2018 legacy test that is associated with that legacy cut score (located just below, at, or just above the cut score)
Student results on a set of items from the 2019 next-generation test that is associated with the statistically defined interim cut score (located just below, at, or just above the cut score) to

27. Linking Model for Locating Cuts (ELA example)
13% Exceeding Expectations
48% Meeting Expectations
% of students
31% Partially Meeting Expectations
8% Not Meeting Expectations

28. Statistical Linking Model (ELA)
13% Exceeding Expectations
48% Meeting Expectations
% of students
31% Partially Meeting Expectations
8% Not Meeting Expectations

29. Zooming In on the Pass/Fail Line (ELA)
472
Partially Meeting Expectations
240
470
EPP
required
% of students
5% Passing & Not Meeting
Expectations
455
220
3% Not Passing & Not Meeting
Expectations
200
440

30. Interim CD Standard for the Classes of 2021 and 2022
Legacy PASSING
but requires an
Educational Proficiency
Plan (EPP) in ELA/math only
Legacy
PASSING and met the MCAS
graduation requirement
Next Gen PASSING
EPP
Next Gen PASSING and met the MCAS
ELA
220–238
240+
455–471
472+
Mathematics
469–485
486+
STE
220
220+
TBD - Summer 2020

31. Interim Adams and Koplik Cut Scores for the Classes of 2021 and 2022
Legacy Proficient
Legacy
Advanced
Next-Gen Proficient
Equivalent
Next-Gen Advanced Equivalent
ELA
Math
STE
N/A

32. MCAS Passing Rates Remain Virtually Identical to Previous Years
Subgroup
Class of 2021
Class of 2020a
Class of 2019a
ELA
Math
ELA and Math
STE
All Three Tests
All Students 95 90 89 93 87 88
Gender
   Female 96 92 91 94
   Male 85 86
   Non-Binary
N/A
Race/Ethnicity
   African American/Black 82 80 77 78
   Asian 97
   Native Hawaiian or Pacific Islander 84
   Hispanic or Latino 83 73 72 71
   Multi-Race, Non-Hispanic or Latino
   Amer. Ind. or Alaskan Nat.
   White
Student Status
   High Needs 74
   Non-Disabled
   Students with Disabilities 69 67 63 64
   English Learner (EL) 68 61 49 62 43 44 40
   Former EL 98
   EL and Former EL 79 66 59 54
   Economically Disadvantaged 75
aTo provide comparable data, results for the Classes of 2020 and 2019 are based on MCAS tests through the spring 2018 and spring 2017 administrations, respectively.
Number and percentage of students earning a full CD after their first grade 10 attempt
CD Requirement
Number
Class of after grade 10
Class of after grade 10
Earned CD
54,533
74%
ELA and Mathematics Proficient or Higher
54,781
75%
ELA Proficient or Higher
65,179
89%
88%
Mathematics Proficient or Higher
55,664
76%
STE Needs Improvement or Higher
67,787
93%

33. Available Resources Student Work Samples
Samples of student work for each score point for ELA, math and STE; ELA essays include annotations for each score point.
MCAS Sample Student Work & Scoring
Student Essay Responses
One essay at each grade in grades 4–8; one constructed response item in grade 3; both essays released at grade 10
PearsonAccess Next
mcas.pearsonaccessnext.com
Released Items
Approximately 50% of items released at grades 3–8; 100% released at grade 10 ELA and math
MCAS Resource Center
mcas.pearsonsupport.com/released-items/
Edwin Analytics
Official embargoed MCAS interactive reports available on Sept. 19; student Competency Determination status available in PE618 Student CD Roster
DESE Security Portal
gateway.edu.state.ma.us
District Analysis and Report Tools (DART)
Success after high school report
Analysis of school and district postsecondary outcomes. See what proportion of the students from your school attended college, required remediation and graduated.
DESE Data Tools

34. Upcoming Release Dates (Tentative)
September 18
Communication about special considerations due to the voided essay prompt and the John and Abigail Adams Scholarship released
September 19
Official embargoed school and district results available in Edwin Analytics
September 23
Embargoed data given to members of the press and superintendents. Messaging bullet points released via On the Desktop communication.
September 24
Public release of official MCAS results on Profiles
September 30
District offices receive printed Parent/Guardian Reports
Week of Sept. 30
Parent/Guardian 5-minute video about Grade 10 reports to be released alongside FAQs, templates and translations available

35. (Use the “Questions” feature to ask questions.)
Questions & Answers
(Use the “Questions” feature to ask questions.)

36. Today: Complete the evaluation form.
Next Steps
Today: Complete the evaluation form.
Responses associated with the name and address used to log in
your input to if you have problems accessing or completing the form.
Later this week/early next week:
Receive an with the Q&A from this session
Recording will be available
Massachusetts Department of Elementary and Secondary Education

37. The Office of Student Assessment Services
THANK YOU
The Office of Student Assessment Services
75 Pleasant Street, Malden, MA 02148