1. Accountability & Assistance Advisory Council Meeting
December 13, 2017
Best Western Royal Plaza Hotel
Marlborough, MA

2. Welcome & introductions01
Welcome & introductions
Agenda 02
Review of discussion norms 03
2017 assessment & accountability results 04
Next-generation accountability system
AGENDA 05
Secondary school redesign strategies 06
AAAC update at Board meeting 07
Next steps & closing

3. 01 Welcome & introductions

4. Welcome & introductions
Council co-chairs: Meg Mayo-Brown & Valerie Annear
New council member: Julia Mejia, Collaborative Parent Leadership Action Network (CPLAN)

5. 02 Review of discussion norms

6. Norms & protocols
Promote risk-taking by not assigning specific comments to individual members in meeting summaries & minutes
Maintain respect before, during, & after meetings
Seek clarification & check understanding to avoid mistakenly attributing ideas to an individual or organization represented on the council
Keep improvements in student learning at the core of the discussions. Students should drive the conversation
Stay engaged in the issues
Actively address implicit biases & instill cultural proficiency in discussions
Encourage discussion from all voices of members. Be additive, not repetitive
Let members know if pre-reading or other pre-meeting assignments require more than 30 minutes of preparation time
Make it fun!

7. 03 Summary of 2017 assessment & accountability reporting

8. Next-Generation MCAS 1. Higher, but achievable, expectations
New achievement levels (Exceeding Expectations, Meeting Expectations, Partially Meeting Expectations, Not Meeting Expectations)
New scale ( )
2. Educator participation & involvement
Framework revisions, test item review, standard setting
3. Coherence across grade spans
Expectations are similar across grades
4. Aligned to readiness at the next level
Signaling preparedness for college & career
5. Next-Generation MCAS results cannot be directly compared to legacy MCAS results
No trend data reported

9. 2017 Next-Generation MCAS state results (grades 3-8)

10. Next-Generation MCAS resources
For parents
For educators
Parent Guide to the MCAS (available in several languages
Annotated parent/guardian reports
Frequently Asked Questions (FAQs)
What are the Achievement Level Descriptors?
Item descriptions for grades 3-8 ELA & mathematics
Parent/guardian report templates & translations
Links to released items, answer keys, & item descriptions
Information about the Next-Generation achievement levels
Sample letter for districts to families
Understanding the Next-Generation MCAS (presentation)
Recording of October 16 reporting session for grades 3-8 ELA & mathematics results
Board of Elementary & Secondary Education October meeting materials

11. Percent of Students Scoring Proficient or Higher
2017 legacy MCAS results
Percent of Students Scoring Proficient or Higher
English Language Arts
Mathematics
Science & Tech/Eng.
2017
2016
Change
Grade 5
46%
47%
-1%
Grade 8
40%
41%
Grade 10
91% 0%
79%
78%
+1%
74%
73%

12. District Totals by Level
2016 & 2017 District levels
District Totals by Level
2016
2017 # %
Level 5 3 1%
Level 4 7 2% --
Level 3 62
16% 13 3%
Level 2
233
61% 20 5%
Level 1 80
21% 16 4%
No level
344
87%
Total
385
100%
396
Insufficient Data1 22 8
1 Districts with insufficient data to be eligible for a level are those ending in grade PK, K, 1, or 2, very small districts, & districts without four full years of data.

13. 2016 & 2017 School levels School Totals by Level 2016 2017 # % Level 54 0%
Level 4 33 2% 31
Level 3
265
16% 81 5%
Level 2
794
49%
129 8%
Level 1
525
32% 70 4%
No level --
1324
81%
Total
1621
100%
1639
Insufficient Data1
233
211
1 Schools & single-school districts with insufficient data to be eligible for a level are schools ending in grade PK, K, 1, or 2, very small schools, & schools without four full years of data.

14. Next-Generation MCAS schools by accountability level# %
Next-Generation MCAS schools
1385
100.0
Insufficient data 10
0.7
No level
1324
95.6
Level 3 24
1.7
Very low assessment participation (less than 90%) 23
Persistently low graduation rates1 3
Level 4 22
1.6
Level 5 4
0.3
1“Persistently low” is defined as a 2016 four-year cohort graduation of less than 67 percent & 2015, 2014, & 2013 five-year cohort graduation rates of less than 70 percent.

15. Legacy MCAS schools by accountability level# %
Legacy MCAS schools (high schools)
298
100.0
Insufficient data 33
11.1
Level 1 70
23.5
Level 2
129
43.3
Low assessment participation (90% - 95%) only 6
Level 3 57
19.1
Very low assessment participation (less than 90%) only 9
Persistently low graduation rates only 3
Level 4
3.0

16. 04 Next-generation accountability system

17. Accountability system design components
Which indicators will be included?
How will the system incorporate both relative (school percentile) & criterion-referenced (targets) components?
What subgroups will drive an accountability determination versus just having data reported?
How will schools be considered to be meeting targets?
How will schools be categorized?

18. Accountability indicators
Measure(s)
Achievement
ELA, math, & science achievement values (based on scaled score)
Student Growth
Student growth percentile
High School Completion
Four-year cohort graduation rate
Extended engagement rate
Annual dropout rate
English Language Proficiency
Progress made by students towards attaining English language proficiency
Additional Indicators
Chronic absenteeism (all schools)
Percentage of students passing all grade 9 courses (high schools)
Percentage of students completing advanced coursework (high schools)

19. Relative component
Accountability percentile calculated using all available indicators for a school
Used to identify the lowest 10 percent of schools, including schools in need of comprehensive support & intervention (CSI)
Same percentile calculation used at the group level to identify low- performing subgroups (schools in need of targeted support & intervention (TSI))

20. Relative component
Comparisons will be made separately for schools serving grades 3-8 & for schools where the only tested grade is grade 10
School-type comparisons no longer necessary (e.g., elementary school compared to only other elementary schools)
Separate high school comparison category is transitional & will not be necessary once all schools are administering Next-Generation MCAS tests
Early simulations suggest some change from previous percentile rankings
Based on one year of data only
Change in comparison group (“universe”)

21. Criterion-referenced component
No longer a focus on “gap-closing”… promoting the idea of “raising the floor”
Gap-closing can occur as a result of a decline in performance by the high- performing group
In addition to meeting targets for the school as a whole, the performance of the lowest performing 25 percent of students in each school will be measured
Every school has a lowest 25 percent of performers
Identified from cohort of students who were enrolled in the school for more than 1 year
Schools will know who these students are

22. Criterion-referenced component
We will continue to report on the high needs subgroup, but will no longer use this group’s data to determine whether a school is meeting targets
Not all schools have a high needs group
Not all high needs groups are created equal

23. Criterion-referenced component
Targets set for each accountability indicator, for the school as a whole & for the lowest performing 25 percent of students in each school
Points assigned based on progress toward target for each indicator, for both the aggregate & the lowest performing 25 percent of students
Declined
No change
Improved
Met target
Exceeded target 1 2 3 4

24. Criterion-referenced component
Example: Non-high school (weighting to be determined)
Indicator
Points assigned
All students
Lowest performing students
ELA scaled score 3 2
Math scaled score
Science achievement 1
ELA SGP 4
Math SGP
EL progress
Chronic absenteeism
Total 19 21
Combined total points (56 possible) 40
Percentage of possible points
71%
0 = Declined · 1 = No change · 2 = Improved · 3 = Met target · 4 = Exceeded target

25. Categorization of schools
Schools will no longer be placed in a vertical hierarchy of levels 1-5
Number of schools that will be placed into a category based upon a relative standing will be cut in half from previous system
Approximately 90 percent of schools could be categorized based on their performance against targets
Most schools will have 50 percent of its categorization based on students that have been in the school for at least 2 years
Category labels are primarily tied to the level of required assistance or intervention
Stronger emphasis on schools commended for success

26. Categorization of schools
Schools without required assistance or intervention
(approx. 85%)
Schools requiring assistance or intervention (approx. 15%)
Schools of recognition
Schools demonstrating high achievement, significant improvement, or high growth
Meeting
targets
Criterion-referenced target percentage
75-100
Partially meeting targets
50-74
Not meeting targets
0-49
Focused/targeted support
Non-comprehensive schools with percentiles 1-10
Schools with low graduation rate
Schools with low performing subgroups
Schools with low participation
Broad/
comprehensive support
Underperforming schools
Chronically underperforming schools
Note: Category names not finalized

27. Overall system design
Schools are placed into an accountability category based on relative & criterion referenced components (e.g., school percentile & performance against targets for all students & the lowest performing 25 percent of students)
Adjustment made for low participation
Schools commended for high achievement, growth, etc. could be identified from among any of the categories within the “schools without required assistance or intervention” bucket

28. Other considerations
Board of Elementary & Secondary Education has not yet discussed weighting of indicators
High schools
2018 assessment transition
Middle/high & K-12 schools
Administering both legacy & Next-Generation MCAS tests
District accountability
Will not be based on designation of lowest performing school
ESE will redesign district & school report cards in 2018
Will include measures of performance/opportunity beyond assessment & accountability results

29. Next steps Upcoming discussions December 4 – Secretary Peyser
December 4 – ESE Roundtable
December 11 – MASS
December 13 – AAAC
December 19 – BESE (system framework)
January 12 – USN
January 17 – SAC
January 23 – BESE (weighting of indicators)

30. 05 Secondary school redesign strategies

31. Updates from the State System of Support (SSoS)
AAAC, December 13, 2017
Ventura Rodriguez, Associate Commissioner for SSoS
Andrea Condit, District Targeted Assistance Liaison

32. History of ‘Big Bets’

33. SSoS focus area in development
Secondary Redesign - A marathon, not a sprint
Office of College, Career & Technical Education (CCTE):
Shailah Stewart, Nyal Fuentes, Lisa Harney
State System of Support (SSOS):
Andrea Condit, Ventura Rodriguez, Erica Champagne, Michael Seymour, David Parker, David Troughton (DSAC)

34. From 2010-2013 DESE designated 48 Level 4 schools
Context
From DESE designated 48 Level 4 schools
Of those, 17 were secondary schools (grades 6-12) representing:
35% of the originally identified schools
20% of all exited schools (4 MS, 1 HS)
67% of current Level 4 schools are secondary (9 MS, 9 HS & 2 MS/HS)
The lowest 10% of all MA secondary schools include almost 44,000 students (15,600 MS & 28,000 HS)
What are our current results?

35. DESE/DHE shared definition of college & career readiness
College & career readiness means a student has the requisite knowledge, skills & experiences in the academic, workplace readiness, & personal/social domains
to successfully navigate to completion an economically viable career pathway, participating in responsible citizenship.
What are we striving to achieve?

36. Key questions guiding the secondary redesign workgroup
What are the key ingredients needed to implement ESE’s vision of what it means to be a high school graduate in our lowest performing secondary schools (grades 6- 12)?
What do these schools need in order to improve & sustain outcomes for all students, so all students graduate ready for college & career?
What can DESE do to refine and/or augment our work to contribute to their success?
To what extent do we need to double-down on the basics vs. introducing new ideas & approaches?

37. DESE initiatives impacting low-performing secondary schools
Turnaround Practices in High Schools **NEW**
High School Redesign in Holyoke & Lawrence
Springfield Empowerment Zone (Commerce)
Turnaround High School PLC (with Barr Foundation) **NEW**
Level 3 & 4 Turnaround High School Network **NEW**
Northeast DSAC High School Network
CCSSO Cross-State Initiatives for Low-Performing High Schools (with Johns Hopkins)
New England Secondary Schools Consortium (NESSC) **NEW**
High Quality College & Career Pathways/New Skills for Youth
Connecting Activities/Work-based Learning/CDE
America Counts Initiative
Dropout Re-Engagement Workgroup
Data: EWIS, Postsecondary DART, ABC’s
MyCAP & Advisory
MassCore
CVTE
Middle Grades Math
Additional strategies

38. Key actions for the workgroup
Develop a deeper understanding of the strengths & challenges of these schools
Promote alignment & coherence of DESE’s multiple efforts that impact secondary schools in turnaround
Identify & support implementation of the unique elements needed to effectively implement DESE’s K-12 strategies in very low-performing secondary schools
Identify what else is out there to support the unique needs of struggling secondary schools?

39. Discussion questions
What feedback do you have about how we are organizing this work?
Are we guided by the right key questions?
Are we focused on the right key actions?
Does the workgroup have the right composition of skills & perspectives to lead this work?
What do our lowest performing secondary schools need in order to improve & sustain outcomes for all students, so all students graduate ready for college & career?

40. Lesson Learned in MA High School Turnaround
Field Guide image

41. 06 AAAC update at Board of Elementary & Secondary Education meeting

42. Update to the Board – December 19, 2017
Valerie Annear to provide a brief update to Board members on recent work of the Council
Massachusetts’ Every Student Succeeds Act (ESSA) state plan in
Changes to state accountability regulations
Assistance provided by the Department …

43. 07 Next steps
Next steps 07