1. NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association November 22, 2011

2. The Challenge Design a school accountability system that: Sets a high proficiency standard (where proficiency is based on career and college ready standards) AND Rewards schools for achieving growth with students, regardless of starting point Moves Michigan toward a higher level of preparation for career and college Fair and equitably applied

3. WHERE ARE WE NOW? Assessing Michigan’s Current Situation

4. College going rates Statewide: 71% of 2008-2009 graduates enrolled in an IHE 73% of those who enroll earn at least one year’s worth of credits Gives a total of 52% of 2008-09 graduates who earned at least one year’s worth of credits By individual school: Median = 63% 25 th percentile: 40% 75 th percentile: 75%

5. 0.5 1 1.5 2 2.5 Density 0.2.4.6.81 Percentage of Graduates Enrolled at an IHE

8. Relationship between new cut scores and college going 0 20 40 60 80 100 Percent Enrolled in an IHE 020406080100 2008 MME Math Percent Proficient (based on new cut scores)

9. Takeaways Michigan students are going to college Even if students are not proficient on new cut scores on the MME, they are enrolling in college. Question: Will those students be successful? Will they pursue challenging majors?

10. Achievement Gap Since 2001, schools have been held accountable on overall student performance… AND the performance of the nine traditional subgroups Put the focus on achievement of all students, as defined by demographic characteristics Caveat: IF you had a sufficient number of students!

11. Economically disadvantaged gaps: Math

12. Ethnicity gaps: Math

13. Economically disadvantaged gaps: Reading

14. Ethnicity gaps: Reading

15. Career and College Readiness in Our Schools: Math 50 th percentile 95 th percentile

16. CCR: Reading 50 th percentile95 th percentile

17. Tension: Is it really important that our students be career and college ready? Evidence points to the importance of higher education. Competitive job market, especially in states like Michigan. Fast-growing occupations (health care, technology) require higher education/specific training Educational inflation

18. NEXT GENERATION ACCOUNTABILITY Charting a New Path

19. Necessary Components Focused consequences and interventions for schools most in need Achievement gap Differentiated accountability = differentiated interventions Fair and equitable Ambitious AND achievable goals

20. Focused Consequences and Interventions Priority Schools Lowest 5% of the Top to Bottom list Priority schools = PLA Schools Aligns federal and state accountability Priority schools must enter a three year cycle of school improvement, with the most highly targeted interventions

21. PLA Schools: Anecdotal Evidence Two “cohorts” of PLA schools: 2010 and 2011. 2010 schools: first year of implementation 2011 schools: planning 2010 schools: About half experienced an increase in percent proficient and increased their improvement rate More meaningful data in another 1-2 years

22. PLA Schools: Anecdotal Evidence What are they saying? ERA Unit doing PLA data visits Being named a PLA school was extremely difficult However, it has fundamentally altered the way the schools are approaching achievement Pushing a fundamental redesign Impetus to address crucial issues Innovative strategies (i.e. flipped learning)

23. Achievement Gap as Central Focus Achievement gaps have not closed to the extent that we need Proposal: Focus on the bottom 30% of students, regardless of demographic, not the traditional subgroups Puts the attention firmly on the lowest achieving students By improving that group, increase school’s overall achievement, and improvement rate

24. Achievement Gap Pros All schools have a subgroup At least 700 schools have no subgroup under AYP traditional subgroups Unmask low performance in high performing subgroups Asks that all schools consider their lowest performing students Schools cannot mask low-performance with overall high performance

25. Achievement Gap Cons: Concern that we will lose focus on demographic subgroups. In the lowest 30% subgroup—approximately 70% of that group are also a member of one or more traditional subgroups. High-achieving schools do not like it People think that “lowering the ceiling on our highest achieving students” will help the schools

26. Achievement Gap: Focus Schools Need to identify the schools with the largest achievement gaps. Using the bottom 30% subgroup, would rank the bottom 10% in terms of largest gap. Using traditional subgroups in a ranking (normative) setting is complicated: Not all schools have a subgroup Comparing schools with the same subgroup Unfairly focuses on students with disabilities Still allows for “masking”

27. Achievement Gap: Our Belief If Michigan is serious about raising the achievement of ALL students, then the bottom 30% is the correct way to go. Distributing accountability to traditionally high achieving schools and asking them to achieve those same results with all students is appropriate. Michigan cannot leave students behind any longer.

28. Differentiated Accountability For All Schools Priority schools = 5% and Focus Schools = 10%-- so what about the other 85%? Need a more nuanced system than pass/fail AYP. Need to integrate performance for all students, bottom 30%, and all subjects (not just reading and mathematics)

29. Use this system to set a proficiency goal with improvement Proficiency target = AMO Set for each school as the increase in percent proficient necessary for that school to reach the overall target proficiency Improvement target If school does not meet proficiency target, can meet an improvement target Set as the increase in percent proficient demonstrated by a high-improvement school in the base year

30. Example with Data End proficiency target: 85% School is at 10% proficient now Need to improve 75% in 10 years, or 7.5% per year. Proficiency target in year 1: 17.5% proficient If does not meet it, must have improved by 3.5% (which is improvement rate for school at the 90 th percentile in base year)

31. Bottom 30% Subgroup as Accountable Subgroup Only one “accountable” subgroup now (still report on nine traditional subgroups) Need to meet a proficiency target for the bottom 30% subgroup (unlikely…) OR the improvement target

32. Notes on this system Offers differentiated AMOs by school Keeps a clear proficiency target in the system Proficiency target is actually an improvement target as well Many of the increases demanded of schools will be greater than we have historically seen, so need the improvement (safe harbor) target

33. Questions What is an “ambitious and attainable” end goal? 100%? 85%? 70%? How do you determine this? Should meeting the target based on improvement be equivalent to meeting it based on straight proficiency? Should we reset each year?

34. Participation Necessary to keep a firm participation target in the system Schools will begin to “game” on who they assess if no clear participation target

35. “Green” School School Name: ABC Schools Reward Proficiency ImprovementParticipationOverall MathAll StudentsYes--Yes Bottom 30%NoYes ReadingAll StudentsYes--Yes Bottom 30%Yes-- WritingAll StudentsYes--Yes Bottom 30%NoYes ScienceAll StudentsYes--Yes Bottom 30%NoYes Social StudiesAll StudentsYes--Yes Bottom 30%Yes-- Rate Improvement Overall Graduation Rate Yes-- Overall Compliance Educator Evaluations

36. “Yellow” School School Name: XYZ Schools Proficiency ImprovementParticipationOverall MathAll StudentsNoYes Bottom 30%NoYes ReadingAll StudentsYes--Yes Bottom 30%Yes-- WritingAll StudentsNoYes Bottom 30%NoYes ScienceAll StudentsYes--Yes Bottom 30%NoYes Social StudiesAll StudentsYes--Yes Bottom 30%Yes-- Rate Improvement Overall Graduation Rate Yes-- Overall Compliance Educator Evaluations

37. “Red” School School Name: MNO Schools Priority Proficiency ImprovementParticipationOverall MathAll StudentsNoYes Bottom 30%No ReadingAll StudentsNo Yes Bottom 30%No WritingAll StudentsNoYesNo Bottom 30%NoYes ScienceAll StudentsYes--Yes Bottom 30%No Social StudiesAll StudentsNo Yes Bottom 30%No Rate Improvement Overall Graduation Rate NoYes Overall Compliance Educator Evaluations

38. Rules for Colors Need to be green on all indicators This makes “green” a more rare indicator; it means that there are no areas of concern To be red, need to be red on all five academic indicators; makes it a more rare indicator Yellow—largest category—can have some red, some green; is indicative of “intervention” needed; use colors within to target Final color is not the key determiner for consequences; priority/focus status is more critical

39. Questions? How to determine the final colors? Balancing public desire for “one” rating with internal knowledge that “one” rating is difficult. Other indicators that should be included?

40. Contact Information Venessa A. Keesler, Ph.D. Evaluation, Research and Accountability Office of Psychometrics, Accountability, Research and Evaluation (OPARE) keeslerv@michigan.gov