1. State Higher Education Assessment Policies: State Higher Education Assessment Policies: Findings from Case Studies Thomas E. Perorazio John J.K. Cole The National Center for Postsecondary Improvement Project 5.1 The University of Michigan Association for Institutional Research 42 nd Annual Forum Toronto, Ontario, Canada

2. Session Goals Discuss Policy Process for Assessment Relate Important State Experiences Share Lessons Learned from Cases Relevant to Institutions

3. The National Center for Postsecondary Improvement Main Project Web Site http://www.stanford.edu/group/ncpi/ University of Michigan - Project Five http://www.umich.edu/~ncpi/

4. NCPI Project Area 5.1 Examine State & Regional Policies What Policies Are in Existence? Impact on Institutional Assessment Practices Impact on Teaching & Learning Utilize Policy Process Framework Analyze Relationships Among Levels State Government and Regional Accreditation Association Policies of Assessment for Student Learning

5. Overview of Project 5.1 Research Benchmarking Assessment Documented Existing Policies SHEAQ Survey Of SHEEO Administrators Literature Review Case Studies

6. Case Study Research Examine State Policies in Detail Explore Critical Issues with State Officials Trace Evolution and Development Learn about Policy’s Impact on Institutions Infer Lessons about the Policy Process Purposes

7. States Selected for Study New York(MSACS) South Carolina(SACS) Washington(NWASC) Missouri(NCA/HLC) Florida(SACS) (with Accreditation Region)

8. Policy Context Historical, Political, Social, & Economic Factors Existing Policies for Accountability, Efficiency, etc. Political Climate in the State for Higher Education Previous Political Action RE: Assessment Governance Structure for Higher Education Relations & Communications between Government, SHEEO, & Institutions Budgetary & Financial Issues for Higher Education

9. Policy Process Framework Five Stages 1. Problem Formation  Recognizing the Need for a State-level Assessment Policy 2. Policy Formulation  Development of Proposed Courses of Action 3. Policy Adoption  Development of Support for a Specific Proposal 4. Policy Implementation  Application of the Policy to the Problem 5. Policy Evaluation  Attempt to Determine the Policy’s Effectiveness

10. Policy Context Historical, Political, Social, & Economic Factors Formation Formulation Adoption Implementation Evaluation History Purpose Design Leadership Links Outcomes Inputs Objectives Processes Outcomes Evaluation Conclusions RE: Policy Process, Lessons for Policy Actors Understanding Relationships Among Policy Levels Policy Process SynthesisCase Analysis

11. Problem Formation & Policy Origination SHEEO Seeks to Focus Goals of Institutions on State Priorities Statewide Planning Task Forces Priorities of Quality, Effectiveness, Prestige, & Efficiency Gubernatorial/Legislative Interest in Performance Desires Information on Results/Success Task Force Studies Data Generation & Collection Measures Concern about Public Perceptions of Prestige/Quality

12. Problem Formation (2) Institutional Actions Initiatives to Engage in Assessment Procedures to Improve Program Quality Programs to Enhance Learning Public Opinion Critical for Political Will Change in Political Power

13. Policy Formulation Quality Assurance Institutional Data Generation/Collection Performance Indicators/Assessment Reports Make Information Publicly Available Increase Information to Policymakers Accountability State Planning & Coordination Institutions Meet State Goals/Targets Centralized Approach to Data Analysis Results Tied to Budgetary Decisions Options for Consideration

14. Policy Formulation (2) Institutional Improvement Management & Effectiveness Link Measures to Accreditation Standards Institutional Self-Evaluation Differentiated by Sector Bring Improvement Through Quality Meet Both State and Institutional Goals

15. Policy Adoption Legislative Action Mandate Authorize SHEEO & Monitor Work with Institutions for Revision SHEEO Authority Originator of Policy Monitor, Collector, & Distributor of Info Mediator between State, Institutions, & Public Four General Methods

16. Policy Adoption (2) Task Forces/Blue Ribbon Committees Authority: SHEEO or Legislature Business Leaders, Institutional Presidents Conduct Study/ Make Recommendations State/System Planning Process Produces Actionable Objectives Assessment In Service of Plan Goals

17. Policy Implementation--Mechanisms Reporting Institutional Statistics New York, Washington, & Florida Performance Funding/Reporting South Carolina Missouri -- FFR Florida System Goals & Institutional Improvement Missouri, Washington Accountability Florida, South Carolina, Missouri

18. Policy Implementation (2) Decentralized State Sets Broad Parameters for Performance Institutions Develop Effectiveness Plans Measurement Defined by Institutions Institutions Report Results up to State State Makes Decisions on Aggregated Data New York & Washington

19. Policy Implementation (3) Centrally-Guided Prescribed State & Institutional Goals Performance Standards Less Variable Central Data Collection & Analysis Findings Utilized in Budget Decisions South Carolina & Missouri

20. Policy Implementation (4) Combination Approach State Expectations & Performance Guidelines Institutional Variability for Compliance Institutional Activity for Internal Improvement Florida

21. Data Collection Centralized Databases v. Institutional Data Levels of Aggregation Instruments/Testing Commonality Associated with Centralization Institutional Reporting To SHEEO, Legislature, Public Data is Reported Up the System Emphasis on Data & Data Systems

22. Data Usage for Decision Making Rewards -- For Meeting Targets MO & SC Incentives -- To Achieve State Goals FL Public Knowledge-- Consumer Information WA & NY

23. Links to Teaching/Learning Improvements Making Assessment Institution-centered Public Accountability Institutions Share Data on Learning Revisiting Indicators Regularly Close Information Loop Focused Goals at Different Inst. Levels

24. Outcomes of Policy Institutional Resistance Disparate Effects “Negative Improvement” Excessive/Burdensome Requirements Indicators Not Useful for Management Institutional Cooperation Focus on Improvement Trust between SHEEO & State Partnership to Develop System

25. Policy Evaluation Revisiting &Adjustment MO & SC Implementation NY & WA Implementation & Evaluation FL

26. Lessons Learned Assessment Must Be Incorporated Into Institutional Management Successful Policies Developed in Consultation with Institutions Culture of Institutions Can be Changed If The Process Contributes to Mgt

27. Lessons Learned (2) Institutions Must Also Be Willing to Form Working Relationships with State Officials Stakeholders at All Levels Must Be Engaged with Assessment Involving Too Many Stakeholders in Development Bogs Process Down

28. Lessons Learned (3) Sustained Commitment of Leadership Required Political Will for Success Required Policy Process Can Be As, If Not More Important Than Its Results

29. Successful Policies Have a Clear & Focused Purpose Differentiate by Sector/Mission Emphasize Institutional Improvement Embrace Simpler, Rather than Complex, Indicator/Reporting Mechanisms Incorporate Priorities of Multiple Stakeholders Provide Useful Data for Decision Makers

30. Important Considerations Context for Assessment Shapes Process Policies Result In Improved Data Systems Process Forces Articulation of Principles New Policies May Not Replace Old Ones Policy Must Be Useful to Instituions