Marvin W. Peterson & Thomas E. Perorazio The National Center for Postsecondary Improvement, Project 5.2 The University of Michigan Ross A. Griffith, Wake Forest University American Association for Higher Education Assessment Conference Denver, CO - June 25, 2001 Institutional Strategies for Integrating Student Assessment with Academic Planning and Management

Session Purposes u Discuss Initial Findings from the Case Study Phase of NCPI Research Project on Institutional Support for Student Assessment u Provide Detailed Examples of Two Institutions That Have Integrated Assessment Into their Academic Management Processes u Outline the Lessons Learned Applicable to Assessment Practitioners

The National Center for Postsecondary Improvement Main Project Web Site University of Michigan - Project Five

NCPI Project Area 5: Student Learning and Assessment 5.3 Academic Programs and Students:  Educational Reform and Innovation in Teaching, Learning, and Assessment Practices  Metadata Project: Review of National Student, Faculty, and Institutional Data Resources 5.1State Policies and Regional Accreditation Practices of Assessment for Student Learning 5.2 Institutional Support for Enhancing Student Assessment and Performance

Project 5.2’s Purpose To Examine How Postsecondary Institutions Support and Promote the Uses and Impacts of Student Assessment Information for Institutional Improvement

Overview of Project 5.2 Phase One Review & Synthesis of the Literature Developed a Conceptual Framework of Institutional Support Phase Two National Survey of Institutions Comprehensive Inventory Based on Conceptual Framework 1394 Usable Responses % Phase Three Case Studies of Institutions with Comprehensive Student Assessment Efforts

Institutional Context Assessment Culture and Climate Institutional Approach to Student Assessment Institution-wide Support for Student Assessment Assessment Management Practices & Policies External Influences on Student Assessment Uses & Impacts of Student Assessment Phase I: Conceptual Framework of Organizational and Administrative Support for Student Assessment

Phase II: National Inventory Research Questions u What Approaches to Student Assessment Have Institutions Adopted? u What Organizational and Administrative Support Patterns and Assessment Management Practices for Student Assessment Have Institutions Developed? u How Have Institutions Used Student Assessment Information and What Impacts Has It Had? u How Do Patterns of Student Assessment Approach, Organizational and Administrative Support Patterns and Practices, and Uses and Impacts of Student Assessment Vary by Institutional Characteristics? u How Do External Groups Influence Institutional Adoption of Various Measures of and Approaches to Student Assessment? u How Are External Influences on Institutional Approaches to and Organizational and Administrative Support Patterns and Practices for Student Assessment Related to the Use and Impact of Student Assessment Information?

Phase III: Case Study Research Seven Institutions With Extensive Involvement in Student Assessment Review of Response Patterns to National Inventory of Institutional Support for Student Assessment (ISSA) Campus Site Visits by Research Teams Collection & Synthesis of Documents, & Records Interviews & Focus Groups With Faculty, Administrators, & Students Research Purposes 1) To Examine the Internal Dynamics of Institutional Support for Student Assessment 2) To Identify Institutional Strategies for Supporting, Promoting & Integrating Student Assessment 3) Assess the Culture & Climate for Using Student Assessment Data for Academic Improvement

Integrating Student Assessment with Academic Management for Institutional Improvement Institution’s Academic Management Approach Use in Academic Decision Making Assessment  Purpose  Structure  Approach Assessment  Leadership  Support  Coordination Links to Management Process/ Improvement

Discussion of Institutions u Student Assessment Strategy u Purposes of Assessment u Assessment Structure u Approach to Assessment u Leadership, Support, & Coordination u Links to Academic Management Processes u Utilization in Academic Decision Making u Summary - Culture for Student Assessment Brief Institutional Storyline Institution’s Academic Management Approach Primary Mission & Governance Pattern Key Academic Management Processes

The Story of Institution Developed a Quality Initiative Based on a Baldridge Model Assessment Adopted as a Means of Academic Improvement Assessment Activity Gradually Integrated into Culture of Quality and Improvement for Many Years

Primary Mission & Governance Pattern Vision Emphasis on Culture of Quality; Sets Goals Statement of Mission Focuses on Continuous Improvement Committed to Using Assessment to Enhance Performance Cultural Core Values Define the University; Drive the Mission & Vision Decision Drivers Priorities Regarding Essential Functions

Strategic Planning Council Seven Step Planning Process Program Management Student Affairs KQIs Office of Assessment, Information, & Analysis Approach to Academic Improvement Key Academic Management Processes

Assessment Purpose -- NMSU Initiating Conditions Internal- President’s Focus on Culture of Quality (early 1980s) External- State Mandated Assessment by Governor (late 1980s) President’s Early Efforts Resisted Culture of Quality Focused on Academic Improvement in Undergraduate Education Adapted From Baldridge National Quality Award Criteria

Assessment Structure -- NMSU Formal, University-wide Strategy Uses Baldridge Quality Award Criteria As an Assessment Framework Comprehensive Approach -- Cognitive and Affective Learning  College and Post-college Experiences  Collected by by Office of Assessment, Information, and Analysis (OAIA) Seven Step Planning Process

Assessment Approach -- NMSU Quantitative & Qualitative Data Gathered From Recruitment Until Post-graduation  Battery of Cognitive Measures During College Years  Regular Collection Schedule  Data on Needs or Expectations of Students Using Local and National Surveys & Listening Posts Assessments are required--Explained in UG Catalog  Student progress tracked and enforced through restrictions on registration and final transcripts

Leadership, Support, & Coordination Leadership Long-term, Continuous Presidential Promotion Strong Support by Provost Through OAIA Other Groups: President’s Cabinet & Deans, Baldridge Category Council Faculty Senate Assessment Committee Student Success Task Force Faculty Provide Significant, but Hesitant, Support Coordination OAIA-- Compiles and Distributes Assessment Data Formal Grid Details Methods, Uses, & Frequencies for Assessment Activity

Links to Academic Management Process/ Improvement Annual Departmental Reports & Course Reviews  Primary Link Between Action Planning & Assessment  Deans Uses Assessment Data to Identify Areas for Improvement  Focus Is on Student Performance and Experience Data  President's Cabinet Reviews Reports & Provides Feedback to Unit Leaders Strategic Planning Council Representatives From Administrative, Academic, and Student Affairs Assessment Info Used: To Evaluate Organizational Performance on Strategic Initiatives (SI) The Development of New Key Quality Indicators (KQI) for Each SI

Links to Academic Management Process/ Improvement OAIA Produces Three Types of Academic Information: 1) Outcomes and Performance; 2) Student and Stakeholder Satisfaction Ratings; and 3) Operational Effectiveness Data Is Distributed Through Dashboards and Profiles  Tracking Systems to Indicate Deviations From Intended Targets  Can Be Accessed Through Campus Network Measurement of of Key Quality Indicators Seven Step Planning Process To Meet Stakeholder Requirements

Utilization in Academic Decision Making-- NMSU Planning Phase I Review of Mission and Analysis of Progress on Current Goals Phase II Identification of Strategic Initiatives Phase III Action Planning and Development Actions Plans Are Developed to Improve Performance on Key Quality Indicators Purpose Is to Increase Organizational Performance Assessment Information Part of KQIs in Dashboards & Profiles Critical to Developing Agenda for Planning Efforts

Utilization in Academic Decision Making Program Management Annual Reports & Reviews Benchmarking Stretch Goals Quality Student Affairs-Academic Affairs Alliance Development of Student Life & Experiences Surveys/Studies Links to Educational uses of Assessment Information NMSU Applies Annually for Baldridge Quality Award

Summary-- Culture for Assessment  Baldridge Quality Criteria Drive Assessment Framework  Integration of Assessment into Quality, Planning, and Program Management Processes  Data-Driven System & Reports  Strong, Central Administrative Support for Quality Approach & Assessment  Key Coordination Role Played by OAIA -- Commitment to Assessment As a Means of Visibility for Quality Emphasis Increasingly for Institutional Improvement

The Story of Assessment Data Essential to Cycles of Institutional Planning, Program Review, & Evaluation Assessment Information Used by Decision Makers at All Levels to Evaluate Progress Towards Goals Administration Expects that Assessment Activity Will Occur for the Purpose of Academic Improvement

Primary Mission & Governance Pattern No Formal Statements Regarding Assessment WFU Administration believes Assessment is a Necessary Management Tool to Maintain & Improve its Quality and Reputation Obtaining Data Crucial to Decision Making

Institutional Strategic Planning Evaluation Committee Institution-wide Assessment Departmental Review Student Life Office of Institutional Research Approach to Academic Improvement Key Academic Management Processes

Assessment Purpose -- WFU Initiating Conditions Internal- Foundations for Institutional Cycles of Planning Initiated by President & Vice-president External- Accreditation Focus on Institutional Effectiveness Program Planning & Evaluation Committees Use Assessment Data To Address Major Campus Needs Ongoing Strategic Planning Evaluate Strategic Plan The Plan for the Class of 2000 Assessment Now a Management Tool in a Continual Process of Planning, Implementation, and Evaluation

Assessment Structure -- WFU 1) University-wide- OIR- Oversees Assessment of the Process Administers National & Local Surveys to Students & Faculty Analyzes, Stores, & Disseminates Data Works With Evaluation Committee & Other Depts. Only Office to Have Dedicated Funds for Assessment Three Level, Decentralized Process 3) Student Life- Focuses on its own programs Focus is on student experiences & satisfaction Important for Intellectual Climate Planning 2) Departmentally Based- Program Review Process Primary Planning Unit on Campus 13-month Process, 7 Year Cycle for All Programs Review of Curriculum & Department/ Program Performance $2,500 Budgeted for Each Review

Assessment Approach-- WFU Departments Conduct Own Assessment of Programs & Satisfaction  Significant Autonomy & Discretion Granted to Evaluate Their Students Primarily Quantitative Focus on Satisfaction & Experiences  Little Emphasis on Cognitive or Affective Measures  Some Determined by Dept. In Departmental Review Institution-wide Surveys of Students, Faculty, & Alumni  National Surveys Administered by Office of Institutional Research  Also Some Locally Developed Instruments & Course Evaluations

Leadership & Support Institutional Support from Executive President’s and Provost’s Seminars, Conferences, & Retreats Executive Council & Reynolda Cabinet Receive Reports & Use the Information in Strategic Planning

Coordination OIR Shares Its Information With Departments Upon Request Student Life Provides Its Data to Departments Information Systems Office Sends Course Evaluation Results to Department Heads Online OIR Role in Accreditation Allows It to Maintain Focus on Institutional Effectiveness Criteria OIR Has the Primary Role In: Administering Institution-wide Student Assessments Coordinating Program Planning Conducting Data Analysis

Information Flow/ Link to Improvement Data Driven Approach to Management Measuring Progress on Strategic Plan Goals Results in Steps Toward Improvement OIR & Departments Report to Evaluation Committee (EC) for the Strategic Plan Reviews Departmental Reports on How SA Info Was Used to Monitor Progress on Goals  EC Monitors 27 Areas of Quality  EC Members Serve As Liaisons to Other Committees and Departments Annual Reports by Departments to Deans Must State Progress on Goals and Set New Ones

Utilization in Academic Decision Making Institutional Strategic Planning Prior Data Streams Produced the Development, Implementation and Evaluation of Plan for the Class of 2000 Recent Student Life Data Critical to Next Wave of Planning on Intellectual Climate Program Review & Departmental Evaluations One Full Cycle Completed Departmental and Curricular Changes, including Program Restructuring or Elimination Departments use OIR Data & their own to Improve Teaching & Learning

Utilization in Academic Decision Making Evaluation Committee Assesses Laptop Initiative and Effect of Technology on Academics Purpose Remains Measurement of Progress Towards Academic & Educational Goals Student Life Units Engage in Planning & Evaluation Many Changes to Services Made Based on Satisfaction Survey Results

Summary-- Culture for Assessment  Academic Management is Well-designed  Integrated Institutional Planning, Program Review, & Academic Evaluation Approach that is Improvement Driven  Student Assessment Integrated into All Academic Management Processes  Strong, Central Support for Data Driven Management  Intensive Involvement of Departments in Program Review  Key Support & Coordination Roles Played by OIR  Commitment to Assessment is Embedded in Culture, but Not as an End in Itself Wide Support for Student Assessment

Conclusions

Lessons & Insights Emphasis is on Integrating Student Assessment into Key Elements & Academic Decision Making Arenas:  Assessment Used at All Levels of Institution  Tied to Academic Improvement  Embedded in Academic Management Policies & Practices  Assessment Data Key to Management & Evaluation Processes Need a Clear Understanding of:  The Institution’s Academic Management Approach & its Key Elements  The Role of Assessment in Their Various Elements

Lessons & Insights An Effective Student Assessment Strategy Requires:  Clearly Understood Purpose for Assessment  Committed, Upper-Level Leadership  Coordination Across Support Offices & Departments  Flow of Information to Decision Makers at All Levels  Link of Data to Problem Identification and Goal Setting Building a Culture that Values Assessment Requires:  Long-term Effort Linking Assessment to Academic Management Processes Concerned with Academic Improvement

Wake Forest Case Study Visit September 2000 u Wake Forest Honored to Be Selected for the Case Study u Faculty, Students and Administrators Were Most Cooperative in Participation of Interviews and Focus Groups u Interviews Included President, Vice Presidents, Deans, Associate Provost, Advising Director, Director of Teaching & Learning Center, Director of Institutional Research, Chief Information Officer, Faculty Senate President, Chair of Curriculum Committee, Chairs of Academic Departments and Members of Evaluation Committee u Focus Groups Included Four Faculty Members and Four Departmental Chairs Active in Assessment and 10 “Good” Students

Lessons Learned u Considerable Assessment With Decided Outcomes Took Place at Institutional Level, e.g. Program Review and Surveys of Students and Faculty u Assessment at Departmental Level Is Decentralized and Varies Considerably u Qualitative Assessment Could Be Emphasized More at the Institutional Level u Combination of SACS Self Study and Strategic Plan Strengthened Assessment at Wake Forest