Exploring Good Practices in Outcomes-Based Assessment Program Review Marilee J. Bresciani, Ph.D. Associate Professor, Postsecondary Education and Co-Director of the Center for Educational Leadership, Innovation, and Policy San Diego State University 3590 Camino Del Rio North San Diego, California, U.S.A

Bresciani, M.J. Overview of Book Describes “good practices” in outcomes- based assessment program review that were generated from multiple case studies. Outlines basic components of outcomes- based assessment program review. Introduces criteria for identifying good practices within outcomes-based assessment program review. Outlines steps for implementing sustainable outcomes-based assessment program review.

Bresciani, M.J. Context Public demand for accountability in higher education Commission on the Future of Higher Education Professor Bensimon’s highlight of our innate intellectual inquiry, “Why is that what I am doing doesn’t work?”

Bresciani, M.J. Purpose The purpose of this study is to use multiple institutional case studies to explore effective outcomes-based assessment program review practices in order to establish criteria that could be adapted at other institutions for implementing pervasive and sustainable outcomes-based assessment program review.

Bresciani, M.J. Framework and Epistemology Astin’s I-E-O Astin, A. W. (1993). Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. American Council on Education; Oryx Press. Papert’s Situating Constructionism Papert, S. (1991). Situating Constructionism. In Constructionism. Edited by Papert and Harel. Cambridge, MA:MIT Press.

Bresciani, M.J. Research Questions What are the components of outcomes-based assessment program review at institutions where this practice is pervasive, effective, efficient, and enduring ? What are the criteria for pervasive, effective, efficient, and enduring outcomes-based assessment program review processes? What are the key steps in implementing pervasive, effective, efficient, and enduring outcomes-based assessment program review processes?

Bresciani, M.J. Methodology The methodology for this study was multiple case studies (Stake, 1995; Merriam, 1988) Multiple sources of information were utilized in this analysis: surveys, case study submissions, document analysis, and clarification sought through and over phone. 3 Year Study Open, axial, and selective coding Member checks

Bresciani, M.J. Sample Criterion sampling A list of 43 institutions generated based on criteria and circulated to 25 assessment scholars for further comment. No additions or deletions were made. 18 institutions completed all phases and were included in the study

Bresciani, M.J. Good Practice Components 1. Program Name and Description 2. Program Mission Statement 3. Program Goals 4. Program Outcomes 5. Student Learning Outcomes 6. Evaluation Methods for Each Outcome

Bresciani, M.J. Good Practice Components (cont’d) 7. Results by outcomes or interpretation of results as they relate to outcomes 8. Decisions & recommendations made about the program for each outcome 9. Level of involvement of program faculty/staff in process 10. Time-frame of evaluation process

Examine the application of the template See program review guidelines from John F Kennedy University

Bresciani, M.J. Criteria for Good Practices of Outcomes-Based Assessment Program Review Criterion 1: Clear Understanding of Goals and Expectations for Program Review Criterion 2: Collaboration Criterion 3: Use of Results Criterion 4: Awards and Recognition Criterion 5: Resources to Support Program Review

Bresciani, M.J. Criteria for Good Practices (cont’d) Criterion 6: Coordination of the Process Criterion 7: Flexibility Criterion 8 : Addressing Barriers Criterion 9: An Evaluation of the Program Review Process

Examine a sample matrix for considering effective, efficient, and enduring OBAPR

Bresciani, M.J. Suggestions for Implementing Outcomes-Based Assessment Program Review (OBAPR) Create a Well-Represented, Well- Respected OBAPR Committee Organize the Committee’s Role and Responsibilities Articulate Expectations for Outcomes- Based Assessment Program Review Plan Short-Range and Long-Range Goals

Bresciani, M.J. Suggestions for Implementing Outcomes-Based Assessment Program Review (OBAPR), Cont. Identify Existing Resources and Processes and Identifying New Resources Establish a Communication Plan Discuss Implementation Barriers and Strategies to Overcome Them Move Forward with Flexibility

Examine sample questions to consider when implementing OBAPR

Bresciani, M.J. Criterion Sample Selection References American Association of Higher Education (1994). Nine principles of good practice for assessing student learning. Retrieved March 17, 2006, from Eckel, P., Green, M., and Hill, B. (2001). On change V— Riding the waves of change: Insights from transforming institutions. Washington DC: American Council of Education. Lopez, C. (1997). Opportunities for improvement: Advice from consultant-evaluators on assessing student learning. Evidence of strong institutional support for assessing student learning. Retrieved March 16, 2006, from es/assessment/index.html.

Bresciani, M.J. Citation Bresciani, M.J. (2006). Outcomes-Based Academic and Co-Curricular Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing