1. Identifying Your Role in Outcomes-Based Assessment Program Review Marilee J. Bresciani, Ph.D. Professor, Postsecondary Education and Co-Director of the Center for Educational Leadership, Innovation, and Policy San Diego State University 3590 Camino Del Rio North San Diego, California, U.S.A. 619-594-8318 Marilee.Bresciani@mail.sdsu.edu

2. Bresciani, M.J. Presentation Overview Overview of Outcomes-Based Assessment (OBA) Program Review Importance of OBAPR Importance of Assessing Student Learning Astin I-E-O Questions to Consider Questions

3. Bresciani, M.J. Ask Yourself These Questions What decision did you make about your program last year? What evidence did you use to inform that decision? What was it that you were trying to influence about your program when making that decision with the stated evidence?

4. Bresciani, M.J. That is Outcomes-Based Assessment (Bresciani, 2006) Most people do capitalize on their innate intellectually curiosity to find out what works Most people just don’t articulate their intended end results (e.g., outcomes) ahead of time Most people don’t document the decisions made based on their results Most people don’t follow up later to see if their decisions made the intended improvement

5. Bresciani, M.J. The Assessment Cycle (Bresciani, 2006) The key questions… What are we trying to do and why? or What is my program supposed to accomplish? or What do I want students to be able to do and/or know as a result of my course/workshop/orientation/program? How well are we doing it? How do we know? How do we use the information to improve or celebrate successes? Do the improvements we make contribute to our intended end results?

6. The Iterative Systematic OBPR Cycle (Bresciani, M.J.) Mission/Purposes Goals Outcomes Implement Methods to Deliver Outcomes (Action Planning) and Methods to Gather Data Gather Data Interpret Evidence Document decisions to improve programs; enhance student learning and development; inform institutional decision- making, planning, budgeting, policy, public accountability External Review Strategic Planning/ Inputs/Capacity -Values influence every aspect of this cycle. Bresciani, M.J.

7. The Purpose Outcomes-Based assessment does not exist for assessment’s sake It is taking what most of us already do, and making it systematic Its purpose is to reflect on the end result of doing - - how well are we accomplishing that which we say we are?

8. Bresciani, M.J. The Purpose, Cont. Strategic and Action Planning are incorporated into it It is intended to inform decisions for improvement and resource re- allocation It helps you link what you do to institutional strategic initiatives and performance indicators

9. Bresciani, M.J. The Purpose, Cont. Makes the purpose of our programs and services clear to students, faculty, parents and other constituents Promotes student responsibility for learning Promotes opportunities for collaboration Outcomes-Based Assessment is not research

10. Importance of Assessing Student Learning Demonstrates contributions to institutional mission and goals And contributions to institutional priorities Assists in informing prioritization of your time as well as other resources

11. Decisions are made at multiple levels - some decisions reside only at one level Bresciani, M.J.

12. Given what OBAPR is, how do you perceive your role in the process? Bresciani, M.J.

13. Why the increased emphasis in OBAPR? Bresciani, M.J.

14. Drivers of Outcomes-Based Assessment ( Ewell, 2003) To improve the underperforming student Competency Movement in Business and Industry International Trade Agreements Bologna Declaration of 1999

15. Bresciani, M.J. Drivers, Cont. Government Conversation, 1985 The Higher Education Re-authorization Act Testimonies in USA, 2002 and 2006 Response to NCLB Legislation Regional Accreditation – flexibility CRAC – 2003, 2004 Both documents focus on student learning

16. Bresciani, M.J. Drivers, Cont. National Commission on the Future of Higher Education Demand for Public Information about Performance Transparency of outcomes and results Comparable measures of quality Demonstration of value-added of the entire educational experience

17. Bresciani, M.J. Drivers, Cont. Accountability requirements handed back to states Performance indicators Discipline Standards could be designed by disciplines AACU Essential Outcomes/Degree Frameworks and Expected Levels of Learning Economic State of the Country

18. Astin’s I-E-O Inputs - Pre-Assessment Environment –Formative Assessment Outcomes – Summative Assessment Bresciani, M.J.

19. The Focus on Assessing Student Learning “The concepts of learning, personal development, and student development are inextricably intertwined and inseparable.” – The Student Learning Imperative

20. Some Questions about Student Learning and Development ( Bresciani, Moore Gardner, & Hickmott, 2009) What do you expect your students to know and be able to do by the end of their education at your institution? And how is your program designed to contribute to that expected learning? What do you do in your programs to promote the kinds of learning and development that your institution seeks? Bresciani, M.J.

21. Some More Questions ( Bresciani, Moore Gardner, & Hickmott, 2009) How do I influence student learning? Is it through the education of my colleagues? Of faculty? Of parents? Of community participants? Bresciani, M.J.

22. Reflection Questions ( Bresciani, Moore Gardner, & Hickmott, 2009) How are you directly or indirectly contributing to student learning? How are you directly or indirectly supporting student learning? How are you directly or indirectly interfering with student learning? Bresciani, M.J.

23. The Ideal for Student Learning Establish collaborations between academic and student affairs to facilitate student learning facilitate student engagement and socio-academic integration evaluate professional development for faculty and staff about effective learning environments Bresciani, Zelna, & Anderson, 2004 Bresciani, M.J.

24. Now, how do you perceive your role in OBAPR?

25. Ideas for Your Involvement Pre-assessment design Cognitive Developmental Testing the efficacy of placement scores Incorporating the developmental aspects into design of placement Working with advisors to see how placement scores are used Bresciani, M.J.

26. Ideas, Cont. Working with Academic and Student Affairs to evaluate the efficacy of all assessment instruments and determine what other measures may be utilized to evaluate student learning and development. Bresciani, M.J.

27. Ideas, Cont. Determining which comparable measures of learning are appropriate for your organization to use Assisting with the development of new comparable measures of learning and testing their efficacy Rubrics Bresciani, M.J.

28. What Else? Bresciani, M.J.

29. Questions to Consider What is your vision for your role in OBAPR? With whom do you need to communicate your vision? How might you need to re-organize your office to meet your emerging role in OBAPR? Bresciani, M.J.

30. Questions, Cont. What resources do you need in order to carry out your vision? Who do you need to collaborate with within your Institution State Nation? Bresciani, M.J.

31. Questions, Cont. In order to make your vision a reality, What do you need to do immediately when you return to your campus? What do you need to do three months from now? What about a year from now? Bresciani, M.J.

32. Resources Each Other NASPA, ACPA, and AACU University Planning and Analysis (UPA) Assessment website http://www2.acs.ncsu.edu/UPA/assmt/

33. Questions?

34. Bresciani, M.J. One Minute Evaluation What is the most valuable lesson that you learned from this workshop? What is one question that you still have? What do you think is the next step that your division/program needs to take in order to implement systematic program assessment?

35. References Astin, A. W. (1993). What matters in college: Four critical years revisited. San Francisco: Jossey-Bass. Bresciani, M.J. (September, 2002). The relationship between outcomes, measurement. and decisions for continuous improvement. National Association for Student Personnel Administrators, Inc NetResults E-Zine. http://www.naspa.org/netresults/index.cfm http://www.naspa.org/netresults/index.cfm Bresciani, M.J., Zelna, C.L., and Anderson, J.A. (2004). Techniques for Assessing Student Learning and Development in Academic and Student Support Services. Washington D.C.:NASPA.

36. References, Cont. Bresciani, MJ.(2006). Outcomes-Based Undergraduate Academic Program Review: A Compilation of Institutional Good Practices. Sterling, VA: Stylus Publishing. Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2010). Demonstrating student success in student affairs. Sterling, VA: Stylus Publishing. Palomba, C.A. and Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in Higher Education. San Francisco: Jossey-Bass.