1. Overview of Institutional Effectiveness Concepts and Measures Andreea M. Serban, Ph.D. Solano Community College July 19, 2006 1

2. Definitions 2 Institutional Effectiveness “The ability of an institution to match its performance to established purposes as stated in its mission” (Alfred et al, p.6) Expression of institutional values and priorities “An internal process of planning and evaluation that is intended to ensure that the college’s performance matches its purposes. A global process that uses assessment strategies to demonstrate accountability” (Midlands Technical College, p.ix).

3. Purposes 3 Clarification of Mission Improvement Identification of Priorities Working Document Planning and Decision Making Enhance Reputation Historical Record Accreditation Requirement Information for Prospective and Current Students

4. Core Concepts 4 Stakeholder Needs MissionResults Resources Costs

5. Institutional Effectiveness and Planning 5 Institutional effectiveness measures are part of the annual evaluation of the strategic plan The information is used to determine progress towards achievement of stated goals in the strategic plan Establish a structure to monitor, report and take corrective actions based on institutional effectiveness measures Departmental effectiveness measures are linked to and, as appropriate, aggregate into institutional effectiveness measures The college needs to have a systematic approach to communicating the data from the institutional effectiveness to internal and external constituents on a regular cycle (annual basis suggested)

6. Modeling and Measuring Effectiveness 6 Outcomes serve as benchmarks for the college mission by comparing results with purpose, within the limits of costs and available resources When outcomes are compared with an institution’s mission and goals, the result is a composite picture of an institution’s effectiveness An indicator that helps create this composite picture describes a condition or result that the community college can produce regularly, report publicly, and use systematically in decision making Source: AACC A core indicator is a measure that describes a critical, widely recognized outcome of mission – one that is clearly responsive to key constituent groups and is produced regularly.

7. 1999 AACC Model Mission and Core Indicators 7 Student Progress Student Goal Attainment Persistence (Fall to Fall) Degree Completion Rate Workforce Development Placement Rate in the Workforce Employer Assessment of Students Licensure/Certification Pass Rates Client Assessment of Programs and Services General Education Demonstration of Critical Literacy Skills Demonstration of Citizenship Skills

8. 1999 AACC Model Mission and Core Indicators (continued) 8 Transfer Preparation Number and Rate Who Transfer Performance After Transfer Developmental Skills Success in Subsequent, Related Coursework Outreach Participation Rate in Service Area Responsiveness to Community Needs

9. Feasibility of the AACC Model 9 Core Indicators for which Data Are Available Core Indicators for which Data Are Not Easily Available Persistence (Fall to Fall) Degree Completion Rates Job Placement Rates (data have limitations) Number and Rate Who Transfer Success in Subsequent, Related Courses Licensure/Pass Rates Participation Rate in the Service Area Student Goal Attainment Employer Assessment of Students Client Assessment of Programs and Services (at a minimum use student surveys) Demonstration of Critical Literacy Skills Demonstration of Citizenship Skills Performance After Transfer Responsiveness to Community Needs

10. Expanding on the AACC Model Examples from Community Colleges 10 Santa Barbara City College – Criteria for selecting institutional effectiveness measures Related to mission and College Plan Reflective of the changes in relevant literature Reflective of the ability of the Office of Institutional Assessment, Research and Planning to obtain the information from the college student system, other college databases/sources or external sources Quantifiable Accepted by the college community as valid

11. Expanding on the AACC Model Examples from Community Colleges 11 Santa Barbara City College (list of measures provided in separate document) Measures of institutional effectiveness developed for and linked to each major area of the college plan Discussion of institutional effectiveness trends and implications for the college included in the annual evaluation of the college plan Annual presentations and discussions with the Board of Trustees and shared governance groups (College Planning Council, Academic Senate)

12. Expanding on the AACC Model Examples from Community Colleges 12 Long Beach City College (annual institutional effectiveness report and latest presentation to Board of Trustees provided in separate documents) Uses the AACC outcome dimensions tied to Astin’s assessment model which differentiates between inputs, process/environment and outcomes Measures of institutional effectiveness tied to the long range goals specified in the Educational Master Plan

13. Implications for Solano Community College 13 The college had previously developed performance indicators related to the goals included in the Strategic Plan as outlined in the “Review and Validation of Solano Community College District’s Strategic Plan” May 2000. Those performance indicators represent a solid base for creating an institutional effectiveness framework related to the Strategic Plan goals and objectives. Institutional effectiveness measures, once developed, should be monitored, reported, discussed and integrated into the annual evaluation of the Strategic Plan.

14. References 14 Alfred, R., Ewell, P., Hudgins, J., and McClenney, K. (1999). Core Indicators of Effectiveness for Community Colleges, Second Edition. Washington, D C: Community College Press, American Association for Community Colleges. Midlands Technical College. (1997). Managing Your Institution’s Effectiveness. A User Guide. Washington, D C: Community College Press, American Association for Community Colleges.

15. Resources 15 Banta, T. W., Borden, V.M.H. (1994). Performance Indicators for Accountability and Improvement. In V.M.H Borden and T. W. Banta (eds.), Using Performance Indicators to Guide Strategic Decision Making. New Directions for Institutional Research No. 82: San Francisco, CA: Jossey Bass. Burke, J. C., Modarresi, S., Serban, A.M. (1999). Performance: Shouldn’t It Count for Something in State Budgeting?. Change, November/December, 31(6), 16-23. Burke, J. C. (1998). Performance Funding Indicators: Concerns, Values, and Models for State Colleges and Universities. In J. C. Burke and A. M. Serban (eds), Performance Funding for Public Higher Education: Fad or Trend?. New Directions for Institutional Research No. 97. San Francisco, CA: Jossey Bass. Cleary, T. S. (2001). Defining Quality Through the Eyes of Campus Stakeholders. Community College Journal, August/September, 41-47. Council for Higher Education Accreditation. (2000). The Common Data Project. CHEA Occasional Paper, August. Washington: DC: CHEA.

16. Resources 16 Ewell, P. T. (1992). Outcomes Assessment, Institutional Effectiveness, and Accreditation: A Conceptual Exploration. Resource Paper for the Council on Postsecondary Accreditation Task Force on Institutional Effectiveness (ERIC Document No. 343513). McLeod, M. W., Cotten, D. K. (1998). Essential Decisions in Institutional Effectiveness Assessment. Visions: The Journal of Applied Research for the Florida Association of Community Colleges, 2(1), 39-42. Nichols, J. O. (1995). A Practitioner’s Handbook for Institutional Effectiveness and Student Outcomes Assessment Implementation. Bronx, NY: Agathon Press. Ronco, S. L., Brown, S. G.. (2000). Finding the "Start Line" with an Institutional Effectiveness Inventory. Paper presented at the Annual Forum of the Association for Institutional Research, Cincinnati, OH, May 21-23, 2000. Roueche, J. E., Roueche, S. D. E., Eileen E.. (2001). Pursuing Excellence: The Community College of Denver. Community College Journal of Research and Practice, 25(7), 517-537. Roueche, J. E., Johnson, L. R., Roueche, S. D., & Associates. (1997). Embracing the Tiger: The Effectiveness Debate and the Community College. Washington, D.C.: The Community College Press.

17. Resources 17 Serban, A.M. (2001). Institutional Effectiveness and Performance Funding for Community Colleges: Where Do Institutional and State Values and Priorities Converge? Presented at the Annual Forum of the Association for Institutional Research. Long Beach, CA, June 4, 2001. Sorensen, James E. (1998). An Investigative Study on the Systematic Application of Effectiveness Indicators for Institutional Improvement in Northwest Community Colleges. Sullivan, M. M., Wilds, P. C. (2001). Institutional Effectiveness: More Than Measuring Objectives, More Than Student Assessment. Assessment Update, 13(9), 4-6.