1. Benchmarking in Community Colleges: Status of Two National Projects Jeffrey A. Seybert Director, Research, Evaluation, and Instructional Development Johnson County Community College George Malo Associate Vice Chancellor for Research and Assessment Tennessee Board of Regents John D. Porter Associate Provost The State University of New York

2. Two National Benchmarking Projects  The Kansas Study Community College instructional costs and productivity Modeled on the Delaware Study Collects, analyzes, and reports data at the discipline level  The National Community College Benchmark Project Involves a wide array of student outcomes, access, workforce development, faculty/staff, human resources, and finance variables Collects, analyzes, and reports data at the institutional level

3. The Kansas Study  Supported by a three-year, $282,000 grant from FIPSE (USDE).  Colleges will be able to analyze faculty workload and instructional cost at the academic discipline level of analysis.

4. Kansas Study History Summer 2002 FIPSE project approval and grant award Fall 2002-Fall 2003 Advisory committee identifies data elements, designs processes, and conducts two pilot studies Fall 2004 Aggregate reports distributed; website opened for peer comparisons 2004 Year 1 project implementation – 50 institutions provided data 2005 Year 2 - 67 institutions participated 2006 Year 3

5. How Kansas Study Works  Data Collection Excel Spreadsheets distributed electronically Data Verification: Missing data and logical errors Partial Data OK (min. 10 disciplines) Confidentiality assured  Annual Reports National Norms and Institutional Data Access to Kansas Study Website for Peer Comparisons

6. Kansas Study Timeline February 1 Data Collection Starts May 15 Data Verification Process Initiated June 18 Participant Institutional Data Due July 15 Data Verification Reports Sent July 5 Data Analyses Begin Early Fall Results Available; Database Opened for Peer Comparisons/ Benchmarking

7. Web Site  Kansas Study Website (www.kansasstudy.org) Public Information General Information Enrollment Form Sample Data Collection Template Sample Report Tables Advisory Committee Participating Institutions Information Available to Participants Only Log In & Password National Norms by Discipline Peer Comparisons

8. Benchmarking Instructional Costs and Productivity: How a System and Campus Use the Kansas Cost Study George Malo Assistant Vice Chancellor for Research and Assessment Tennessee Board of Regents

9. The Kansas Cost Model  Purpose Colleges will be able to analyze faculty workload and instructional cost at the academic discipline level of analysis Provides comparative data important to accountability processes and decision making at both the system and institution levels  Participants Community Colleges  Context Modeled in part on the Delaware cost model for universities Tennessee Board of Regents

10. TBR Context  Why do we participate? Part of TBR and State initiatives Defining Our Future Strategic planning process Accountability reporting Performance funding Cost model replaces Board’s former cost study Opportunity for national comparisons as well as a management tool for campuses Provides consistency across system institutions Useful for system policy and management decisions Tennessee Board of Regents

11. Strategic Planning  Provides benchmarks  Annual monitoring of key program variables  Documentation of activities  Use in Decision-Making  Part of presidential evaluations Tennessee Board of Regents

12. Performance Funding  F ive points awarded as part of assessment standard  A ll 13 community colleges in TBR system must participate  R eport on 4 key indicators (to be discussed later in presentation)  S ubmit a report providing evidence of the usage of the Kansas model for institutional planning and improvement. Tennessee Board of Regents

13. Campus Uses of Kansas Study  Program Review or Academic Audit  Look at high risk or outlier programs  Look at Peer Costs  Staffing & tenure decisions  Problem–solving tool  SACS accreditation documentation Tennessee Board of Regents

14. Caveats to Avoid Misuse  Tool for institutional decision-making  Support credible case-making and informed decision-making  Must be used as trend data  Should not be used for inferences of an institution as a whole  Prerequisite for assessing the adequacy of accountability Tennessee Board of Regents

15. System Level Ad Hoc Committee  Institutional input through Academic Affairs committee for appropriate use of data  Adoption of key indicators as standard for framing instructional productivity and effectiveness reports  Development of common questions that would lead institutions to evaluate their decisions Tennessee Board of Regents

16. System Level Key Indicators  FTE students taught per FTE instructional faculty by discipline  Student credit hours per FTE faculty as a percentage of national norm by discipline  Percentage student credit hours taught by full-time faculty Tennessee Board of Regents

17. Comparison Group Selection  Each institution will construct its peer groups according to three standards, two for System use and the third for institutional use only System reporting as an aggregate System reporting per discipline At the institution level, each discipline may select peers Tennessee Board of Regents

18. Evidence of Accountability/Productivity  For each key indicator, the TBR System adopted questions to guide institutions in the analysis of their own data What significant changes can be detected over the last three years for the indicator? How does this three-year profile for the indicator compare to that of institutional peers by CIP? What factors have contributed to the changes at your institution around the key indicator? Are you satisfied? Why, and if not, how do you plan to make any alterations to adjust key indicator? Tennessee Board of Regents

19. Case Making – A Central Goal  Data must be used for responsible decision making Can the institution make a case, from its analysis of the allocation of faculty, that it is moving toward improvement in instructional management? Can the institution make the case that it is effectively using its faculty ? Can the institution make the case that it is moving toward improvement in contributions to the institution, system, state, students, or the public? Do these contributions reflect a responsible use of resources ? Tennessee Board of Regents

20. System Wide Assistance  Programming for data collection  Programming for analysis of data  Templates for reporting of data  Revisions to/formulation of policies and guidelines Tennessee Board of Regents

21. Campus Uses of Kansas Study  Documenting accreditation compliance  Planning institutional change  Predicting academic/financial impact Tennessee Board of Regents

22. Accreditation  Institutional Effectiveness  Adequate Faculty  Sound financial base and adequate resources Tennessee Board of Regents

23. Annual Program Documentation Program % of Adjunct MCC/KS Cost per Credit Hour MCC/KS Faculty/Student Ratio MCC/KS Biology 2004 2005 27% 30% 57% 33% $59 $66 $49 $66 18.4 23.0 19.0 23.1 Business 2004 2005 35% 40% 19% 35% $72 $79 $88 $76 17.8 19.0 9.8 18.2 Computers 2004 2005 27% 43% 35% 33% $89 $88 $106 $101 13.6 15.9 10.7 15.5

24. Document Staffing by Program Program SCH 2003/2005 % Adjunct MCC/KS Student/Faculty Ratio MCC/KS Automotive245/236 0 14% 7.8 14.3 Economics471/570 24% 28%13.0 24.2 Architecture283/148 25% 16% 7.3 11.3 Office Administration 2074/1855 60% 28%12.0 14.0 Psychology1872/2736 60% 43%17.0 27.0 Early Childhood1036/946 65% 54%15.7 18.3

25. Document Cost by Program ProgramMCC Cost per SCH KS National Cost MCC Student/Faculty Ratio Computer Information Systems 2004 2005 $89 $106 $88 $101 13.6 10.7 College Math 2004 2005 $83 $80 $74 $70 18.4 19.5

26. New Program Planning KS S/F Ratio KS Cost Urban Peer S/F Ratio Urban Peer Cost Criminal Justice 2004 2005 19.4 20.9 $69 $66 19.5 20.2 $60 $62 Elec Technologies 2004 2005 12.1 10.8 $149 $184 10.7 11.2 $152 $160

27. The National Community College Benchmark Project Involves a wide array of student outcomes, access, workforce development, faculty/staff, human resources, and finance variables Collects, analyzes, and reports data at the institutional level

28. Purposes To collect and report community college benchmark data on a national basis To provide data for comparisons and benchmarks of instructional, workforce-development, and other community college activities

29. NCCBP History 2003Project Designed and Piloted 2004First year implementation; interactive project website designed and launched; 110 institutions participated - SUNY System (30 Colleges) - TN System (13 Colleges) 2005Second year implementation; 113 institutions participated - SUNY System (30 Colleges) - TN System (13 Colleges) - PA Colleges (13 of 14) 2006Third year implementation; enrollment opened in Feb. - SUNY System (30 Colleges) - TN System (13 Colleges) - PA Colleges (14 Colleges) - FL System (28 Colleges)

30. How NCCBP Works  Data collection Excel spreadsheets distributed electronically Data verification: Missing data and logical errors Partial data OK; no peer comparison for missing data Confidentiality assured  Cost: $1,000/year per institution  Annual reports Aggregate data delivered electronically Access to NCCBP Web site for peer comparisons  Website: www.NCCBP.orgwww.NCCBP.org

31. Data-collection Form FORM 4: Credit Students Who Enrolled Next Term and Next Fall Column 1Enter unduplicated total credit students (including those who withdrew from all courses) at the end of the fall 2003 term. Do not I include high school students. Column 2Enter total students from Column 1 who graduated or completed certificates before the next (spring 2004) term. Column 3Enter total students from Column 1 who enrolled in the next (spring 2004) term. Column 4Column 3 / (Column 1 - Column 2) Column 5Enter total students from Column 1 who graduated or completed certificates before next fall (fall 2004) term. Include graduates and completers in Column 2. Column 6Enter total students from Column 1 who enrolled in the next fall (fall 2004) term. Column 7Column 6 / (Column 1 - Column 5) Column 5 Total from Col 1 Who Graduated before Fall 2004 Column 6 Total from Col 1 Who Enrolled in Fall 2004 Column 7 Fall-Fall Persistence Rate 0% Column 1 Total Credit Students at End of Fall 2003 Term Column 2 Total from Col 1 Who Graduated before Spring 2004 Column 3 Total from Col 1 Who Enrolled in Spring 2004 Column 4 Next Term Persistence Rate 0%

33. Benchmark Categories Completion & Transfer Rates Persistence Rates Transfer Student Performance Student Satisfaction Student Performance Measures Career Preparation Academic Success Access & Participation Market Penetration Workforce Development Section Size, SF Ratio, Faculty Load Student Services Staff HR Statistics Instructional & Professional Development Costs

34. 2006 Timeline March Data collection begins. May Data-collection instruments are due. June Data confirmation reports are distributed. July Data updates are due. September Aggregate reports are distributed. Web site is opened for peer comparisons.

35. 2004 Participants

36. College Characteristics Campus Environment Institution Type Institutional Control Academic Calendar Credit Enrollment Minority Students Percent State Revenue Operating Budget Faculty Unionized Service Area Population Unemployment Rate Household Income Service Area Percent Minority

37. National Community College Benchmark Project: A System/State Perspective John D. Porter Associate Provost The State University of New York

38. NCCBP: A Valuable Resource for Systems & States  NY’s CCs are funded based on annual full-time student equivalents (FTE)  CCs need to benchmark their operations to maintain & expand state support  CC’s are as complex as research universities, which is not understood by most decision makers  NCCBP fills a critical void

39. SUNY’s Community Colleges  SUNY’s community colleges enroll 208,374 students  50.3% of SUNY’s overall enrollment  Campuses range in size from 21,000 to 1,500  Located throughout New York Stake, including New York City  One CC awards bachelor and master degrees (FIT)  These institutions have every conceivable governance/funding arrangement

40. SUNY Support of NCCBP  SUNY’s benchmarking has here-to-fore focused on “intra” measures  NCCBP offers the potential to benchmark against true peers and other states  For the past three years, SUNY has encouraged campus participation by paying the subscription fee  This year, all 30 community colleges will participate in NCBBP  SUNY’s hope is that other states and systems will see the value of this project and participate

41. NCCBP: A Valuable Resource for Systems & States  NCCBP has generally been conceived as a tool for campuses  Systems/States need this type of resource, since most community colleges are funded based on enrollment  SUNY requires CCs to plan enrollments 5 years into the future; also update the institutional mission every five years  NCCBP has the potential for developing reports tailored to the needs of Systems and States

42. Issues?  Participation needs to reach a critical mass – 300 institutions?  Gaining support for NCCBP on campus (some don’t want to be compared)  Funding – shifting cost to the campus at some point in the future  Accuracy/quality of data? How best to achieve?  Important that NCCBP keeps the cost of participating low

43. National Community College Benchmarking Project George Malo Associate Vice Chancellor for Research and Assessment Tennessee Board of Regents

44. Uses of NCCBP  Strategic planning  Performance funding  Documenting accreditation  Policy development/analysis/evaluation Tennessee Board of Regents

45. Where Does MCC Excel? Core Course Success MCC NCCBP Median NCCBP Peers Benchmark Success in English Comp I 200476%71% 78% 200588%84%88%90% Success in College Algebra 200471%59%55%72% 200584%73%83%86%

46. Example of Performance Funding Indicators MCC 2004 NCCBP 2004 Peer NCCBP 2005 Peer Success in College Courses 89%87%89% Success in College Algebra 71%60%83% College English Success for DSP Students 92%87%83% GPA at Transfer Institution 2.722.902.91

47. Outcomes of Educational Programs Career Program Completers ReportMCC NCCBP Median NCCBP Peers Benchmark Related Employment 200468%69%64.5%68% 200575%66%68%78% (75 th ) Continuing Education 200415%16%23%16% 200515%20%45%20% (mdn)

48. Policy Questions Tennessee Board of Regents

49. Questions