1. Program Review for Academic and Administrative Departments http://www.rcc.edu/administration/academicaffairs/effectiveness/review.cfm Kristina Kauffman, Associate Vice Chancellor Institutional Effectiveness David Torres, District Dean Institutional Research Kristina Kauffman, Associate Vice Chancellor Institutional Effectiveness David Torres, District Dean Institutional Research

2. Program Review Goals Extrinsic Resource allocation Improvement of student learning through the provision of systematic feedback Intrinsic Strengthening programs through self-improvement and self- determination Strengthening the bonds within the college community and fostering cooperation between units Generating continuous and ongoing dialogue about how student learning can be enhanced through program and service improvements Evaluating the unit’s contribution to achieving the district goals and strategic initiatives Extrinsic Resource allocation Improvement of student learning through the provision of systematic feedback Intrinsic Strengthening programs through self-improvement and self- determination Strengthening the bonds within the college community and fostering cooperation between units Generating continuous and ongoing dialogue about how student learning can be enhanced through program and service improvements Evaluating the unit’s contribution to achieving the district goals and strategic initiatives

3. Types of Program Review Instructional Annual Comprehensive Student Services Administrative Instructional Annual Comprehensive Student Services Administrative

4. Types of Research Support Data = Numbers Information Services Raw data from Datatel Reports organized around regular requests such as registration statistics Institutional Reporting Current term enrollments Current term FTES Distributions Current term assignments Scheduling patterns Institutional Research Past term(s) data including all in Institutional Reporting’s list above CCCCO Referential MIS files Data = Numbers Information Services Raw data from Datatel Reports organized around regular requests such as registration statistics Institutional Reporting Current term enrollments Current term FTES Distributions Current term assignments Scheduling patterns Institutional Research Past term(s) data including all in Institutional Reporting’s list above CCCCO Referential MIS files

5. Types of Research Support Information = Data in a meaningful context, organized for decision making Institutional Reporting Summaries or organized reports about the current term Institutional Research Reports on Employment in the Region Summaries or organized reports about past terms Matriculation reports Reports on Student Engagement and/or satisfaction Institutional Effectiveness Reports to the State, Federal Government and Accrediting Agencies Reports such as the McIntyre Environmental Scans Discipline Program Review Documents Information used to “lobby” administration Information = Data in a meaningful context, organized for decision making Institutional Reporting Summaries or organized reports about the current term Institutional Research Reports on Employment in the Region Summaries or organized reports about past terms Matriculation reports Reports on Student Engagement and/or satisfaction Institutional Effectiveness Reports to the State, Federal Government and Accrediting Agencies Reports such as the McIntyre Environmental Scans Discipline Program Review Documents Information used to “lobby” administration

6. Types of Research Support Analysis = Projections or qualitative observations often resulting in recommendations aligned with the college mission, community need or transfer requirements Institutional Research Employment projections Institutional Effectiveness Scenarios such as those found in the McIntyre Report Statewide and local economic and demographic projections Analysis of trends in Education law and politics Analysis of trends in Accreditation requirements Units undergoing Program Review Each unit must determine for itself which data, information and the analysis from other respected sources is relevant for its review. Ultimately the unit must prepare its own analysis of its unit and its future as a capstone to data, information and analysis provided by others. Units are not expected to generate their own data or information. They are expected to be able to identify additional data or information needs and advise Institutional Effectiveness and Research of those needs. As resources permit units will be provided with data and information or Institutional Effectiveness and Research will collaborate with units in collection of data. Analysis = Projections or qualitative observations often resulting in recommendations aligned with the college mission, community need or transfer requirements Institutional Research Employment projections Institutional Effectiveness Scenarios such as those found in the McIntyre Report Statewide and local economic and demographic projections Analysis of trends in Education law and politics Analysis of trends in Accreditation requirements Units undergoing Program Review Each unit must determine for itself which data, information and the analysis from other respected sources is relevant for its review. Ultimately the unit must prepare its own analysis of its unit and its future as a capstone to data, information and analysis provided by others. Units are not expected to generate their own data or information. They are expected to be able to identify additional data or information needs and advise Institutional Effectiveness and Research of those needs. As resources permit units will be provided with data and information or Institutional Effectiveness and Research will collaborate with units in collection of data.

7. Instructional Program Review Comprehensive Goals – focuses on each discipline’s courses, pedagogy, assessment plan, and future goals and objectives related to the improvement of student learning (4 year cycle). Forms Data Annual Goals – Resource allocation and assessment update Forms Data Comprehensive Goals – focuses on each discipline’s courses, pedagogy, assessment plan, and future goals and objectives related to the improvement of student learning (4 year cycle). Forms Data Annual Goals – Resource allocation and assessment update Forms Data

8. Instructional Program Review Data for Comprehensive (con’t) Demographic information (ethnicity, gender, age group, enrollment status) Course Enrollments by time of day offered Counts of sections offered Student Retention by course Student Success rates (two measures) Data for Comprehensive (con’t) Demographic information (ethnicity, gender, age group, enrollment status) Course Enrollments by time of day offered Counts of sections offered Student Retention by course Student Success rates (two measures)

9. Instructional Program Review Data for Comprehensive (con’t) FTES generated by course Weekly Student Contact Hours by course Full-Time Equivalent Faculty by course Percentages of student enrollments taught by Full-Time and Part-Time faculty by course Data for Comprehensive (con’t) FTES generated by course Weekly Student Contact Hours by course Full-Time Equivalent Faculty by course Percentages of student enrollments taught by Full-Time and Part-Time faculty by course

10. Instructional Program Review Data for Comprehensive (con’t) Class enrollment by course and term Grade point averages and grade distributions by course and term Student success rates (two measures) by Demographics by term Degrees and certificates awarded, if applicable Data for Comprehensive (con’t) Class enrollment by course and term Grade point averages and grade distributions by course and term Student success rates (two measures) by Demographics by term Degrees and certificates awarded, if applicable

11. Instructional Program Review Data for Comprehensive (con’t) Class enrollment by course and term Grade point averages and grade distributions by course and term Student success rates (two measures) by Demographics by term Degrees and certificates awarded, if applicable Data for Comprehensive (con’t) Class enrollment by course and term Grade point averages and grade distributions by course and term Student success rates (two measures) by Demographics by term Degrees and certificates awarded, if applicable

12. Instructional Program Review, Data Examples

23. Instructional Program Review Data for Annual Class enrollment by course and term Retention and Success Rates for each course FTEF, WSCH for each course Data for Annual Class enrollment by course and term Retention and Success Rates for each course FTEF, WSCH for each course

24. Student Services Goals Resource allocation State program goals and align future goals with the College’s mission and goals. Collect and analyze data on key performance indicators, service area outcomes and student learning outcomes, program activities, and accomplishments. Examine and document the effectiveness of student support services. Develop recommendations and strategies concerning future program directions and needs (e.g. budget, staffing, and resources). Assure the accuracy of program information. Comply with Accreditation Standards, Federal and State law, Title 5, Student Equity, VTEA, matriculation (including prerequisite and co-requisite standards), ADA (American with Disabilities Act), and other legal or certification requirements. Forms Data – Developed by units Goals Resource allocation State program goals and align future goals with the College’s mission and goals. Collect and analyze data on key performance indicators, service area outcomes and student learning outcomes, program activities, and accomplishments. Examine and document the effectiveness of student support services. Develop recommendations and strategies concerning future program directions and needs (e.g. budget, staffing, and resources). Assure the accuracy of program information. Comply with Accreditation Standards, Federal and State law, Title 5, Student Equity, VTEA, matriculation (including prerequisite and co-requisite standards), ADA (American with Disabilities Act), and other legal or certification requirements. Forms Data – Developed by units

25. Administrative Units Goals – Same as Student Services Forms Data - First year the departments were asked to develop assessment plans and begin compiling data for reasonable measures of their department’s effectiveness Goals – Same as Student Services Forms Data - First year the departments were asked to develop assessment plans and begin compiling data for reasonable measures of their department’s effectiveness

26. Focus Group Analysis Findings The review process was useful because it helped units take stock of effectiveness and what resources were needed. It moved units to be more comprehensive about identifying resources and tying those resources to programmatic needs. Need more training on budget and data analysis Believe improvements are being made in teaching and learning as a result of program review Believe that the resource allocation process is critical to the fate of some improvements Changes as a result Vice Presidents for Business Services engaged in working with units on annual review and budget plans More data given at the beginning of the process Enhanced training on analysis Increasingly simplified forms Findings The review process was useful because it helped units take stock of effectiveness and what resources were needed. It moved units to be more comprehensive about identifying resources and tying those resources to programmatic needs. Need more training on budget and data analysis Believe improvements are being made in teaching and learning as a result of program review Believe that the resource allocation process is critical to the fate of some improvements Changes as a result Vice Presidents for Business Services engaged in working with units on annual review and budget plans More data given at the beginning of the process Enhanced training on analysis Increasingly simplified forms

27. Integration with Budget and Planning

28. Norco Campus Ranking Criteria CriterionDescriptionPoints Enrollment GrowthPotential for new position to contribute to enrollment growth as evidenced by high ratio of WSCH:FTEF or other documentation 20 Student Persistence Potential for new position to contribute to increased term-to-term persistence, as evidenced by high current persistence rates (or proxy measures, such as student success rates) in discipline or potential for improvement in persistence associated with adding a full-time position. Also applicable to positions in counseling, basic skills and support services. 20 Increased Student Success Potential for new position to contribute to increased student success as evidenced by high student success rates in discipline or the potential for a new faculty position to contribute to higher success rates. 20 Student Learning Outcomes Maximum points awarded to disciplines (service areas) that have both identified SLOs and begun to measure their achievement. Mid-range points awarded to those that have defined SLOs but not yet begun to measure them. Zero points to disciplines and service areas that either have not defined SLOs or have not turned in program reviews. 10 Improved Quality of Student Experience A somewhat subjective criterion designed to address disciplines and service areas for which quantitative data pertaining to above criteria are not readily available. Points awarded based on the extent to which requesting unit has demonstrated that adding the position requested will enhance the quality of the student experience at Norco. 10 Current Full-time Staffing Extent to which department or service area depends upon part-time positions to deliver services. Maximum points awarded for disciplines with no full-timers; near maximum points awarded for those with only one FT position currently. 20

29. Accreditation Response Continue with current program review, add SLOs for all programs (we have them for all courses and GE) Need better integration with budget (Norco process was in first year of cycle) Continue with current program review, add SLOs for all programs (we have them for all courses and GE) Need better integration with budget (Norco process was in first year of cycle)

30. Attitude Change: Empowerment Units get advice about lobbying for resources Units are provided extensive support while going through the process, particularly in the beginning Every round of program review utilizes increasing analytical skills Units analyze their own data to improve student learning and enrollment management Units can confirm or refute operating assumptions Units get advice about lobbying for resources Units are provided extensive support while going through the process, particularly in the beginning Every round of program review utilizes increasing analytical skills Units analyze their own data to improve student learning and enrollment management Units can confirm or refute operating assumptions