Creating and Revising Curriculum: Program Review Randy Beach, ASCCC Executive Committee Michael Bowen, Ventura College Thais Winsome, Mission College

Description Course and program self-assessment should be the heart of an academic program review, which can lead faculty to a thoughtful, data-informed evaluation of their courses and course sequences leading to certificates and degrees. In this breakout session, participants will discuss how program review can facilitate those conversations and decisions, what elements should be present in a program review process to make it useful for curriculum review, and how accreditation standards are an important consideration. One particular focus will be the role of program student learning outcomes as an important part of program review.  (Effective Practices Strand)

Why are you here today? What do you hope to learn today? What are you struggling with locally?

Why Do Program Review? Self-evaluation and assessment Resource allocation (budget planning) Master planning (facilities, technology, education, etc..) Other long and short-term planning and institutional effectiveness, grants Accreditation compliance

Program Review as Self-Evaluation Reflections and research into past performance provide guidance for future planning Qualitative and quantitative data analysis helps tell your narrative Useful tool for self-evaluation in the context of shifting institutional priorities Critical tool in the context of federal/state/local initiatives

Curriculum and Program Review: Benefits Provides a structured approach to self-evaluation of curriculum Integrates data, including SLO data, into curriculum discussions Provides a way to support curriculum decisions with resource allocation Connects curriculum development to long term strategic and educational master planning

Curriculum Components in Program Review Student learning outcomes data disaggregated by demographics Student learning outcomes data disaggregated by course/section attributes Review of code (SAM/CIP/SOC/TOP) assignment and alignment (especially in CTE areas) Review of staffing needs and professional development to support program growth Any others?

Curriculum Components in Program Review Listing of courses and programs Last date of course and program review/revision Listing of course and program outcomes Completion rates for programs and courses Student program/course achievement data disaggregated by demographics and section attributes

Questions to Ask: Student Achievement What percent of students are completing your courses? How many are completing the programs? Is there a demographic aspect to completion rates? How diverse is your student population (gender, ethnicity, fulltime vs. part-time)? Is the program welcoming to nontraditional students? Is there a difference in completion rates for online vs. on-ground? What is your baseline for “good enough”?

Questions to Ask: Faculty Efficacy Does the diversity of your faculty support student success and the curriculum? What professional development strategies exist that support curriculum? How do the qualifications of your faculty serve the growth and development of your program? What is the role of part-time faculty, advisory committees, administrators, and others in the development of curriculum? Others? Not sure of the thought behind this slide.

Questions to Ask: Resource Allocation What elements of your curriculum development require new or augmented resources? Is additional research needed? Are new faculty or support personnel necessary for specialized curriculum? Does your expanding curriculum require ne instructional or support equipment? Are the resource allocations for existing instructional materials sufficient to support the curriculum? Others? Not sure of the thought behind this slide.

CSLO vs PLO and Curriculum Development What is the interplay between course outcomes and program outcomes?

ILOs, GELOs, PLOs: A Reminder Institutional Learning Outcomes (ILOs) derive from your mission and reflect the characteristics of an educated person General Education Learning Outcomes (GELOs) are the philosophical/pedagogical foundation for transfer agreements and can assess transfer readiness Program learning outcomes support overall ILOs and GELOs Review the ILOs and GELOs when designing PLOs

Starting with the end in mind Identify the skills, abilities, knowledge needed for student success in workplace/transfer/course sequence Work backward to create/revise program focus and pathway Choose/create courses to support program outcomes Work with external groups (advisory/university) Course outlines should align with the program outcomes to ensure meaningful outcomes assessment Foundational principle of “Guided Pathways” model

Outcomes: Beginning with the End in Mind Outcomes state what we want students to know and do; the pathway grows around those ideas Randy

Outcomes, Program Review, Accreditation ER 11: Student Learning and Achievement “...The institution publishes for each program the program’s expected student learning and any program-specific achievement outcomes. Through regular and systematic assessment, it demonstrates that students who complete programs, no matter where or how they are offered, achieve the identified outcomes...”

Outcomes, Program Review, Accreditation IB1 “The institution demonstrates a sustained, substantive and collegial dialog about student outcomes, student equity, academic quality, institutional effectiveness, and continuous improvement of student learning and achievement.” IB2 “The institution defines and assesses student learning outcomes for all instructional programs and student and learning support services.” IB5 “The institution assesses accomplishment of its mission through program review and evaluation of goals and objectives, student learning outcomes, and student achievement. Quantitative and qualitative data are disaggregated for analysis by program type and mode of delivery.”

Why Outcomes Assessment in Program Review? Outcomes assessment incorporates the students’ “voices” and needs Strengthens curriculum changes and resource allocation justification with data; serves as a narrative for budget development Ties program improvements to quantifiable data; allows for short or long-term action plans based on outcomes data Provides important evidentiary documentation for accreditation Not sure of the thought behind this slide.

Outcomes Analysis in Program Review Are your program outcomes aligned with the course outcomes? What assessment tool(s) is used to assess program or course SLOs? Does the SLO accurately reflect exit expectations of students enrolled in the course? The program? How do the CSLOs map to your course objectives in core program courses? How do your PLOs capture the breadth and depth of the program? Are the PLOs aligned to ILOs? Has your discipline developed a rubric to measure student proficiency in this SLO?

Program Review & the 2-Year CTE Review CTE programs required to review every two years per ED Code 78016(a) 78016. (a) Every vocational or occupational training program offered by a community college district shall be reviewed every two years by the governing board of the district to ensure that each program, as demonstrated by the California Occupational Information System, including the State-Local Cooperative Labor Market Information Program established in Section 10533 of the Unemployment Insurance Code, or if this program is not available in the labor market area, other available sources of labor market information, does all of the following

Program Review & the 2-Year CTE Review (1) Meets a documented labor market demand. (2) Does not represent unnecessary duplication of other manpower training programs in the area. (3) Is of demonstrated effectiveness as measured by the employment and completion success of its students. (b) Any program that does not meet the requirements of subdivision (a) and the standards promulgated by the governing board shall be terminated within one year. (c) The review process required by this section shall include the review and comments by the local Private Industry Council … which review and comments shall occur prior to any decision by the appropriate governing body.

Examples of CTE Review Mendocino College Southwestern College

Four-Code Alignment Project ASCCC and CCCCO effort to align California Codes to Federal Codes SOC/CIP/SAM/TOP Ensures proper reporting for financial aid and CTE/SWP funding metrics Faculty-driven process; lots of opportunity for discussion Provides a structure for colleges to walk from occupations backwards to program and course design Establishes where the course is in the program in terms of capstone courses, foundation course Code-alignment process helps college identify meta-majors in Guided Pathways models “Code Check” should be routine part of Program Review Are the occupations the programs serve still relevant? Are the courses properly assigned as introductory, intermediate and capstone

Program Review Processes If process is prohibitive or convoluted, where is the value in curriculum development? How much faculty time is devoted to program review processes? Does the program review software “talk to” the curriculum software? Do you have a successful software solution you can share?

Next Steps… Know your processes Where does your program review information go? How does it get prioritized and included in the planning and budgeting process? What is the reporting back structure? Contribute to improvement What committee(s) oversee the process on your campus? Does your curriculum committee have a mechanism to provide input?

Questions? Randy Beach (rbeach@swccd.edu) Michael Bowen (mbowen@vcccd.edu) Thais Winsome (thais.winsome@missioncollege.edu)