1. Creating and Revising Curriculum: Program Review
Randy Beach, ASCCC Executive Committee
Michael Bowen, Ventura College
Thais Winsome, Mission College

2. Description
Course and program self-assessment should be the heart of an academic program review, which can lead faculty to a thoughtful, data-informed evaluation of their courses and course sequences leading to certificates and degrees. In this breakout session, participants will discuss how program review can facilitate those conversations and decisions, what elements should be present in a program review process to make it useful for curriculum review, and how accreditation standards are an important consideration. One particular focus will be the role of program student learning outcomes as an important part of program review.  (Effective Practices Strand)

3. Why are you here today? What do you hope to learn today?
What are you struggling with locally?

4. Why Do Program Review? Self-evaluation and assessment
Resource allocation (budget planning)
Master planning (facilities, technology, education, etc..)
Other long and short-term planning and institutional effectiveness, grants
Accreditation compliance

5. Program Review as Self-Evaluation
Reflections and research into past performance provide guidance for future planning
Qualitative and quantitative data analysis helps tell your narrative
Useful tool for self-evaluation in the context of shifting institutional priorities
Critical tool in the context of federal/state/local initiatives

6. Curriculum and Program Review: Benefits
Provides a structured approach to self-evaluation of curriculum
Integrates data, including SLO data, into curriculum discussions
Provides a way to support curriculum decisions with resource allocation
Connects curriculum development to long term strategic and educational master planning

7. Curriculum Components in Program Review
Student learning outcomes data disaggregated by demographics
Student learning outcomes data disaggregated by course/section attributes
Review of code (SAM/CIP/SOC/TOP) assignment and alignment (especially in CTE areas)
Review of staffing needs and professional development to support program growth
Any others?

8. Curriculum Components in Program Review
Listing of courses and programs
Last date of course and program review/revision
Listing of course and program outcomes
Completion rates for programs and courses
Student program/course achievement data disaggregated by demographics and section attributes

9. Questions to Ask: Student Achievement
What percent of students are completing your courses?
How many are completing the programs?
Is there a demographic aspect to completion rates?
How diverse is your student population (gender, ethnicity, fulltime vs. part-time)? Is the program welcoming to nontraditional students?
Is there a difference in completion rates for online vs. on-ground?
What is your baseline for “good enough”?

10. Questions to Ask: Faculty Efficacy
Does the diversity of your faculty support student success and the curriculum?
What professional development strategies exist that support curriculum?
How do the qualifications of your faculty serve the growth and development of your program? What is the role of part-time faculty, advisory committees, administrators, and others in the development of curriculum?
Others?
Not sure of the thought behind this slide.

11. Questions to Ask: Resource Allocation
What elements of your curriculum development require new or augmented resources?
Is additional research needed?
Are new faculty or support personnel necessary for specialized curriculum?
Does your expanding curriculum require ne instructional or support equipment?
Are the resource allocations for existing instructional materials sufficient to support the curriculum?
Others?
Not sure of the thought behind this slide.

12. CSLO vs PLO and Curriculum Development
What is the interplay between course outcomes and program outcomes?

13. ILOs, GELOs, PLOs: A Reminder
Institutional Learning Outcomes (ILOs) derive from your mission and reflect the characteristics of an educated person
General Education Learning Outcomes (GELOs) are the philosophical/pedagogical foundation for transfer agreements and can assess transfer readiness
Program learning outcomes support overall ILOs and GELOs
Review the ILOs and GELOs when designing PLOs

14. Starting with the end in mind
Identify the skills, abilities, knowledge needed for student success in workplace/transfer/course sequence
Work backward to create/revise program focus and pathway
Choose/create courses to support program outcomes
Work with external groups (advisory/university)
Course outlines should align with the program outcomes to ensure meaningful outcomes assessment
Foundational principle of “Guided Pathways” model

15. Outcomes: Beginning with the End in Mind
Outcomes state what we want students to know and do; the pathway grows around those ideas
Randy

16. Outcomes, Program Review, Accreditation
ER 11: Student Learning and Achievement
“...The institution publishes for each program the program’s expected student learning and any program-specific achievement outcomes. Through regular and systematic assessment, it demonstrates that students who complete programs, no matter where or how they are offered, achieve the identified outcomes...”

17. Outcomes, Program Review, Accreditation
IB1 “The institution demonstrates a sustained, substantive and collegial dialog about student outcomes, student equity, academic quality, institutional effectiveness, and continuous improvement of student learning and achievement.”
IB2 “The institution defines and assesses student learning outcomes for all instructional programs and student and learning support services.”
IB5 “The institution assesses accomplishment of its mission through program review and evaluation of goals and objectives, student learning outcomes, and student achievement. Quantitative and qualitative data are disaggregated for analysis by program type and mode of delivery.”

18. Why Outcomes Assessment in Program Review?
Outcomes assessment incorporates the students’ “voices” and needs
Strengthens curriculum changes and resource allocation justification with data; serves as a narrative for budget development
Ties program improvements to quantifiable data; allows for short or long-term action plans based on outcomes data
Provides important evidentiary documentation for accreditation
Not sure of the thought behind this slide.

19. Outcomes Analysis in Program Review
Are your program outcomes aligned with the course outcomes?
What assessment tool(s) is used to assess program or course SLOs?
Does the SLO accurately reflect exit expectations of students enrolled in the course? The program?
How do the CSLOs map to your course objectives in core program courses?
How do your PLOs capture the breadth and depth of the program? Are the PLOs aligned to ILOs?
Has your discipline developed a rubric to measure student proficiency in this SLO?

20. Program Review & the 2-Year CTE Review
CTE programs required to review every two years per ED Code 78016(a)
(a) Every vocational or occupational training program offered by a community college district shall be reviewed every two years by the governing board of the district to ensure that each program, as demonstrated by the California Occupational Information System, including the State-Local Cooperative Labor Market Information Program established in Section of the Unemployment Insurance Code, or if this program is not available in the labor market area, other available sources of labor market information, does all of the following

21. Program Review & the 2-Year CTE Review
(1) Meets a documented labor market demand.
(2) Does not represent unnecessary duplication of other manpower training programs in the area.
(3) Is of demonstrated effectiveness as measured by the employment and completion success of its students.
(b) Any program that does not meet the requirements of subdivision (a) and the standards promulgated by the governing board shall be terminated within one year. (c) The review process required by this section shall include the review and comments by the local Private Industry Council … which review and comments shall occur prior to any decision by the appropriate governing body.

22. Examples of CTE Review
Mendocino College
Southwestern College

25. Four-Code Alignment Project
ASCCC and CCCCO effort to align California Codes to Federal Codes
SOC/CIP/SAM/TOP
Ensures proper reporting for financial aid and CTE/SWP funding metrics
Faculty-driven process; lots of opportunity for discussion
Provides a structure for colleges to walk from occupations backwards to program and course design
Establishes where the course is in the program in terms of capstone courses, foundation course
Code-alignment process helps college identify meta-majors in Guided Pathways models
“Code Check” should be routine part of Program Review
Are the occupations the programs serve still relevant?
Are the courses properly assigned as introductory, intermediate and capstone

26. Program Review Processes
If process is prohibitive or convoluted, where is the value in curriculum development?
How much faculty time is devoted to program review processes?
Does the program review software “talk to” the curriculum software?
Do you have a successful software solution you can share?

27. Next Steps… Know your processes
Where does your program review information go?
How does it get prioritized and included in the planning and budgeting process?
What is the reporting back structure?
Contribute to improvement
What committee(s) oversee the process on your campus?
Does your curriculum committee have a mechanism to provide input?

28. Questions? Randy Beach (rbeach@swccd.edu)
Michael Bowen
Thais Winsome