Comprehensive Evaluation: Institutional Effectiveness Committee Recommendations Presentation to College Council Office of Institutional Effectiveness & Institutional Effectiveness Committee November 11, 2017

Background Evaluate Continuous Improvement Processes (3 year cycle) Program Review Student Learning Outcomes (SLO)/Administrative Unit Outcomes Assessment Planning College Council reviewed/informed of: Evaluation plan (fall 2016) Data collection efforts (spring 2017) Preliminary themes presented (spring 2017)

Discussion Plan Institutional Effectiveness Committee (fall 2017) Review detailed results Make recommendations for improvement Review Institutional Effectiveness Committee recommendations with campus College Council (November 2017) Planning councils (December 2017- February 2018) Academic Senate – 10+1 (December 2017- February 2018) Classified Senate (December 2017- February 2018) Final Recommendations Academic Senate 10+1 (March 2018) College Council (March 2018)

Evaluation Process Two component evaluation: Content: Focus Groups with each of the four wing planning councils, Academic Senate, & Classified Senate Campus-wide perception and opinion survey Content: Program Review process & structure SLO/AUO Assessment process & structure Planning process & structure Support Timeframe Impact Communication TracDat

Main Themes & Considerations Desire to keep processes the same for a period of time Lessen the culture of fear Dialogue & utility of processes needs improvement AUO data collection, peer review, staff/management hiring, staff development Some indication process cycles are too short Integration across processes not easily observable College goals not always driving plans, linked afterwards Broaden participation, communication and training

Program Review Develop a stronger mechanism for departments to identify and request department specific data for instructional programs. Consider redesigning current form, timeline for requests and communications about the process. Improve peer review process for both instructional and support departments. Consider process, participants, training and timeline. Redesign SLO/AUO synthesis inside program review to be more meaningful and less redundant. For instructional wing, consider SLO area to be more focused on program SLO evaluation. Improve prompts and support for CTE faculty completing both the biennial and core indicator areas in Program Relevancy.

SLO/AUO Assessment No changes to course SLO process. Evaluate each department’s AUOs/KPIs to ensure they are measurable and meaningful. Clarify AUO process, three-year cycle and training materials to emphasize that AUOs/KPIs can change annually as needed.

Planning/Annual Resource Requests (ARRs) Better articulate the relationship between college goals/objectives/priorities, departmental planning strategies and SLOs/AUOs. Strengthen planning strategies to be less focused on resource needs and more focused on improvement. Improve the integration of categorical, ancillary and ASOCC monies into the planning/ARR process. Improve classified and management staffing ARR process and prioritization of positions. Consider linkage to staffing plans or as a separate process. Clarify the prioritization and funding mechanism for Professional Development ARRs. Increase communication about the results of Annual Resource Requests to departments and the campus.

TracDat Improve navigation and streamlining of program review module. Develop training videos to help on demand support. Improve tracking reports of processes in TracDat. Support further integration of technology into TracDat (e.g., Canvas, Microsoft BI).

Overall – All Processes Increase classified staff participation and collaboration in all phases of program review, AUOs and the planning process. Dialogue about results of program review, SLOs and planning strategies outside of the process. Consider department, division or wing meetings as possible discussion arenas. Consider length and alignment of process cycles. Ongoing training of processes to keep purpose and outcomes in the forefront.

Comprehensive Evaluation Report Data Analyzed by Office of Institutional Effectiveness Click Here for Details

Comprehensive Evaluation: Preliminary Themes from Focus Groups Presented to College Council May 2, 2017

Preliminary Themes: Program Review Improved since prior cycle with data analysis and distinction between Instruction/Support wings More program-specific data support needed CTE areas duplicating external accreditation Tying CSLO/AUO results into program review not clear/seems redundant Peer review process needs refining (training, timing, incorporating feedback) Classified staff involved to varying degrees (mostly not involved or only on front end; no closing the loop)

Preliminary Themes: SLO/AUO Assessment SLOs compliance-focused and not being used in meaningful way Culture of fear surrounding assessment Feels punitive, so results don’t show need for improvement AUOs useful, but cumbersome Data collection problematic (sustainability, difficulty getting data, unsure of data needed) Need help developing assessments/KPIs

Preliminary Themes: Planning Integration across processes not easily observable College goals not always driving plans; typically addressed after strategies developed ARR and BSB processes working to get resources Perception some use BSBs in lieu of ARRs Staff/Management ARR prioritization/funding not meeting department demand and not reflecting “critical” or “mass” campus needs ARR decisions not being communicated widely

Preliminary Themes: Overall Overall perception that only minor tweaks are needed in processes Desire to keep processes the same for a period of time (changes burdensome to re-learn) Provide ongoing training for all processes for all constituents (keep fresh) Lessen culture of fear Dialogue outside of processes needs improvement Concerns about broad enough participation within departments (“experts” within departments do it all) Results from program review/assessment primarily being used for ARR justification

Preliminary Themes: TracDat PROS: Data collected in central location Reporting/summarizing capabilities Better than prior MS office based collection CONS: Navigation within TracDat not intuitive Reports hard to read & access Involves continuous training to use