360 Degrees: Conducting a Comprehensive Evaluation of Your Integrated Planning Processes Bri Hays Jill Baker San Diego Mesa College RP Conference April 11, 2014

Introductions and Context What is one thing you would like to learn in today’s session?

Session Learning Outcomes Describe 360-degree feedback approach and applications to integrated planning Apply process evaluation principles to integrated planning Discuss research methods appropriate to a comprehensive evaluation of integrated planning processes Outline an evaluation plan for college integrated planning processes Identify strategies for facilitating the use of data for action planning and improvement

360 Feedback Approach Applying the 360 approach to process evaluation Multi-source feedback provides a more comprehensive perspective of performance Feedback from various stakeholders at differing levels Foundations in the business world (HR) and I/O psychology Applying the 360 approach to process evaluation Evaluand Information Source 1 Information Source 2 Information Source 3 Information Source 4 Information Source 5

The Impetus for Process Evaluation Is the process working as intended? How can we improve the process for the end-user? Could additional efficiencies be achieved?

What is your definition of integrated planning?

Integrated Planning Is… “…the linking of vision, priorities, people, and the physical institution in a flexible system of evaluation, decision-making and action. It shapes and guides the entire organization as it evolves over time and within its community.” –Society for College and University Planning

Integrated Planning Process Evaluation: What You Need Process Model List of Key Stakeholders Outline of Process Goals List of Evaluation Research Questions Process Timeline or Calendar

Getting Started: What’s The Model Step 1 Step 2 Step 3 Step 1 Step 2 Step 3 Step 1 Step 2 Step 3 Step 4 Step 5

Integrated Planning Components Program Review Outcome Assessment Resource Request Process Mission and Vision Institutional Plans (Ed Master Plan, Facilities Plan, Tech Plan, HR plan) Prioritization Process (Committees) Allocation Process Follow-Up/Evaluation

Sample IP Model

Integrated Planning Stakeholders Students Prioritization Committees Faculty Managers Staff Community

Identifying Integrated Planning Process Goals What are the overarching goals of the college’s integrated planning process? Increase transparency in decision-making Clearly link data, planning, and resource allocation Inform the strategic planning priorities What are the operational objectives of the integrated planning process? Increase efficiency of the integrated planning process Eliminate redundancies Align planning and resource allocation calendars

Outlining Process Evaluation Research Questions & Goals Was the process carried out as intended? How effective and efficient was the flow of information? Did the correct information make its way to the appropriate end point? Was the information used as intended? How do stakeholders perceive the process? Did the process/activities work as planned? Were resources allocated according to the plan? How can the process be improved?

Framing the Evaluation What are the goals of the evaluation? What data are needed to address the evaluation goals? Who are the key stakeholders in the process? How do the results of the evaluation need to be reported out and to whom? How will the information gained from the evaluation be used? Program Review Resource Allocation Strategic Planning

Setting Evaluation Goals Set goals for the evaluation itself based on the goals/objectives of the process (or process redesign)

A Mixed Methods Approach What methods will you utilize to assess the integrated planning process? Examples include: Surveys Focus Groups Campus-wide forums Committee feedback or debriefing sessions Program review and resource request content Institutional effectiveness data/key performance indicators

Process Goal/Objective Evaluation Activities Person(s) Responsible The Evaluation Plan The evaluation plan should clearly illustrate the alignment of process goals/objectives, evaluation goals, evaluation activities, and the timeframe during which activities will take place Process Goal/Objective Evaluation Plan Goal Evaluation Activities Timeline Person(s) Responsible Improve transparency of resource request prioritization Assess stakeholder perceptions of transparency Survey of faculty, staff, and students Focus group with department chairs and supervisors Focus group with resource allocation committees Spring 2013 Campus Based Researcher; Dean of IE; Program Review Committee Clarify connections between planning, data, and resource allocation Assess the program review utilization of data and links between unit and college goals Content analysis of unit goal form data TaskStream reports on goal mapping Summer 2013 Dean of IE; Campus Based Researcher

Sample Evaluation Plan

Conveying the Evaluation Results What deliverables are best suited to your college? How can the evaluation results be turned into action items? What structures and processes are in place at your college to ensure the results are reported out and action is taken?

Sample Results Summary Table

Closing the Loop Formative Evaluation Cycle Plan Implement Assess Analyze Identify Action Items Formative Evaluation Cycle

From Data to Action Analyze all information collected and organize it by topic or component Provide a conceptual framework for moving forward with specific recommendations Recommendations, after being vetted, become goals for the next cycle… …Which inform the following year’s evaluation plan

Sample Evaluation Executive Summary

Paving the Way for Continuous Quality Improvement Start with a baseline Results of the baseline study inform changes to the integrated planning process for the following year Goals/objectives for the subsequent cycle are based on the results of the previous cycle’s evaluation Develop a calendar of activities to meet new goals and objectives—list person(s) or group(s) responsible Track progress toward goals/objectives Address the limitations of the previous cycle’s evaluation

Next Steps How do you integrate results into the larger assessment of the college’s institutional effectiveness? What process does your college have for assessing the effectiveness of resource allocation?

Thank You! Bri Hays Campus Based Researcher San Diego Mesa College bhays@sdccd.edu Jill Baker Dean of Institutional Effectiveness jibaker@sdccd.edu