Program Review, Service Unit Outcomes and Planning Eva Bagg, Associate Dean of Institutional Effectiveness Maria Narvaez, Planning Analyst July 8, 2009

Agenda Overview LBCC program planning & review Timeline Components of program planning & review How to begin developing mission statement, goals & SUOs Purpose & principles of assessment Unit assignment Sneak preview of TracDat Questions/Sign-up

Overview Today’s workshop will –Provide context for SUOs –Guidance to prepare content for program planning and review Customized Training sessions with each administrative unit will follow today’s workshop –Sign up for a time that works for your unit

LBCC program planning & review: a historical perspective Program Review – every 3 yrs Dept. Planning -No formal process SUOs Non-Instructional Dept. Planning -process had problems Program Review – every 6 yrs SLOs, But no assessment plans Instructional

The new LBCC program planning & review process Program ReviewPlanning Outcomes Assessment Student Success & Institutional Effectiveness

Timeline for first implementation of new process July – customized training sessions & units draft SUOs July/ August – feedback to units August – hands-on support to enter information into TracDat Program Reviews are DUE on Oct. 1 st * Program Plans/ Reviews submitted to Validation Teams and then to “inter-level” groups then to VP Level. * We hope most Admin Units will have their program reviews completed by College Day.

Program planning & review Mission statement Goals Purpose & priorities Service Unit Outcomes Assess and Evaluate Use assessment results to inform what to improve Design strategies for improvement Develop a plan Identify resources needed Plan to improve

Mission Statement A unit mission statement should: –identify the unit’s purpose –reflect how the unit supports a student learning environment –be distinctive –articulate the essential functions/activities of the unit –identify the primary stakeholders of the unit –align with LBCC’s mission See handout for examples

Goals Goals are: –overarching general statements that describe the department's strategic direction. –usually not measurable but rather, they represent coordinated efforts of intended improvements for the department’s processes –what the department plans to do and are reflective of long-term priorities –representative of consensus of staff aspirations for the program –primarily used for general planning and are used as the starting point to the development and refinement of outcomes. See handout for examples

How to get started (1) Meet with your unit to discuss and gain consensus on your unit mission –If your unit comprises multiple functional areas, make sure each area is represented when you meet. –Discuss the purpose of your unit (unit mission) What are the primary functions or activities of your unit? Who are your stakeholders? Who do you provide services to and/or who benefits from the services?

How to get started (2) Meet with your unit to discuss and gain consensus on your unit goals –Review the goals stated in your last program review. How far have you come to achieving these goals? Are they still relevant? –Decide which previous goals your unit will retain (and/or refine). –Write any new goals that state what you want to improve or modify about your unit in the next 3-5 years. (try to limit to 5)

What are Service Unit Outcomes? LBCC’s definition of a SUO:  Outcomes describe the desired quality (timeliness, accuracy, responsiveness, etc.) of key functions and services within the support or administrative unit.  SUOs are expressed as statements of what users of the service experience, receive or understand as a result of a given service interaction. These should address both process and satisfaction indicators.

What are Service Unit Outcomes? Types of SUOs Process outcomes –The level or volume of unit activity –The efficiency with which the unit’s processes are conducted –Compliance with external standards of “good practice in the field” or regulations –Effectiveness of the service in accomplishing intended outcome Client learning can also be an outcome, but the results are used to improve the effectiveness of the unit Satisfaction outcomes –Satisfaction-oriented administrative objects range from overall “client/customer satisfaction” as included on institutional graduating student surveys to more specific surveys of client satisfaction with components of individual services.

How to develop SUOs (1) Consider how your unit will determine the extent to which it accomplishes its goals (What will you assess?) –Aim to develop SUOs that make your unit goals measurable –Use existing data as much as possible –Focus on essentials you can address –Incorporate assessment procedures into routine unit operations –Use a cycle of assessment so that all outcomes will be assessed within a 3-year period rather than assessing all outcomes each year. –Try to limit to 2 SUOs per functional unit

How to develop SUOs (2) Understand the relationship between goals and outcomes – Goals are what we plan to do. They are helpful for guiding the implementation of improvement activities but are sometimes too general, broad, or vague for developing specific tools to assess the effectiveness of services provided in support of student learning. Goals state what we will do or cause to happen. – Outcomes are what we expect will happen when we do what we planned. At a minimum, outcome statements should describe stakeholder behaviors that staff would accept as evidence that the support office is performing its function. Where possible and meaningful, write SUOs beginning with the client as the noun in the statement (ie., the client will receive or experience…). Outcomes state what will happen as a result of what we do.

How to develop SUOs GoalsOutcomes Broad, future-oriented statementsBrief statements of results What the program or office is striving toward Describe behaviors of stakeholders Refer to processRefer to results of processes Used in policy making and program planning Used to assess effectiveness and provide improvement information. Understand the relationship between goals and SUOs, continued

Practice exercise: Goals or outcomes Faculty, staff, and students will have access to electronic resources from off- campus locations. Learning Resource Services will increase access to its resources by implementing remote user access to electronic resources.

Practice exercise: Goals or outcomes The Office of Institutional Effectiveness will provide planning, assessment, and evaluation services to meet the needs of the College. Faculty, administrators, and staff use information to make decisions or assess the effectiveness of their area.

Practice exercise: Goals or outcomes The Office of Risk Services will promote peace, order, and safety on campus by deterring and preventing criminal activity. The number of crimes reported on both the LAC and PCC campuses will decrease annually. Faculty, staff, and students will feel safe when they are on the campus.

Writing an Outcome Statement: The 3 M’s 1.Meaningful: How does the outcome support the departmental mission or goal? 2.Manageable: What is needed to foster the achievement of the outcome? Is the outcome realistic? 3.Measurable: How will you know if the outcome is achieved? What will be the assessment method? - StudentVoice

See handout for more SUO examples

Why assess? The ultimate goal of assessment is to pose questions and gather information to bring about improvement.

Why Assess? Student Support and Administrative units define their mission, establish goals identify service unit outcomes and develop plans to assess them, and collect assessment data so that key processes that meet the needs and expectations of students, parents, employers, faculty and other stakeholders can be improved on a continuous basis.

Principles of assessment Assessment at LBCC is To enhance quality of services provided in support of teaching and learning Faculty and staff driven Conducted at the level of the unit (not about individual evaluation) On-going and cyclical Done in order to improve

Assessment plan

Note: For each functional area within a unit, do only the first 3 columns for fall 2009.

Assessment plan Intended OutcomeAssessment Task Criteria /Expected Level of Achievement Results of AssessmentUse of Data/Plan Write the intended outcome in this column. Student Learning Outcomes (SLOs) are identified in the course outline of record. Program Level Outcomes, either SLOs or service unit outcomes (SUOs) are identified statements for a program/area that will be assessed to measure the effectiveness of the program/unit. Explain the assessment task for each outcome in this column. Make sure that a balance of direct and indirect assessment methods are used program-wide. Describe what the assessment is, how the assessment task will be implemented, who will be responsible for the task, when the assessment will be conducted, and where activities will be carried out. The expectations for success should be explained in this column. Establish a minimum expected score for achieving the outcome (e.g. %, fraction, actual number); determine the acceptable success standard (e.g. satisfactory, a passing grade, narrative indicator); as well as the students/clients to be included (e.g. successful completers, all queried, random sample). This information initially will be based upon reflection and discussion among necessary parties. Later, consideration of previous outcomes assessment effort(s), external reports (if available) should be used for this determination. Internal and external conditions should be considered. (Conduct and document assessment activities.) Once the assessment data is collected analyze the information through small group discussion and in relationship to previous assessment data (if available). Report the actual results and compare with the original expectations in this column. Highlight key findings from the data. Develop supportable conclusions from this information. Determinations will then be made on what action should be taken. Report here what the department/program/unit members have done or are doing as a result of the findings. Identify any changes or steps, who has made or is making the modifications and indicate the time frame for any changes to improve the outcome. The actions should generally be substantive and detailed; although there are times when no action is necessary and this can be stated as well. Indicate what the data explains about the process and what resources are necessary to implement the needed program or course improvements to student learning. ( Conduct follow-up assessment to determine the effectiveness of the program or course improvement.)

Administrative Units Completing Program Plan/ Program Review Superintendent-President's Office Institutional Effectiveness Community Relations and Marketing Academic Affairs Academic Services Administrative Services Risk Services Facilities Fiscal Services Instructional and Information Technology Services Human Resources Classified Academic Professional Development Student Services LBCC Auxiliary Financial Aid/Veteran Affairs Admissions & Records Counseling/ Student Support Development Student Affairs Outreach and Recruitment Economic & Resource Development Economic Development Workforce Development Institutional Grants

Your Unit Assignment Pull together a work group for your unit –Composition: Ideally, everyone (directors, manager, supervisors, classified staff) should be involved; however, if this is not feasible, ensure that there is representation for: each functional area classified staff if applicable, for each campus and/ or for each shift

Work group assignment Answer two questions about your unit: –What do you do? (core functions & services) You may want to start by having each functional rep describe their core function. From a comprehensive list of all functions and services, distill the core functions and services of the department. –What do you hope to improve or modify in the next 3-5 years? Review goals from the last Program Review. Are they still relevant? Draft mission statement Draft goals for next 3-yr cycle.

Work Group Assignment For each functional area within your unit Draft SUOs –Start by jotting down possible results of your goals ( make your goals measurable) (Optional) For each of your SUOs, complete the “Assessment Task” and “Criteria for Success” columns of the Assessment Plan template

End Products from Work Group Draft department mission statement Draft department goals Draft SUOs (for each functional area within your department) Please have these ready and available for the customized training session. Note: We will send you a template in which to capture the above end products from your work group.

Customized Training Feedback and refinement of mission, goals and SUOs Assessment plan Introduction toTracDat –Identify point person in unit to enter info into TracDat

Program/Unit description screen

Goals screen

Outcomes screen

Questions? Sign up for customized training