The assessment process For Administrative units Steve Culver Director, Assessment and Analytics Fall 2017

WHAT IS ASSESSMENT? WHAT IS ASSESSMENT? For administrative units, the assessment process provides the opportunity to systematically improve the quality of their processes. Focus is on meaningful outcomes. Measuring those outcomes. Examining the results in relation to established targets. Developing plans for improvement. Implementing those plans. Evaluate resulting changes.

WHY DO ASSESSMENT? WHY DO ASSESSMENT? Identify strengths and areas for improvement Provide evidence for stakeholders, including regional accreditors Highlight program contributions to stakeholders Encourage collaboration within and among units Create a unit vision and ideal Encourage systematic review, particularly of critical functions, services, processes BUT the PRIMARY REASON is to obtain data to improve the unit, its functions and its processes, particularly as they relate to the institution’s mission and strategic plan.

STEPS IN THE PROCESS: State administrative outcome to be assessed. WHAT IS ASSESSMENT? STEPS IN THE PROCESS: Review unit mission/strategic plan and the core functions of the unit. State administrative outcome to be assessed. Identify an assessment methodology/measure. Identify target. (what would be considered success?) Share findings for each outcome addressed (% achieved, # met, etc.) Reflect on findings and describe planned changes to be implemented.

WHAT IS ASSESSMENT?

STARTS WITH THE MISSION STATEMENT A concise statement about the values and purpose of the unit. What would your perfect unit look like? What does it take to achieve a high level of performance? You are already collecting measurements Why? How do these relate to your mission or functions? Consistent with institutional and college mission Structure of an administrative unit mission statement: “The mission of (name of your unit) is to (your primary purpose) by providing (your primary functions or activities) to (your stakeholders).” Additional clarifying statements as necessary.

Checklist for Administrative Outcomes Is it aligned with the mission statement? Is it important to management and reflects key results of the unit’s work? Is it possible to collect accurate and reliable data for the outcome, given existing resources? Can it be used to identify areas to improve? Can it be measured in more than one way? (ex: High quality results to requests as evidenced by a survey response from requestors served).

ADMINISTRATIVE OUTCOMES Suppose you already have several measures for which you already collect data, e.g., # of reported crimes on campus. The outcome being measured is “members of the campus community will be able to operate in a safe and secure environment.” (a good outcome is stated so that it cannot be measured by a single method.) Other examples The admissions office serves as a link between higher education and the community to counsel and advise students, parents, and community members on the opportunities provided at the university. The Office of the Registrar oversees the student registration process to ensure students are on the path to graduation. The Bursar’s Office maintains accurate and timely processing for accounts payable.

WHAT TO CONSIDER WHEN CHOOSING A MEASURE MEASURES WHAT TO CONSIDER WHEN CHOOSING A MEASURE Compatible with the outcomes Provide reasonably accurate, useful information Information should be directly controllable by the unit Data should not be difficult to collect or access Examples include operational data, content analysis, point of service assessment, national surveys, locally developed surveys Results should be easily interpreted and unambiguous Yield results specific enough to show where improvements could be made Use methods that can assess both strengths and weaknesses Use multiple methods Direct and indirect Direct measures – observable behavior Indirect measures – infer, not directly observable

BE CAREFUL WHAT YOU ASK FOR….. MEASURES BE CAREFUL WHAT YOU ASK FOR…..

METHODOLOGY MAPS CAN HELP WITH PLANNING: MEASURES METHODOLOGY MAPS CAN HELP WITH PLANNING: An example of what an assessment methodology map might look like Administrative Outcome Direct Assessment Methods Indirect Assessment Methods   Reported crime #’s # of students using free rides after 8:00 pm Focus group Senior exit survey 1-year Alumni survey Outcome #1: members of the campus community will be able to operate in a safe and secure environment. Summarized every month Report compiled at the end of each semester Conducted every year, in the spring Conducted by OSPIE in December & May Conducted every three years

TARGETS WHAT IS A PERFORMANCE TARGET? Defines the unit’s achievement expectations for each outcome-method pair (a benchmark for judging success) Is usually expressed as percentages or numbers expected Is achievable but rigorous Is realistic for the unit Examples: --- 95% of the results will be proposed, analyzed, and provided to the client within two weeks. --- The office of Research will receive an evaluation of satisfactory or above for support provided by 90% or more of faculty who submitted grant proposals and completed the assessment instrument.

ACTION PLANS WHAT MAKES A GOOD ACTION PLAN? Action plans: Describe the changes a unit intends to implement to address deficiencies identified as a result of the assessment Describe specific changes with timelines if possible Should be discussed among all members of the unit, as applicable, to involve them in the improvement or planning process Changes can be to processes, policies, procedures, the assessment process/measure

CLOSING THE LOOP SEVERAL WAYS TO “CLOSE THE LOOP” #1: Assessment results show that program has achieved its outcome. Data were collected, results show outcome achieved given the specific target; program plans to continue to measure outcome in future assessment cycles. #2: Assessment results show that program has not achieved its outcome. Data were collected, results show outcome not achieved given the specific target; program developed/implemented action plan to improve performance; program examined impact of action plan by re-measuring and collecting data on outcome in a future cycle. #3: Program has determined that the assessment plan is ineffective and needs to be revised. Data were collected; program determined that plan was not accurately measuring the outcome or that outcome needed revision; program documented changes to be made in the plan and re-measured in future cycle.

REPORTING

Taskstream as the repository for our work Reports due: Assessment plans for the year - August 25th Assessment results report - June 29th Two major areas in Taskstream: Standing Requirements includes mission & outcomes Annual Assessment Plan includes assessment results report and assessment plans for the year

SUMMARY

QUESTIONS