1. Using program evaluation to assess your strategic plan
Karen pettus and Tom thompson

2. Learning outcomes
Participants will be able to name different types of program evaluations and determine appropriate use of each type
Participants will be able to identify the key metrics they use to evaluate student success in relation to their strategic plan and program goals and the appropriate data collection and analysis to use
Participants will be able to describe methods for using evaluation results and make recommendations for program change

3. High cost of a college education

4. Using your strategic plan
Mission statement
Consider the CAS Disability Resources & Services Mission – a three part mission
Program goals and objectives
Tom’s Notes: Every DRS department should have a mission (why you exist) and it is best practice to develop broad departmental goals (think CAS DRS Mission statement) along with annual goals and objectives (projects, processes, programs you are working on annually).

5. Four steps to program evaluation
1. designing your program/departmental evaluation
2. collecting data
3. analyzing and sharing the data
4. improving your processes and procedures
Tom’s Notes: Your approach could be an internal, external or combined process. In Illinois, where I worked in a community college, we were required to do a comprehensive review every five years and both Academic and Student Affairs had their own rubrics.
What you collect and how you collect it will determine its usefulness for analyzing and assessing your findings. Begin with the end in mind, i.e. you are collecting “x” data for the purpose of doing what? Process and service delivery improvement could/should be the driver.

6. designing your program evaluation
Cal State Univ, Fullerton
Univ of South Carolina
Came to campus, April 2015 as Interim and Evaluator (One director for 42 years)
Began informal assessment of processes, policy/procedures, Data collection & usage
Recommended & completed a formal eval, Nov (3 days, 3 evaluators)
Completed first Internal Evaluation in 2005
Made long term and short term recommendations
Began Systematic data collection
3 years ago +$150,000; just notified $355,000 additional funding
Tom’s Notes: Use CSUF as a “Case Study”. I was hired as an Interim and to evaluate the functioning of the department…starting late in a fiscal year.
1. We elevated the implementation of our Clockwork data base Reviewed all written documents related to operations of DSS Planned and executed 65 different outreach/training events Gathered usage data and student feedback on note taking, accessible materials and new student processes.

7. collecting data Cal State univ, fullerton Univ of south Carolina
Greatest areas of need
Create written policies & procedures
Evaluate note taking, delivery of accessible furniture & new student processes
Educate and collaborate with faculty more
Concerns
Length of time to register with office
Increase in students using test proctoring
Tom’s Notes: 1. Created a Letter from the President to all faculty re role of DSS Started developed general and specific guildelines for accommodations Conducted two surveys of note taking and informal assessment of accessible furniture Started processes of evaluating delivery of accessible materials: production time, quality issues, time spent on editing and roles of staff Conducted a pilot project with students and faculty for onine exam accommodations.

8. analyzing and sharing the data
Cal State Univ, Fullerton
Univ of south carolina
Sorted and organized findings
Shared data in meetings, short memos, s and in reports with key recommendations
Formal External eval – included an exit interview, key recommendations and printed report
Created easy to read dashboard
Focus on student success and retention
Highlighted successes, not just needs
Tom’s Notes: I used immediate findings and data to serve as a ‘call for response’, i.e. need to purchase more accessible furniture and improve the storge and delivery process (consulting with Moving Services). Often shared short ‘snippets’ of data in s with my supervisor and AVP. The External Eval has 20 recommendations and will be used to fuel major changes in the future. Used data about problems hosting exam accommodations and overflow into our AT lab to leverage several remodeling projects.

9. improving your processes and procedures
Cal State Univ, Fulerton
Univ. of south carolina
Accessible furniture – no enough in stock, loss or theft, storage, delivery
Lack of policy/procedures – created general guidelines, accommodations appeal process, new ‘UPS’, Prez Letter
Revising new student processes
apply concepts of Universal Design
Furniture–As purchase new furniture, all classes have 1 adjustable table
View ourselves as consultants
Benchmark other similar sized institutions
Tom’s Notes: Overview of some types of changes made, using both formal and informal data. Revising the new student processes is the toughest process to overhaul as it is related to staffing, use of a newer data base, staff attitudes about the process and procedures

10. Application – What to do next?
What do you already know that is not functioning well?
What is your ‘capacity’ to undertake these steps and who can assist?
Where can your efforts have the biggest ‘bang’ for your time?
Once you make changes, evaluate their impact – gather feedback, notice differences and celebrate your victories!