Counseling & Psychological Services MORE THAN SATISFACTION: ASSESSING CLINICAL OUTCOMES IN COUNSELING CENTERS International Assessment and Retention Conference New Orleans, LA June 13, 2009

Counseling & Psychological Services Today’s Presenters Jacqueline Alvarez, Ph.D. Director of Counseling and Psychological Services, Oregon State University CAPS Assessment Committee Member Beth Dyer, B.A. Information Technology Expert at Counseling and Psychological Services, Oregon State University CAPS Assessment Committee Chair. Student Affairs Assessment Council Member Michele Ribeiro, Ed.D. Staff Psychologist at Counseling and Psychological Services, Oregon State University. CAPS Assessment Committee Member. Student Affairs Assessment Council Member

Counseling & Psychological Services Other Contributors Shailagh Clarke, Ph.D. Staff Psychologist at Counseling and Psychological Services, Oregon State University CAPS Assessment Committee Member Kimberly Miller, Ph.D. Pre-doctoral Intern at Counseling and Psychological Services, Oregon State University CAPS Assessment Committee Member

Today’s Objectives 1. Explore how assessment can be used for learning outcomes for clinical services. 2. Identify ways to overcome barriers to clinical assessment common to counseling centers. 3. Become familiar with a successful model of clinical outcome assessment that produces data that can influence client treatment. 4. Identify at least one learning outcome that you can assess at your home institution. 5. Identify strategies for implementing a comprehensive clinical outcomes assessment program.

How We Got Started The University Began to Require Effective Assessment It became important to demonstrate: 1. How our work aligned with the mission of the university 2. How counseling helps retain students 3. How counseling helps student matriculate

How did CAPS respond to the push to assess our services? Two staff members began developing assessment instruments for Counseling and Psychological Services. We began with simple client satisfaction surveys. OSU Student Affairs Assessment Council was established six years ago, offering a place for all offices to learn together, try new tools, and to work collaboratively. CAPS staff members joined the Student Affairs Assessment Council, and established an Assessment Committee within CAPS. This year, three new members joined the CAPS Assessment Committee, infusing new energy and ideas.

Barriers to Doing Clinical Assessment Well Staff Resistance to Assessment Lack of Expertise within Staff regarding Program Evaluation and the Assessment of Learning Outcomes Limited Time and Resources

Overcoming Barriers to Clinical Assessment Staff Resistance Make assessment data meaningful to staff members Link assessment measures to agreed upon mission and goals Facilitate staff participation Reward participation

Overcoming Barriers to Clinical Assessment, Continued Lack of Expertise in Program Evaluation and Identification of Learning Outcomes Use a committee model Provide training and continuing education opportunities Invite members outside the agency who have more expertise (e.g. a faculty member in your math/statistics department) Recognize that assessment is an ongoing process and that the tools will develop over time

Overcoming Barriers to Clinical Assessment, Continued Limited Time and Resources Create a standing time to do assessment work Add assessment responsibilities to some staff members’ job descriptions, and take away other responsibilities

So, What Have We Done? Methodology Administered a 43 item self-report survey to students using CAPS services in the 6 th and 7 th weeks of each 10-week term for three terms. Revised and updated survey regularly. Hence, the method does not change but the content does.

Information About Our Sample Data from one term (Winter 2009), sample of convenience N=126 with a 51% return rate In which session did you start to see improvement/change occur? CountPercentage First/Intake1213% % % 11 or more33% No change occurred/No change yet 89%

How the Content of Our Assessment Has Developed Began with satisfaction surveys Moved to assessing learning outcomes Now we assess the relationship among: o Student Goals o Student Learning Outcomes o Content of Counseling Sessions

Three Areas of Assessment Stress and Mood Management Personal and Relationship Skills Academic Success

Stress and Mood Management Overall self-care Positive sleep habits Eating habits Regular exercise Overall mood Coping with mental and emotional difficulties

Personal and Relationship Skills Communication skills Relationships with others Identifying and expressing what I need Relationships with my partner Making relationship decisions Insight into my personality Self-esteem My level of connection to the OSU community

Academic Success Time management Motivation Test anxiety Attending classes Study skills Concentration Procrastination Effectiveness as a student

The assessment revolves around three questions we ask about the previously identified areas

First Question “It was important to me to experience change in…” Rated on a 5-point Likert scale, ranging from “Very True” to “Very Untrue”

Second Question “Please describe changes or growth, if any, since you began counseling in…” Rated on a 5-point Likert Scale, ranging from “Much Improved” to “Much Worse”

Third Question “Which of the following issues did you discuss in counseling?” (Check all that apply) Rated as endorsed or not endorsed

Results Slide 1: Graph illustrating… % of clients who expressed a desire to experience change in that particular area AND % of clients who said the area of focus was discussed in counseling Slide 2: Graph illustrating…% of clients reporting each level of improvement Slide 3: Results summary statements

Stress and Mood Management: Coping with Mental and Emotional Difficulties Are topics that clients say are important to change being discussed in counseling?

Coping with Mental and Emotional Difficulties Overall, how much improvement did clients report?

Results for Coping with Mental and Emotional Difficulties 94% indicated that it was important to experience change 74% indicated it was important to experience change in this area and reported improvement 20% wanted change, but did not experience change

Personal and Relationship Skills: Communication Skills Are topics that clients say are important to change being discussed in counseling?

Communication Skills Overall, how much improvement did clients report?

Results for Concentration 67% indicated that it was important to experience change 54% indicated it was important to experience change in this area and reported improvement 13% wanted change but did not experience change 7% did not indicate that it was important to change, but did experience positive change

Academic Success: Concentration Are topics that clients say are important to change being discussed in counseling?

Concentration Overall, how much improvement did clients report?

Results for Communication 67% indicated that it was important to experience change 45% indicated it was important to experience change in this area and reported improvement 22% wanted change but did not experience change 3% did not indicate that it was important to change, but did experience positive change

Important to Change as related to Discussed in Counseling Change desired and discussed Change desired, not discussed Relationships with others Coping with mental/emotional difficulties Expressing needs Communication skills Overall mood Self-care Self-esteem Self-motivation Effectiveness as a student Concentration

General Conclusions By looking at the relationships among the variables (client goals, counseling content, and client change), we get more meaningful information than simply asking, “Did you improve?” or “Did counseling help you stay in school?”

Some specific things we’ve learned through our assessment process Students come to counseling with all kinds of hopes, expectations and goals Students’ personal and relationship skills and ability to manage mood improve at higher frequencies than academics More students report greater change in mood management than academic skills Students report that they talk about mood, personal and relationship skills at higher frequencies than academics

Clinical Uses of Assessment Data The thing that makes assessment compelling is that, when done well, it can give us good information about our work. If the data are meaningful, we are often moved to begin exploring how we can do things more effectively to better meet students’ needs.

Clinical Implications We’re doing very good work in helping students positively change their mood, personal skills and relationship skills.

Clinical Implications Continued Counselors may need to assess students’ goals and expectations more completely, especially regarding students’ hopes for academic change.

Clinical Implications Continued Counselors may need to more overtly discuss academic performance throughout counseling sessions.

Clinical Implications Continued Counselors may need to make explicit to student the relationships among the variables.

Next Steps Clinical Service Assessment Outreach and Program Assessment Training of pre-doctoral interns and other trainees

How to improve your own clinical assessment Work with other people! Begin to identify learning objectives. Explore alternative factors that affect learning and may warrant assessment.

Let’s design one learning outcome that you could effectively assess at your institution.

Questions?

Contact Information Oregon State University Counseling and Psychological Services 500 Snell Hall Corvallis, OR