Student Learning and Program Outcomes Applicable to Disability Offices? 2015 AHEAD Conference July 15, St. Paul, Minnesota

Presenters Jean Ashmore Past President AHEAD current AHEAD representative to CAS Rice University Disability Director Emerita Karen Agee Past President College Reading & Learning Association Univ. of Northern Iowa, Reading & Learning Coord., Emerita Paul Harwell Texas A&M University Tom Thompson Past AHEAD Board Member Harper College Disability Director Emeritus

Lunch & Learn Session Topics Introducing CAS student learning and development outcomes Linking student outcomes and program standards Developing systematic practices for assessing program outcomes Managing outcome data Demonstrating continuous improvement and accreditation requirements

Why is DS self-assessment critically important? Gathering formative and summative data (metrics) is a primary means of: - Understanding what is working well - Zeroing in on how to improve services - Justifying additional resources Demonstrating the impact of DS on student life, growth and success - Answering how do these supports impact engagement, persistence and completion

CAS - Counsel for Advancement of Standards in Higher Education 47 sets of Standards & Guidelines for DRS and other functions usually within Student Affairs Disability Resources & Services standards revised in 2013 (Karen Agee, chair; Jean Ashmore, expert) Mission and Program the first of 12 areas of self- assessment CAS member associations include AHEAD, CRLA, NACADA, and 38 other professional organizations

Program Outcomes Defined strategically to be measurable Linked to DS mission, program of services, and resources – all locally developed Applicable – Within the DS office, e.g., tech training, testing, alternative format production – Across campus, e.g., trainings, outreach, lectures

CAS Student Learning & Development Outcomes Adopted by CAS in 2008 (in current form): Based on Learning Reconsidered 2: Implementing a Campus-Wide Focus on the Student Experience (Keeling, 2006) Applicable, in part or whole, to all 47 areas of CAS standards Expected to be defined locally and to be measured directly and indirectly

SL&D Outcomes Student Outcome Domains 1.Knowledge acquisition, construction, integration and application 2.Cognitive complexity 3.Intrapersonal development 4.Interpersonal competence 5.Humanitarianism and civic engagement 6.Practical competence

Dimensions Each of the 6 Outcome Domains has Dimensions: – Example – SLO domain of Interpersonal Competence has dimensions Meaningful Relationships and Interdependence – Example – SLO domain of Practical Competence has dimensions Pursuing goals, Communicating effectively, Technological competence, Managing personal affairs, Managing career development, Demonstrating professionalism, Maintaining health & wellness, Living a purposeful & satisfying life Not all domains will apply to each campus entity

Domains and Dimensions Domain - Practical Competence Dimensions Communicating effectively - Disclosing & articulating about one’s use of accommodations Technical competence - Using technology effectively Managing personal affairs - Balancing personal life and college demands Managing career development - Preparing for transition to employment or grad school – issues of disclosure & using accommodations

How can SLOs be used by DS? Step 1: Define one’s DS mission and program of activities Step 2: Review SLOs critically, looking for links to program services & activities Step 3:Write specific student outcomes that are measurable, directly & indirectly Step 4:Collect outcome data regularly, and analyze, report, and adjust program services accordingly

How can SLOs be used by DS? Assessing SLOs formalizes what every DS professional does all the time anyway, to answer questions like these: – How well did that session go? – What did those students think our session accomplished? – Are we providing good services? – Do student employees know appropriate strategies? Assessing program outcomes formalizes the bigger questions we hope to affect: o Are we retaining our clients at this institution? o Are our clients academically successful here?

Methods of Assessing SLOs? Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus. Direct methods Students demonstrate learning via some form of standardized test focusing on aspects of student learning Examples: CAAP, CLA, ETS Proficiency Profile (formerly MAPP), GRE subject tests, PRAXIS exams Indirect methods Students report perceptions of their learning and the educational environment that supports that learning Examples: CSEQ, NSSE, NSLLP Performance-based methods Students represent learning in response to assignments or projects that are embedded in their educational experiences Examples: Successful solution of a math problem not previously understood, demonstrated writing skill, panic replaced by confidence But we DS professionals can create more direct methods for assessing learning than standardized tests!

Examples of SLOs within DS Domain 1: Knowledge Dimension: Relating knowledge to daily life Students who complete a DS intake workshop will demonstrate an increase of at least 50% in knowledge of the responsibilities of students and DS as demonstrated by scores on pre- and post-workshop quiz

Examples of SLOs within DS Domain 1: Knowledge Dimension: Relating knowledge to daily life Students participating in at least three individual DS sessions in the semester will demonstrate 50% improved knowledge of accommodations appropriate for their disability or disabilities as measured by their score on the assessment checklist administered by DS staff

Examples of SLOs within DS Domain 2: Cognitive complexity Dimension: Critical thinking Students who work with a DS staff member to identify and acquire accommodations for a disability will, at the end of the session, be able to articulate understanding of the nature of his disability barriers, appropriate accommodations agreed upon, and steps the student has decided to take to acquire those accommodations

Examples of SLOs within DS Domain 6: Practical Competence Dimension: Demonstrating profession alism By the 10 th hour of training, student staff will demonstrate development of professionalism as measured by the program’s professionalism rubric created during the 2 nd hour of training

Examples of SLOs within DS Domain 6: Practical Competence Dimensions: Communication Competence Pursuing Goals Students in group DS coaching sessions will demonstrate communication and goal-setting skills learned in sessions #2-3 by writing and ing to the DS session leader and student members of the group a statement of academic goals with no more than 3 errors of spelling, grammar, and punctuation

Examples of SLOs within DS Domain 6: Practical Competence Dimension: Managing personal affairs Students will demonstrate use of a time management system on Day 8 of the 12-session Effective Study Strategies course as measured by instructor analysis of in-class presentations, documented by artifacts [such as calendar, electronic schedule, and completed assignments]

Examples of SLOs within DS Domain 6: Practical Competence Dimensions: Pursuing goals Managing personal affairs Students who work with a DS staff member on strategies for self-advocacy will document successful attempts to achieve agreed-upon goals at the end-of- semester session.

Why SLOs in DS? The purpose of setting and measuring student learning & development outcomes in DS is to build excellent programs Growth in DS #s = DS expansion = new resources All driven by outcome data

DS Program Outcomes Concept: Assess and evaluate overall program of disability resources & services Foundation: Have a strategic plan for DS with identified goals and an operational plan with specific short range goals Regularly collect input from constituencies: – Students – Staff – Faculty – Institutional partners

Examples of Program Outcomes DS example Strategic Goal: “Promote systemic improvement in campus accessibility for facilities, digital systems and campus procedures” – FY Outcome – Build alt format production into the course development process and train 12 faculty DS example Strategic Goal: “Provide consultation and training to faculty, staff and students that results in greater accessibility” – FY Outcome – Create a course substitution process with support of the faculty, administration and DS.

Examples of Program Outcomes merged with SLOs Program Outcome – Interpreter and transcription services will be used appropriately and be cost effective SLOs – Practical Competence & Interpersonal Competence -students using interpreters or transcriptionists will articulate the guidelines for these services and follow required notification timelines with 95% compliance

Sample Assessment Schedule for a DS Program See Appendix A

Managing Outcome Data Develop comprehensive data collection practices Develop routine reporting cycles and distribution Use data as evidence of program’s budget needs – Key point here – doing assessment and evaluation takes time and skill. In smaller offices, if a Director is to do this work, other staff are needed to see students

Continuous Improvement & Accreditation 360 degree thinking – data leads to action plan = continuous assessment = continuous improvement Accreditation agencies expect – Evidence of effectiveness from both academic and co- curricular programs – Strategic plans with goals and objectives and program assessment for continuous improvement – Demonstration by disability offices of value to students

Conclusion: Disability programs must think strategically! need to develop measurable student learning outcomes! must incorporate program assessment into all elements of DS activity on campus as good business practice! need to go in this direction before being forced to by those who do not understand our work!

Contact Information Karen Agee – Jean Ashmore – Paul Harwell – Tom Thompson –