Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs College of Letters & Science University of Wisconsin-Madison

Snapshot of Student Academic Affairs Associate Dean Deputy Assoc. Dean for Advising CCAS Career Services FIGS LSAC AAP Deputy Assoc. Dean for Policy Assistant Deans For Policy Deputy Assoc. Dean for Pathways American Indian African American Southeast Asian Chicano/a Undergrad Research Scholars Summer Collegiate Experience Writing Fellows

Letters & Science Student Academic Affairs Demographics Approximately 18 units Size varies from 15 staff members to one Some units have assigned populations Some units have a campus-wide mission College encompasses nearly 17,000 students Familiarity and use of tracking and evaluations tools varied widely

SAA – Vision for Assessment Each unit would assess program quality, productivity, need, and demand Continuously improve quality of programs offered by capturing voices of students and other constituents Enhance program planning and budgeting Establish baseline data to explore longitudinal trends and patterns Satisfy institutional/state/national reporting and review requirements

Charge to Assessment Committee Develop instruments to measure student perceptions and overall quality of student experience Develop standardized expectations for unit directors/coordinators Provide staff with a practical, helpful tool Begin systematizing data collection to acquire retention and longitudinal data Pilot project in place for academic year

L&S SAA Assessment Committee Made up of a cross-section of staff in SAA Determined that a handbook would be the best route to go Developed timeline and content areas Split into teams to work on each section

Factors for Success Involved those who will implement and use the Handbook Process involved on-going negotiation and compromise Wanted to take the assessment picture down to very small and practical increments Took existing systems into account Developed flexible, adaptable plan Encourage risk-taking

Foundation Differentiates between tracking for annual reporting and collecting data for program improvements (Service Quality versus Student Learning Outcomes) Does not ask unit directors/coordinators to get into determining retention rates Next steps will be defining and measuring learning outcomes

Handbook Introduction Two-hour session with unit coordinators/directors in May 2002 Associate Dean discussed her vision for assessment for our division of student academic affairs Assessment Team walked everyone through the contents of the Handbook

Handbook Contents Associate Dean’s vision for Assessment in Student Academic Affairs Ethics in assessment Expectations for unit directors regarding collecting usage statistics and program improvement data, developing annual reports, and conducting self-studies Primer on the basic tools of assessment A resource section

Ethics in Evaluation Ensure ethical treatment of research participants Avoid harm Informed consent Freedom from coercion Ensure ethical collection of data Ensure ethical interpretation and presentation of data

Primer on Statistical Concepts Includes brief discussion and cautions regarding interpretation of data Also addresses managing data, including issues of confidentiality and security Has changed significantly since evolution of on-line survey tools such as Zoomerang.com and Survey Monkey

On-Going Assessments Capturing student names/IDs in ISIS and/or stand-alone databases (discouraged) Collect demographic information about program participants Capture information about advising appointments Establish systems to regularly assess student satisfaction with services Obtain feedback for program improvement Systematically gather data to assess retention and persistence (done at the divisional and college level)

Ways to Collect On-Going Data Phone calls s Advising contacts Resource use Workshop attendance Events in general Website hits Number of students turned away from services

Annual Reports Each SAA unit will submit an assessment plan in September each year. Each unit will submit an annual report due in January each year. Responsibility of unit director/coordinator Snapshot of previous year’s accomplishments and a look toward future initiatives and challenges

Contents of Annual Report Table of Contents Executive Summary Mission Statement and Core Values Strategic Plan if applicable Overview of past year Future goals and initiatives

Self Studies Ten-year cycle for self studies Dean’s office has determined expectations for self studies Additional details are outlined in the handbook

Assessment 101 Selecting Assessment Tools Statistical Concepts Interpretation Managing Data Surveys Interviews

Selecting Assessment Tools Quantitative Advantages and disadvantages of each Encourages multiple measures and methods

Surveys Gathering demographic information Types of questions Ways to gather survey data Tips Survey Kit by Arlene Fink (Sage Publications)

Interviews Role of human subjects committee FAQ prior to interviews Focus groups versus in-depth individual interviews Analyzing interviews

Resource Section List of campus resources List of web-based resources Books on assessment Article by Upcraft & Schuh from About Campus (1998) AAHE – 9 Principles of Good Practice for Assessing Student Learning Handbook can be found at:

Where are we now? Got the tracking and evaluation systems established in all units Have had two sets of assessment plans and annual reports turned in Obtained feedback from unit directors/coordinators and began updating handbook SAA has spend the past nine months going through its own assessment and evaluation process

Final Thoughts Assessment can be intimidating, but knowing where you are is essential to determining where you want to go! Just pick someplace to begin – don’t get overwhelmed by the big picture Develop a culture that embraces continuous improvement and inclusion of student voices in the planning

Stretch and Take a Break! Discuss with your colleagues your biggest challenge regarding assessment

Self-Evaluation of Assessment Practices Tool for evaluating “where you’re at” regarding assessment in your unit/department Identify strengths upon which to build Identify aspects to improve or enhance Catalyst for important conversation and action