Assessment of Student Affairs Initiatives for First-Year Students National Conference on First-Year Assessment October 12-14, 2008 San Antonio, Texas Jennifer R. Keup Director, National Resource Center for The First-Year Experience & Students in Transition

Goals for Today  Clarify the relationship between FYE, Student Affairs & Assessment for you  Identify and discuss challenges to FYE assessment in Student Affairs  Review a Comprehensive Assessment Model Discuss Assessment Outcomes Assessment Audit Assessment Approaches Feedback Loops

What is Assessment?  “Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness.”  Effectiveness includes: Student learning outcomes Clientele satisfaction Compliance with professional standards Comparisons with other institutions  Assessment guides good practice

Introductions and a Question  Introduce yourself: Name Institution  Venn Diagram Exercise Identifying the relationships between FYE, student affairs, & assessment at your institution. Where do you fit in this model on your campus?

Venn Diagram Exercise: Identify the Relationships Assessment The First-Year Experience Student Affairs You

Venn Diagram Example: Identify the Relationships Me Assessment Student Affairs The First- Year Experience

What is the assessment process?  Identify expected/desired outcomes.  Plan programs and services that accomplish outcomes.  Implement programs and services.  Assess the learning responsive to the outcomes through sound data collection & interpretation  Use results to improve learning opportunities.

Why is this so hard? Challenges of First-Year Assessment in Student Affairs

Barriers to Assessment  Lack of Commitment & Support from Leadership  Lack of Assessment Expertise  Lack of Resources Fiscal: $$$ Human: Professional Staff Support  Fear of Results

Let’s “Talk Turkey” What are some of the barriers that are unique to ASSESSMENT of the First-Year Experience? What are some of the barriers that are unique to ASSESSMENT in Student Affairs? What is really going on for you?

Purposes of FYE Assessment  Backgrounds & characteristics of entering students  Entering students’ needs  Entering students’ satisfaction with orientation & FYE programs  Outcomes of orientation & FYE programs  Institutional climate for first-year students  Comparisons to national standards  Cost-effectiveness Gain Information About:

How do I approach this? Model of Assessment for First-Year Initiatives in Student Affairs

Steps for Assessment 1. Define the problem 2. Determine the purpose of the study 3. Determine where to get the information needed 4. Determine the best assessment methods 5. Determine whom to study 6. Determine how the data will be collected

Steps for Assessment (cont.) 7. Determine what instruments will be used 8. Determine who should collect the data 9. Determine how the data will be analyzed 10. Determine the implications of the study 11. Report the results effectively

1.Define the problem 2. Determine the purpose of the study Identifying meaningful and measurable outcomes of Student Affairs Initiatives for First-Year Students

Identifying Outcomes What are the questions we need data to answer? What do we want to know? How do these answers relate to FYE initiatives in Student Affairs?

Types of Outcomes  Program Outcome: What your program is accomplishing? What is the degree to which it is being accomplished?  Student Learning Outcomes: What the students are learning as a result of programs & services? What is the degree of learning attained?  Student Development: What the students are learning that advances their abilities to perform effectively in a life or work situation?

Examples of Outcomes  Persistence  Analytical & critical thinking skills  Knowledge of university requirements  Interpersonal/Communication skills  Creation of a sense of community  Identity exploration and development  Develop educational career goals/Declare a major  Institutional climate

Examples of Outcomes (cont.)  Cohort/generation characteristics  Multicultural competence  Civic engagement/Develop as democratic citizens  Ability to identify, seek, & utilize organizational resources and student programs  Physical health & emotional wellness  Leadership skills  Moral and Ethical Development

Writing Outcomes  Be clear about what you are assessing Program Outcome Student Learning/Development Outcome  Make sure that your outcome is meaningful  Clearly state what you want students to know and be able to do Incremental steps, stages, points of progress Indicate the point that represents fulfillment/success

Outcomes Exercise  Pick an outcome of a program or service for First-Year Students on your campus that is managed by a Student Affairs unit or department.  Define the outcome in specific terms: What are you assessing? How is it meaningful? What do you want students to be able to know and do?  Key question: Is this outcome measurable?

3. Determine where to get the information needed Conducting an audit of current assessment activities of Student Affairs Initiatives for First-Year Students

What to look for?  Are student data collected?  What is being collected?  By whom?  Where are they housed?  Are these data currently being utilized? If so, how are the results communicated? If not, why not?  What future data collection efforts are planned?

Potential Sources of Data  Admissions/Registrar’s Data  Institutional participation in national surveys of first-year students  Program/course evaluations  Utilization statistics of SA services/programs  Satisfaction surveys for SA services/programs

Potential Sources of Data (cont.)  Home-grown IR surveys  SA departmental surveys, focus groups, or student interviews  Accreditation reports  Grants  Strategic planning processes  Self-study processes/CAS  Other sources?

“Who do I know?” Exercise Write the name of a “responsible party” from your campus next to as many of the “Potential Sources of Data” items as possible.

Who do I talk to?  Exercise: How many were you able to fill in? Who were the key players?  Resources: Student Affairs information, assessment & research specialist FYE Director (or other equivalent) IR Director VC/VP for Student Affairs Outside agencies

4. Determine the best assessment method 6. Determine how the data will be collected Assessment methodologies

Methods to Collect Data  Quantitative Describes and analyzes “what is” Examples  Surveys  Analysis of existing data  Qualitative Collect data about the meaning of events & activities to the people involved Examples  Interviews  Focus Groups  Observations

Methods to Assess Outcomes  Indirect Self-reported comments that reveal a perceived increase in understanding or appreciation The perception is usually not verified through any demonstration of knowledge acquisition or observation  Direct Demonstration of abilities, information, knowledge, etc. as the result of participation in a program or utilization of a service

Indirect Assessment Methods  Examples Satisfaction measures Program evaluations Self-ratings of skills Self-assessment of change Agreement with statements Inventories Informal peer-to-peer conversations  Pro/Con of indirect assessment methods

Direct Assessment Methods  Examples Portfolios Capstones Performances Common assignments Exams Observations of behavior/performance Standardized tests Work samples  Pro/Cons of direct assessment methods  Making indirect methods direct

10. Determine the implications of the study 11. Report the results effectively Closing the loop

Implications  Grounded in Steps 1 & 2: Purpose & Problem Have you answered the original assessment question? What have we learned about our outcome of interest?

Effective Reporting: Audience  Who is your client?  Use it as an opportunity to connect with your list of “responsible parties” for data.  Who should know about this information? If the information can inform practice, whose practice will it inform?  Know the climate and context of your university Venn Diagram List of Challenges

Effective Reporting: Structure & Dissemination  Reporting formats: Paper Web Presentation  Executive summary  Visual identity  Leverage information channels that already exist Responsible parties for data  Student Affairs Program Review