Copyright 2007 Assessment 101: A Guide for Student Affairs ACPA National Convention Atlanta, GA March 29, 2008 Becki Elkins Nesheim Dianne Timm Kimberly.

Slides:



Advertisements
Similar presentations
Assessment of Academic Advising Assessing for Excellence Conference Central Carolina Community College Joni Pavlik & Brian Merritt April 16, 2008.
Advertisements

Introduction to Assessment – Support Services Andrea Brown Director of Program Assessment and Institutional Research Dr. Debra Bryant Accreditation Liaison.
What “Counts” as Evidence of Student Learning in Program Assessment?
Evaluation for 1st Year Grantees Shelly Potts, Ph.D. Arizona State University
Best Practices in Assessment, Workshop 2 December 1, 2011.
Guidelines and Methods for Assessing Student Learning Karen Bauer, Institutional Research & Planning, Undergraduate Studies; Gabriele Bauer, CTE.
Learning Objectives, Performance Tasks and Rubrics: Demonstrating Understanding and Defining What Good Is Brenda Lyseng Minnesota State Colleges.
USING RUBRICS TO ASSESS THE PERFORMANCE OF NEW PROFESSIONALS IN STUDENT AFFAIRS Tony Ribera & Sarah Fernandez A Presentation at the 2009 NASPA Region IV-E.
The Purpose of Action Research
NCCEA Annual Conference Waynesville, NC Assessment Basics: Implementing and Sustaining a Comprehensive, Outcomes-Based Assessment Plan October 19, 2006.
A brief overview What is program evaluation? How is an evaluation conducted? When should it be used? When can it be used? Used with Permission of: John.
Department Plans for Assessment of Student Learning Teachers Talking February 16, 2011.
Assessment of Student Affairs Initiatives for First-Year Students National Conference on First-Year Assessment October 12-14, 2008 San Antonio, Texas Jennifer.
ASSESSMENT WORKSHOP: SESSION 1 ADMINISTRATIVE SUPPORT SERVICES ACADEMIC AND STUDENT SUPPORT SERVICES PRESENTED BY THE DIVISION OF INSTITUTIONAL EFFECTIVENESS.
Comparing Assessment to Educational Research. In this session, we will… Construct and discuss the (related but different) goals and attributes of assessment.
Principles of High Quality Assessment
SURVEYS, OBSERVATIONS, AND RUBRICS OH MY! ASSESSING CAREER SERVICES Jessica M. Turos Bowling Green State University Career Center.
How to Write Goals and Objectives
Measuring Learning Outcomes Evaluation
1 KapCC Student Learning Outcomes Assessment Framework Faculty Senate SLO Committee Presented by Kristine Korey-Smith.
Catherine Wehlburg, Ph.D. Office for Assessment & Quality Enhancement.
Developing an Assessment Plan Owens Community College Assessment Day Workshops November 13-14, 2009 Anne Fulkerson, Ph.D. Institutional Research.
UOFYE Assessment Retreat
Strategic Planning with Appreciative Inquiry
1. What is it we want our students to learn?
Reaffirmation of Accreditation: Institutional Effectiveness Southern Association of Colleges and Schools February 2008 Stephen F. Austin State University.
How to Develop a Project Evaluation Plan Pat Gonzalez Office of Special Education Programs
Learning Outcomes at the University of North Alabama Dr. Andrew L. Luna Institutional Research, Planning, and Assessment.
NAVIGATING THE WATERS: USING ASSESSMENT TO MAKE A DIFFERENCE Amy Harper, Area Coordinator, Fordham University Greer Jason, PhD, Assistant Dean of Students,
Assessment 101: Back-to-Basics An Introduction to Assessing Student Learning Outcomes.
ASSESSMENT  Are you having an impact?  How do you know?  Are the programs and services you work on getting better?  How do you know?
Assessing Student Learning: What Kapi‘olani Community College is Doing Kristine Korey-Smith, Assessment Coordinator Louise Pagotto, Interim Vice Chancellor.
Outcomes Assessment 101 Assessment Showcase 2009: Best Practices in the Assessment of Learning and Teaching February 11, 2009 Gwendolyn Johnson, Ph.D.
Academic Assessment at UTB Steve Wilson Director of Academic Assessment.
October 31, Dialog about SLOs, assessment, and existing practices at TC Identify course level SLO to assess this semester Align SLO with TC’s institutional.
 “…the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and.
Connecting Course Evaluations to Program Evaluations Marilee J. Bresciani, Ph.D. Associate Professor, Postsecondary Education and Co-Director of the Center.
Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,
Unit 1 – Preparation for Assessment LO 1.1&1.2&1.3.
Comp 20 - Training & Instructional Design Unit 6 - Assessment This material was developed by Columbia University, funded by the Department of Health and.
Building Your Assessment Plan Esther Isabelle Wilder Lehman College.
Assessing Student Learning Lynn Merklin Assistant Provost Office of Institutional Effectiveness August, 2014.
Inquiry and Investigation. What was the TOPIC? PROBLEM? CIVIC INQUIRY?
Unwrapping the Common Core (Oklahoma Academic Standards) Moving Forward One Step At A Time Standards.
Teaching Today: An Introduction to Education 8th edition
Creating a S.I.M.P.L.E. & Comprehensive Assessment Plan for Residence Life and Housing Dr. Mike Fulford Emory University Student Affairs Assessment Conference.
Presenters: Erin Nunn, Jarrett Kealey, and Katelin Getz Ohio University.
Fourth session of the NEPBE II in cycle Dirección de Educación Secundaria February 25th, 2013 Assessment Instruments.
“Outcomification”: Development and Use of Student Learning Outcomes Noelle C. Griffin, PhD Director, Assessment and Data Analysis Loyola Marymount University.
Susan DeMatteo June 13, 2013 Putting EOF on the Map! Mapping Strategies for Assessment Design.
A Basic Guide to Academic Assessment Presented by Darby Kaikkonen Director of Institutional Research.
Beginning with the End in Mind: Choosing Outcomes and Methods
Student Affairs Assessment Council Wednesday, October 28, 2015.
Does Your Academic Program Make a Difference? Knowing What Your Students Learned Rosslyn M. Smith (moderator) Katherine A. Austin Valerie Osland Paton.
Understanding Assessment The Basics Office for Institutional Effectiveness and Assessment.
1 Learning Outcomes Assessment: An Overview of the Process at Texas State Beth Wuest Director, Academic Development and Assessment Lisa Garza Director,
Intentionally Targeting Student Learning Outcomes through Course Assessment and Design Patty Roberts, Ph.D. Patricia Noteboom, Ph.D NASPA INTERNATIONAL.
Identifying Outcomes Peggy Maki Senior Scholar Assessing for Learning American Association for Higher Education
Developing Program Learning Outcomes To help in the quality of services.
ASK STANDARDS Assessment and Accountability CNS 610 Written & Narrated by: Kelcie Dixon Western Kentucky University.
Student Learning Outcomes (SLOs) Module #2: Writing SLOs Office of Academic Planning & Accountability Institutional Effectiveness Moderator: Dr. Cathy.
Assessment 101: Or Why and How to Assess ACRAO Spring Conference 2011 Presenter: Gladys Palma de Schrynemakers.
ASSESSING STUDENT LEARNING AND DEVELOPMENT By Dr. Clarice Ford.
The Assessment Process: A Continuous Cycle
What are Learning Outcomes and how to create good Learning Outcomes
Consider Your Audience
Effective Outcomes Assessment
Assessment planning: It’s the parts that make up the whole!
Presented by: Skyline College SLOAC Committee Fall 2007
Presentation transcript:

Copyright 2007 Assessment 101: A Guide for Student Affairs ACPA National Convention Atlanta, GA March 29, 2008 Becki Elkins Nesheim Dianne Timm Kimberly Yousey

Copyright 2007 Goals of the program: Understand key assessment terms Learn the assessment cycle and how to use it to carry out assessment projects Develop strong learning and program outcomes Plan an assessment project from start to finish

Copyright 2007 Assessment “Any effort to gather, analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness.” (Upcraft & Schuh, 1996, p. 18) A systematic process which includes research, data gathering, data analysis, and evaluation in an effort to guide good practice and improve student learning and development. (Erwin, 2002; Palomba & Banta, 1999)

Copyright 2007 Evaluation The process of analyzing results in a manner that makes the results usable. (Upcraft, 2003; Upcraft & Schuh, 1996) It is part of the assessment process. (Upcraft & Schuh, 1996).

Copyright 2007 Evaluation FORMATIVE EVALUATION Provide feedback Modify and/or shape a program or service Improve a program or service SUMMATIVE EVALUATION Judge a current program or service for quality and/or worth Based on previously established standards (Palomba & Banta, 1999)

Copyright 2007 Research The method of gathering information to support and/or guide the assessment process. MEASUREMENT Methods used to gather information Qualitative or Quantitative or combination of the two (Upcraft, 2003)

Copyright 2007 Assessment Cycles How you organize your assessments Assessment systems vs. ad hoc systems Adaptable to different institutions and goals

Copyright 2007 Example #1: Comprehensive Assessment Model Usage Numbers Student needs Student satisfaction Environments Learning Outcomes Comparable (Benchmarking) Using National Standards (i.e. CAS) (Upcraft & Schuh, p. 27)

Copyright 2007 Example #2: Questions to Guide Process What’s the problem? What’s the purpose? Who will be studied? What’s the best assessment method? How do we decide who to study? How should the data be collected? What instrument(s) should we use? Who should collect the data? How should we record the data? How do we analyze the data? How do we report the results? How do we use the results? (Upcraft & Schuh, p. 316)

Copyright 2007 Example #3: Assessment Cycle Identify Outcomes Gather evidence Interpret Evidence Implement Change (all centered around Mission/Purpose, Educational Objectives) (Peggy Maki)

Copyright 2007 Example #4: Practical Inquiry Approach What topic are you interested in? What would you like to know about your topic? (Question) How can you answer your questions? (Method, data collection) What did your data tell you? (analysis) How does that inform practice? (implications/report) What is the next question? (feedback loop) (Yousey)

Copyright 2007 Example #5: George Washington University Office of Academic Planning and Assessment Setting goals Develop strategies for collecting information Implemeting your plan Using the data

Copyright 2007 Example #6: University of Georgia

Copyright 2007 Example #7: California Lutheran University Office of Assessment and Educational Effectiveness Gather Evidence Communicate and reflect Decision Making/Planning/Policy Development

Copyright 2007 A Comprehensive Model:

Copyright 2007 Assessment Cycle in Action A brief example

Copyright 2007 Quick Assessment…. Need a stretching break?? Take 15 But before you go we have some homework: think of an issue/topic your office is facing that you would like to assess

Copyright 2007 Putting the Cycle Into Action: Creating an Assessment Plan begins by going step-by-step through the cycle

Copyright 2007 Step #1: Learning Outcomes Objectives which help define in clear and simple terms how student affairs programs and services contribute to student learning and fit institution mission. (Bloland, Stamatokos, & Rogers, 1996) Objectives or goals that can be measured to show student learning through student affairs programs and services. (Bresciani, Zelna, & Anderson, 2004)

Copyright 2007 Learning Outcomes Cognitive Learning Outcome General skills, intelligence, higher order cognitive development Affective Learning Outcome Attitudes, values, and self-concept (Alexander & Stark, 2003)

Copyright 2007 Outcome : An outcome is the desired effect of a service or intervention, but is much more specific than a goal. It is participant or output centered. Example : As a result of participating in the Pathways to Success Program students will increase their first year grade point average. (Compared to the similar goal statement, grade point average is a more specific outcome compared to academic achievement.) Developing Outcomes

Copyright 2007 Good Outcome Statements Student learning outcomes translate intentions into actions Describe what students should demonstrate or produce Use action verbs Align with other intentions (institutional, departmental)

Copyright 2007 Good Outcome Statements Map to practices Are collaboratively authored Reflect/complement existing national criteria Are measureable Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution.

Copyright 2007 SMART Outcomes S pecific: Clear and definite terms describing expected abilities, knowledge, values, attitudes, and performance M easurable: It is feasible to get the data, data are accurate and reliable, it can be assessed more than one way A ggressive but A ttainable: Consider stretch targets to improve program

Copyright 2007 SMART Outcomes R esults-oriented: Describe what standards are expected of students T ime-bound: Describe where you would like to be within a specified period of time Adapted from Paula Krist, Director of Operational Effectiveness and Assessment Support, University of Central Florida, May 2006.

Copyright 2007 Individual Exercise Pick one program/service where you know that learning is occurring Make a list in your own words of what they are learning Write a “quick and easy” learning outcome: By the end of this program students will…. (think, feel, do differently)

Copyright 2007 Group Exercise Gather by area of interest Share your outcomes/give feedback

Copyright 2007 Step #2: Planning Methods Begins with: How best to measure your outcome? (Qualitative, Quantitative and Mixed Methods) Need help? See Methods Guide Ends with: Creating a plan of the who/what/where/when/how of your assessment

Copyright 2007 Qualitative Researcher(s) gathers information from the world around them. (Bogdan & Biklen, 2003) Researcher(s) make multiple constructions and interpretations of the world they are studying. (Merriam & Associates, 2002)

Copyright 2007 Quantitative Research designed to gather information and test a particular hypothesis or theory. Uses simple to complex statistics. (Erwin, 1991)

Copyright 2007 Things to consider: How best to answer your question? (Qualitative, Quantitative and Mixed Methods) What’s in your toolbelt? Timeline Resources ($$, time, human) Sampling KISS

Copyright 2007 Measuring Outcomes When measuring learning outcomes make sure that the measure… M atches: Measure directly matches to the outcome it is trying to measure A ppropriate methods: Uses appropriate direct and indirect methods T argets: Indicates desired level of performance

Copyright 2007 Mesuring Outcomes U seful: Measures help identify what to improve R eliable: Based on tested, known methods E ffective and E fficient: Characterize the outcome concisely Adapted from Paula Krist, Director of Operational Effectiveness and Assessment Support, University of Central Florida, May 2006.

Copyright 2007 Group Exercise Look at your outcomes and discuss some ways (or methods) you can answer the question Bounce off different ideas, why would one method be better than others?

Copyright 2007 Step #3: Collecting Methods Easy if you plan ahead Who – is being asked, who is conducting What – method Where – logistical space or web space When – timing is everything How – are you carrying it out, analysis?

Copyright 2007 Individual Exercise (if time) After discussion a method how would you plan the: Who? (who is doing work, who is participating) What? (method, what do you need to design?) Where? When? How? Remember KISS!!

Copyright 2007 Step #4: Report and Use Know your audience Short is better Focus on results AND implications (So now and So what) Use non-technical language Check for content, grammar, etc. before distributing Follow-up with those implementing results

Copyright 2007 Guides to help you along the way: Recommended reading Choosing your methods guide

Copyright 2007 References CLUs assessment system. California Lutheran University, Office Assessment and Educational Effectiveness. Retrieved on-line May 21, 2007, from Erwin, T.D. (1991). Assessing student learning and development : A guide to the principles, goals, and methods of determining college outcomes. San Francisco: Jossey-Bass. The course assessment cycle. George Washington University, Office of Academic Planning and Assessment. Retrieved on-line May 21, 2007, from Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus. Upcraft, M.L., Schuh, J.H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossy-Bass Yousey, K.M. (2006). The best of both worlds: Coupling research and assessment. About Campus, Sept-Oct,