Farris Child Cynthia Wong Dan Chandler FF

1. Describe the differences between direct and indirect evidences of learning. 2. Identify instances where either direct or indirect evidences would be appropriate. 3. Explain why direct evidences of learning are important. 4. Show how changing the assessment of an outcome can lead to direct verses indirect evidence. FD

IndirectDirect  Capture perceptions or opinions  Capture signs that students/ advisors are probably learning or have the skill sets being assessed  Less clear and less convincing  Requires those being assessed to demonstrate their knowledge or skills  Tangible, visible, self explanatory evidence of what has and hasn’t been learned or what has or hasn’t been done  Clear and convincing DD

Indirect Direct  Surveys  Questionnaires  Interviews – asking questions which responses can’t be validated  Questions asking if a student has the skills or knowledge regarding an outcome (Yes / No type questions)  Direct Observations  Pre test / Post test  Evidence we can see which clearly demonstrates skill sets or knowledge = no doubt  Questions asking the student to explain, identify, define, list, etc.  Possibly multiple choice questions (depends on how the outcome is written) DD

Indirect Direct  Ask advisors if they participated in professional development  Or ask how many hours of professional development they acquired.  Look at an advisor’s transcript, signed CEU form, certificate of attendance, etc. DD

Indirect Direct  Use a Likert scale asking students the extent to which they know what needs to be done in order to get off academic probation  Have the student explain to their advisor how to get off academic probation ◦ OR ask a student to correctly identify which statement describes how to get off academic probation (out of several options) ◦ OR ask the student to explain how to get off academic probation on a survey with an open ended question DD

Indirect Direct  Ask student if they know what they need to do in order to move forward with their career goals  Advisors use a rubric with defined levels identifying how prepared the student is with their career goals (graduate school, internship, and employment preparedness, etc.)  Ask student to list the next steps for their career goals DC

Surveys Questionnaires Pre-test/post-tests Direct Observations Focus Groups Interviews

 Fusch (2013) suggests we should move beyond surveys by having students create learning products which allow us to form a more direct assessment of what students have learned  Campbell (2005) states, “We must gather evidence from multiple sources. Evidence must reflect both direct and indirect measures, and be both quantitative and qualitative.”  Suskie (2009) proposes that, “No assessment of knowledge, conceptual understanding, or thinking or performance skills should consist of indirect evidence alone.” CF

◦ Indirect – Ask yes/no, Likert type, etc. questions ◦ Direct – Ask students to identify, locate, interpret, etc.  OR- use open ended questions requiring the student to explain, list, recall, describe, etc. FD

◦ Indirect –  Outcome = Students can identify what they must do in order to get off probation  Question = Do you know how to get off probation (Yes or No) OR- Please indicate the level to which you understand what you need to do in order to get off probation ◦ Direct  Question = How does a student get off probation? (open ended or have them find and select the correct response out of several possibilities).  Because the outcome key word is identify – Have a multiple choice question with several possibilities and one correct option. If the student can identify the correct option then this would be an example of a direct evidence. DC

◦ Indirect –  Outcome = Students can find their progress report  Question = Do you know where to go to find your progress report? ◦ Direct  Question = Which of the following steps (in order) would you take in order to access your progress report? (This question would list several possibilities and require the student to identify which steps they would take.)  OR we could ask the student to, “Please show me how to find your progress report.” CF

Think about a Student Learning Outcome or Process and Delivery Outcome that you trying to assess:  How could we gather direct evidence for this outcome?  How could we gather indirect evidence for this outcome? FF

 Aiken-Wisniewski, S., Campbell, S., Higa, L., Kirk-Kuwaye, M., Nutt, C., Robbins, R. & Vesta, N. (2010). Guide to Assessment in Academic Advising Second Edition. Monograph Series Number 23. National Academic Advising Association.  Suskie, L. (2009). Assessing Student Learning. 2 nd Ed. San Francisco, CA. John Wiley & Sons, Inc.  Campbell, S. (2005, December). Why do assessment of academic advising? Academic Advising Today, 28(4). Retrieved from Academic Advising Today: Do-Assessment-of-Academic-Advising.aspx Do-Assessment-of-Academic-Advising.aspx  Fusch, D. (2013). Assesing Student Learning Outcomes: Surveys Aren’t Enough. Retrieved from Academic Impressions Web Site: surveys-arent-enough?qq=18798a430987mW surveys-arent-enough?qq=18798a430987mW  Robbins, R. & Zarges, K.M. (2011). Assessment of Academic Advising: A Summary of the Process. Retrieved from NACADA Clearinghouse of Academic Advising Resources Web Site: of-academic-advising.aspxhttp:// of-academic-advising.aspx