Use Assessment Data to Influence Student Motivation Leah Parsons Simpson Director of Assessment University of Kentucky College of Pharmacy June 2017

Session Outcomes Distinguish between poor and quality feedback Anticipate student interpretation of various feedback types Use assessment data to create feedback reports that can provide evidence of learning and needs for improvement for both students and administration June 2017

The Typical Graded Exam Scenario: You are a college freshman in a general- education Chemistry course. Your teacher returns your first exam and it looks something like this. How do you use this information? (Think like a freshman) June 2017

The Typical Faculty Exam Report Scenario: You are the lead instructor for a general- education Chemistry course. After the first exam is graded, you receive a report that looks like this. How do you use this information? June 2017

The Power of Feedback Feedback has an average effect size of 0.79 on learning (Hattie, 1999) Feedback with higher effect sizes contain a “informational component” Powerful feedback has three components (Hattie & Timperley, 2007) Feed Up: Make the goals of the course/assignment apparent Feed Back: Explain the student’s current progress Feed Forward: Help students understand what they need to do to get better June 2017

The Power of Autonomy Self-Determination Theory (SDT) states that all humans have three basic psychological needs: autonomy, competence, and relatedness (Deci & Ryan, 1985, 1994, 2000) Autonomy in learning has been linked to greater student satisfaction, persistence, and deeper learning (Seiver & Troja, 2014; Wielenga-Meijer, Taris, Wigboldus & Kompier, 2012) Perception of autonomy-support leads to greater school achievement (Mih & Mih, 2013) Feedback can be autonomy-supportive when it Feeds Up, Feeds Back, and Feeds Forward (Hattie & Timperley, 2007) June 2017

Feedback for the Student Scenario: You are a college freshman in a general- education Biology course. Your teacher returns your first exam and it looks something like this. How do you use this information? (Think like a freshman) June 2017

Feedback for the Faculty Scenario: You are the lead instructor for a Cell Bio-Chemistry course. After the first exam is graded, you receive a report that looks like this. How do you use this information? June 2017

How Do Students Perceive this Type of Feedback? A study Methods: Students were divided into two groups: one received enhanced performance feedback, the other received traditional feedback in the form of grades and grading keys.  A 39-item survey instrument was administered to 139 third-year doctor of pharmacy students.  The survey contained six statements asking students to rate their reasons (relative autonomy) for reviewing feedback. Results: There was a statistical difference, t(52) = -2.07, p =0.043; d = .577, between the two groups in terms of reviewing ExamSoft feedback, indicating enhanced feedback via ExamSoft had a moderate impact on students’ autonomy. This feedback is motivational – students are more motivated to review their ES feedback than a simple grade which is what they often get. It should be autonomy-supportive, as it meets their needs more than simple grades. June 2017

How Do Students Perceive this Type of Feedback? Anecdotal They BEG for this type of feedback. They request other faculty to use this feature. They note that the feedback helps prompt targeted conversations with faculty about their progress. June 2017

How Can You Provide This Feedback? Let’s Play a Bit with the Technology You Have Available Canvas? DtL? Blackboard? ExamSoft? Excel? June 2017

Canvas LMS For information on how to align outcomes to question banks in Canvas click here. June 2017

Canvas (continued) June 2017

Blackboard Your institution must have Outcomes Assessment for this to work. Information about how you can use this feature can be found here. June 2017

D2L Your institution must have Brightspace (Analytics) for this to work. Information about this feature can be found here. Information about how you can use this feature can be found here. June 2017

Competency Progress Across Courses Filter & Search Displays progress for one or more competencies across all courses that utilize the competency. Data is grouped by Competency, Learning Objective, and Assessment Activity, and shows a summary of progress for each course. This report is available in the Analytics Portal and the Administrative Console. Use the Progress on Competencies Across Courses report to address the question "How do student achievement rates vary across courses?" This report enables educators to compare student performance against common competencies across courses.

Program Learning Outcomes Evaluation Program Learning Outcomes Reports This Accreditation report enables program chairs, curriculum committees, and accreditation bodies to gain insight into how well all program participants have performed in their programs, based on the defined program learning outcomes. Each selected program learning outcome displays for your selected semester with achievement results split by each aligned master course within the program learning outcome. By using the Program Learning Outcome Evaluation report, curriculum committees can get an accurate understanding of how program participants are doing, aiding them in making more informed decisions when designing programs. For Program chairs, the report will be beneficial when an institution is going after accreditation. The report gives accreditation bodies fact-based and accurate information about whether an institution is performing well enough to fulfill the accreditation requirements.

Program (or Course) Outcomes Institution (or Program) Outcomes Aligned Outcomes Learning Outcomes Alignment Report Learning Outcomes Alignment Report displays the outcome relationships between two selected levels of the organization (such as a course, program, department, or institution). View alignment of learning outcomes at multiple levels, e.g.: Institutional Outcomes Program Outcomes Course Outcomes Detect alignment and assessment gaps The Learning Outcome Alignment report enables educators to visualize, analyze, and collect evidence on the mapping between learning outcomes that are defined at different levels of the organization hierarchy, such as academic programs, departments, colleges, courses, etc. This report can also display alignment gaps. The report displays alignment indicators between two sets of learning outcomes, which the user selects through the report filters. The data can filter through one or more semester(s). An alignment indicator displays as a green checkmark reflecting that there is an alignment between the corresponding learning outcome. Use report filtering to select the display of alignment gaps. A red triangle indicates that at least one assessment is aligned. Assessments are necessary to demonstrate the achievement of a learning outcome. A red triangle crossed in black indicates that there are no aligned assessments. A highlighted column or row header indicates a lack of any alignments for the outcome or the org unit.

Excel Files Microsoft Excel – Data from assessment items can be collected via Microsoft Excel. After an assessment is collected, faculty may choose to map items to specific course content and outcomes. In order to meet the assessment reporting intent, the spreadsheet should contain the following information: Student name, student performance for each outcome, student overall performance, aggregate performance for each outcome, aggregate assessment performance. Individual outcome feedback to students is possible; faculty may choose to email students their performance individually or via the Microsoft Word mail merge function. For information on how to give feedback using this process click here. June 2017

Don’t Forget to Leave Feedback for Me Enter to win a Starbucks gift card! June 2017