Making Grades Meaningful – Building a Community of Practitioners Wednesday, June 28, 2017 American Society for Engineering Education Conference & Exposition

Standards-based Grading (SBG) Research Team Adam Carberry Arizona State University Sara Atwood Elizabethtown College Matt Siniawski Loyola Marymount University Heidi Diefes-Dux Purdue University

Overview of Workshop SBG 101 SBG Participant Panel Roundtable Discussion & Design Wrap-up Q&A

Overview of Workshop SBG 101 SBG Participant Panel Roundtable Discussion & Design Wrap-up Q&A

Traditional, Score-based Grading   Student Homework Total (%) Quiz Total (%) Midterm Exam (%) Final  Total Course Grade John 78 80 77 83 B- Bill 50 60 90 87 72 C- Susan 70 75 C+ Felicia 85 95 B Jane 100 93 A What do these student course grades represent? knowledge, effort, or ability? some combination of all three? none of the above?

Traditional, Score-based Grading Such a system inherently fails to meet the conditions for sound assessment of student work and learning So why do we use it? long tradition and remains widely used and promoted internationally no studies to understand, manage and improve, i.e., the essence of ignored and deplored

Call for Change US Department of Education (2006) urged improvement and increased accountability to monitor student learning in higher education Specific suggestions: Make course objectives explicit to students beyond course syllabi, so that faculty can follow student development towards achieving them Provide appropriate feedback to effectively increase student learning

Standards-based Grading (SBG) assessment of student’s mastery of essential standards or learning outcomes for a course Provides clear, meaningful, and personalized feedback Directly connects students’ development toward specific objectives (or standards) Provides fairness, transparency, and flexibility in the grading process Promotes learning and encourages continuous improvement Makes grades meaningful Provides a highly effective tool for program assessment

Learning Objectives are the Starting Point of the SBG System Assessment Assignments Include practice with feedback

Weighting of objectives SBG Gradebook (For each student/team) Assignments Learning Objectives Weighting of objectives Course grade Weighting of assignments

SBG Gradebook Student: Jane Doe (02/15/2016) Project 1 Assignments Design & Build (DB) Modeling (M) Testing & Eval (TE) Comm & Org (CO) Teaming (T) 1: Design description 2 - 3 2: CAD model and 2D print 3: Mechanism model and calculations 4: Project report Learning Objective Scores 2.0 2.5 3.0 Project Score 2.4 Project Grade A- Progress Level: 3 - Strong development 2 - Demonstrates appropriate development 1 - Approaching appropriate development 0 - Needs practice and further development or not assessed Objective Weighting: DB – 25%; M – 25%; TE – 20%; CO – 15%; T – 15%

Overview of Workshop SBG 101 SBG Participant Panel Roundtable Discussion & Design Wrap-up Q&A

SBG Participant Panel Mike Foster Beverly Kris Jaeger-Helton Blake George Fox University Beverly Kris Jaeger-Helton Northeastern University Blake Hylton Ohio Northern University Debra Mascaro University of Utah

Question 1 How long have you been using SBG and in what courses?

Question 2 What are some of the reasons why you are trying out standards-based grading?

Question 3 What have been some of your challenges with SBG? What have you done to overcome these challenges?

Question 4 Do you have any tips and recommendations for those interested in using SBG?

Question 5 How have you managed to integrate SBG into learning management systems (LMS)?

Overview of Workshop SBG 101 SBG Participant Panel Roundtable Discussion & Design Wrap-up Q&A

Break Time & Form Groups Pick a course you want to work on Form working groups (~5 minutes) First year engineering courses Project/Design courses Content courses Skills courses (Matlab, CAD, etc.) Meet your group during the next 5-10 minute break

Learning Objectives are the Starting Point of the SBG System Assessment Assignments Include practice with feedback

Activity: Learning Objectives (10 minutes) For your course, write 3-4 most fundamental learning objectives. For 1 objective, write 2-3 sub-objectives. Share with a partner and help each other improve the learning objectives. Prioritize most relevant and critical Justify quantitative targets for each Identify Product Design Specifications … Create and maintain CAD drawings Fabricate a working Prototype

Use strong action verbs to identify observable behaviors

Activity: Assignments (10-15 minutes) Plan the assignments designed to assess that learning objective. Share with a partner and discuss approaches to the course activities. Every day, in group Practice problems Help from instructor, TAs, and other students Feedback: real-time help + solutions, participation graded Weekly, individual 20 minute quiz in class Feedback: immediate explanation of solution by instructor, self-graded with rubric Monthly, individual Exam Culmination of unit Combines concepts Feedback: graded by instructor with rubric

Activity: Rubric (10-15 minutes) For your learning objective, write a 3-5 level rubric listing expectations of performance at each level. Share with a partner and help each other improve the rubrics. Fully Achieved Plot command used Hold or equivalent is used Time is independent variable Both plots formatted correctly Appropriate title Appropriate axis commands and labels Partially Achieved 1-2 errors from the fully achieved list Underachieved 3-4 errors from the Fully Achieved list No Evidence More than 4 errors from the Fully Achieved list

Overview of Workshop SBG 101 SBG Participant Panel Roundtable Discussion & Design Wrap-up Q&A

Wrap-up Web Resources Website - http://www.sbghub.lmu.build

Wrap-up Web Resources Website - http://www.sbghub.lmu.build SBG Hub Forum – create a Login to join

Wrap-up Web Resources YouTube Channel – Standards-based Grading

Wrap-up Literature Resources ASEE PRISM Article – Made to Measure

Wrap-up Literature Resources Vaishnav, S. & Carberry, A. (2017). Student perspectives on standards-based grading used in engineering project-based courses. American Educational Research Association (AERA) Annual Conference, San Antonio, TX. Diefes-Dux, H. A. (2016). Student reflections on standards-based graded. ASEE/IEEE Frontiers in Education Conference, Erie, PA. Carberry, A., Siniawski, M., Atwood, A., & Diefes-Dux, H. (2016). Best practices for using standards- based grading in engineering courses. American Society for Engineering Education Annual Conference & Exposition, New Orleans, LA. Hylton, J. B., & Diefes-Dux, H. A. (2016). A standards-based assessment strategy for written exams. American Society for Engineering Education Annual Conference & Exposition, New Orleans, LA. Marbouti, F., Diefes-Dux, H. A., & Madhaven, K. (2016). Models for early prediction of at-risk students in a course using standards-based grading. Computers & Education, 103, 1-15. Marbouti, F. (2016). A standards-based grading model to predict students’ success in a first-year engineering course (Unpublished dissertation). Purdue University, West Lafayette, IN. Atwood, S., Siniawski, M., & Carberry, A. (2014). Using standards-based grading in engineering project courses. Design in Engineering Education Division (DEED) – American Society for Engineering Education Annual Conference & Exposition, Indianapolis, IN. Carberry, A.R., Siniawski, M., & Dionisio, J. (2012). Standards-based grading: Preliminary studies to quantify changes in student affective and cognitive behaviors. ASEE/IEEE Frontiers in Education Conference, Seattle, WA. Siniawski, M., Carberry, A., & Dionisio, J. (2012). Standards-based grading: An alternative to score- based assessment. American Society for Engineering Education Pacific Southwest Section Conference, San Luis Obispo, CA.

Most Recent/Upcoming Work ASEE 2017 Nathan Hicks & Heidi Diefes-Dux – Grader Consistency in using standards-based rubrics REES 2017 Heidi Diefes-Dux – Student self-reported use of standards-based grading feedback and resources Eunsil Lee, Adam Carberry, Sara Atwood, Heidi Diefes-Dux & Matt Siniawski – Faculty perceptions before and after implementation of standards-based grading

Overview of Workshop SBG 101 SBG Participant Panel Roundtable Discussion & Design Wrap-up Q&A

SBG Website | http://www.sbghub.lmu.build Sara Atwood atwoods@etown.edu Adam Carberry adam.carberry@asu.edu Heidi Diefes-Dux hdiefes@purdue.edu Matthew Siniawski matthew.siniawski@lmu.edu Making Grades Meaningful - Standards-based Grading for Engineering Project Courses, Grant No. DUE-1503794 SBG Website | http://www.sbghub.lmu.build