Novel Original Transformative Feedback for Automated Individualized Learning David Garmire – University of Hawaii Eric Torng – Michigan State University James J. Pembridge – Embry-Riddle Aeronautical University Cinda Heeren – University of Illinois-Urbana Diana Franklin – University of California, Santa Barbara Franz Hover – Massachusetts Institute of Technology Dawn M. Reinhold – Michigan State University Michael Dickey – North Carolina State Ali Mehrizi-Sani – Washington State University Timothy A Bigelow – Iowa State University

Objective of Innovation Provide feedback to students Promote self-guided learning Provide in-time, high-quality assessment Provide feedback to instructors Promote just-in-time lectures Improve time-efficiency for formative assessments Identify strugglers early Support research in engineering education

Desired Outputs Out-of-Class Assessment Tools: Promote: Individualized Formative Automated Open-Source/Open-Data/Extendable Infrastructure Easy to Use Flexible Dynamic Promote: Student reflection Creative solutions

Challenges to Implementation Available software Cumbersome Expensive Proprietary Discipline Specific Balance between feedback and over-dependence Psychological effectiveness of tool Funding to develop, implement, and support Time Dissemination of current platforms and content Automation of higher-order assessments Academic reward structure

Successes Higher achievement of student learning outcomes Less procrastination on student assignments Students more proactive about their own learning Increased emphasis by students on mastery based outcomes, less on grades Immediate formative student feedback Enabled just in-time lecture Less student frustration More in-class participation

Impact Documentation We made PowerPoint slides with cartoons. Class/learning analytics Likert questions Completion times, grade improvement Qualitative student feedback Anecdotal Student Evaluations Focus Groups Assessing student learning 1, 3, and/or 5 years after course completion

Scaling-Up Innovation Market it to department, college, and university colleagues. Arrange informal meetings at discipline specific scientific meetings. Make the tool modular and easily customizable. Allow for open-data plan for other instructors to download previously created tools for their specific courses.