Physics 1A: Questions, Questions and more Questions Liz Angstmann Physics Talks will be limited to 15 minutes, including any questions or discussion. We would like the Showcase to be as collaborative as possible, and we encourage you to factor in time for discussion if appropriate. As a general guide, we recommend that you plan to present no more than 10 slides to ensure that you stick to time.

Why did we need online tests? Students were struggling with the final exam; they have tutorial problems but often do not try these for themselves, they have solutions videos (our problem solving workshops are designed to help address this also) Students “collaborated” on the online quizzes, students who did not receive help from peers were unhappy that other students did, we reduced the weighting of our online quizzes in response to this The assessment policy changed and we were no longer compliant. We had a 70% final exam that students had to pass. We did not want to reduce the weighting of invigilated assessment. Now: 10% x 2 invigilated tests, 10% at home quizzes, 20% lab, 50% final exam Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase Considerations Research has shown that answering questions is one of the most effective ways to study a subject. Practice testing and distributed practice works well: Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58. It would be nice to align assessment with what works in terms of learning: make it not in the student’s interest to cheat on the online quizzes Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase The Plan…. Create a large bank of questions (5 three part questions based on each of the 36 lectures) – started with lecture learning outcomes Run secure invigilated online tests in the lab based on these banks Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase STACK questions STACK allows for: Algebraic answers Randomized variables Randomized strings Multi-part linked questions in a bank Carry on marking when a student makes a mistake Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase Examples Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase Examples Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase In the lab SEB: Safe Exam Browser Session 2C 2018 Engineering Education Showcase

When you enter: Put your bag and phone away under the desk Login to the computer Place your student card on your desk Wait for instructions before starting! To open your test click on this folder in the dock and then choose the one for your course. Once you start you have 40 minutes or until the end of the exam session, on the hour, whichever comes first!

2018 Engineering Education Showcase Results: Test 1 Physics 1A Average 5.68 Higher Physics 1A Average 6.48 Held during week 8 of semester Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase Results: Test 2 Physics 1A Average 4.98 Higher Physics 1A Average 5.69 Held during week 13 of semester Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase Results: Final exam Physics 1A y = 0.64x + 21 Correlation coefficient: 0.77 Exam improvement from 2017: 12% Higher Physics 1A y = 0.60x + 21 Correlation coefficient: 0.76 Exam improvement from 2017: 6.4% Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase What went wrong? Technical issues with questions, fixed now…. Significant figures STACK update….. Would be nice to be able to categorize questions on difficulty as well as lecture Security update during week of in class quiz MyExperience: a lot of students did not like these even though they did better. But overall I would count it a success. Nothing too drastic happened and the students did better on the exam. Session 2C 2018 Engineering Education Showcase

2018 Engineering Education Showcase Other things This semester we are trialing scanning exam papers and marking them online for our Physics 1B courses. This was piloted by David Kellerman in engineering. Session 2018 Engineering Education Showcase

2018 Engineering Education Showcase Thanks to: Question fixing team Elliot Griffiths Jeremy Nugent Bryony Lanigan Digital uplift team James Vassie Jeremy Nugent Laura House Physics IT (SEB) Kristien Clayton David Jonas Session 2C 2018 Engineering Education Showcase