1. Physics 1A: Questions, Questions and more Questions
Liz Angstmann
Physics
Talks will be limited to 15 minutes, including any questions or discussion. We would like the Showcase to be as collaborative as possible, and we encourage you to factor in time for discussion if appropriate. As a general guide, we recommend that you plan to present no more than 10 slides to ensure that you stick to time.

2. Why did we need online tests?
Students were struggling with the final exam; they have tutorial problems but often do not try these for themselves, they have solutions videos (our problem solving workshops are designed to help address this also)
Students “collaborated” on the online quizzes, students who did not receive help from peers were unhappy that other students did, we reduced the weighting of our online quizzes in response to this
The assessment policy changed and we were no longer compliant. We had a 70% final exam that students had to pass. We did not want to reduce the weighting of invigilated assessment.
Now: 10% x 2 invigilated tests, 10% at home quizzes, 20% lab, 50% final exam
Session 2C
2018 Engineering Education Showcase

3. 2018 Engineering Education Showcase
Considerations
Research has shown that answering questions is one of the most effective ways to study a subject. Practice testing and distributed practice works well: Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4-58.
It would be nice to align assessment with what works in terms of learning: make it not in the student’s interest to cheat on the online quizzes
Session 2C
2018 Engineering Education Showcase

4. 2018 Engineering Education Showcase
The Plan….
Create a large bank of questions (5 three part questions based on each of the 36 lectures) – started with lecture learning outcomes
Run secure invigilated online tests in the lab based on these banks
Session 2C
2018 Engineering Education Showcase

5. 2018 Engineering Education Showcase
STACK questions
STACK allows for:
Algebraic answers
Randomized variables
Randomized strings
Multi-part linked questions in a bank
Carry on marking when a student makes a mistake
Session 2C
2018 Engineering Education Showcase

6. 2018 Engineering Education Showcase
Examples
Session 2C
2018 Engineering Education Showcase

7. 2018 Engineering Education Showcase
Examples
Session 2C
2018 Engineering Education Showcase

8. 2018 Engineering Education Showcase
In the lab
SEB: Safe Exam Browser
Session 2C
2018 Engineering Education Showcase

9. When you enter:
Put your bag and phone away under the desk
Login to the computer
Place your student card on your desk
Wait for instructions before starting!
To open your test click on this folder in the dock and then choose the one for your course. Once you start you have 40 minutes or until the end of the exam session, on the hour, whichever comes first!

10. 2018 Engineering Education Showcase
Results: Test 1
Physics 1A
Average 5.68
Higher Physics 1A
Average 6.48
Held during week 8 of semester
Session 2C
2018 Engineering Education Showcase

11. 2018 Engineering Education Showcase
Results: Test 2
Physics 1A
Average 4.98
Higher Physics 1A
Average 5.69
Held during week 13 of semester
Session 2C
2018 Engineering Education Showcase

12. 2018 Engineering Education Showcase
Results: Final exam
Physics 1A
y = 0.64x + 21
Correlation coefficient: 0.77
Exam improvement from 2017: 12%
Higher Physics 1A
y = 0.60x + 21
Correlation coefficient: 0.76
Exam improvement from 2017: 6.4%
Session 2C
2018 Engineering Education Showcase

13. 2018 Engineering Education Showcase
What went wrong?
Technical issues with questions, fixed now….
Significant figures
STACK update…..
Would be nice to be able to categorize questions on difficulty as well as lecture
Security update during week of in class quiz
MyExperience: a lot of students did not like these even though they did better.
But overall I would count it a success. Nothing too drastic happened and the students did better on the exam.
Session 2C
2018 Engineering Education Showcase

14. 2018 Engineering Education Showcase
Other things
This semester we are trialing scanning exam papers and marking them online for our Physics 1B courses. This was piloted by David Kellerman in engineering.
Session
2018 Engineering Education Showcase

15. 2018 Engineering Education Showcase
Thanks to:
Question fixing team
Elliot Griffiths
Jeremy Nugent
Bryony Lanigan
Digital uplift team James Vassie Jeremy Nugent Laura House
Physics IT (SEB)
Kristien Clayton
David Jonas
Session 2C
2018 Engineering Education Showcase