Maintaining student engagement with formative assessment (using Questionmark) Tracey Wilkinson Centre for Biomedical Sciences Education School of Medicine, Dentistry and Biomedical Sciences

Plan why use objective assessments? advantages of regular formative assessment Questionmark – types of questions available – examples – have a go! exchange ideas final word

Why use objective assessments? diagnostic (often early) self-assessment (feedback to students) continuous, formative (feedback to students and teachers) summative (counted towards final mark)

Formative assessment improves academic performance feedback element particularly important – allows both teacher and student to monitor progress engagement – summative element increasing requirement for online / computer-assisted assessment

Advantages of online tests – advantages for lecturer reduced time commitment logistics of individual feedback (especially for large classes) allow testing of wide range of subjects more frequent assessment possible analysis of questions – advantages for student convenient access immediate feedback (individual) monitor progress

Disadvantages of online tests time and effort to construct good questions higher order questions (Bloom’s taxonomy) written communication skills or creativity not easily tested

Questionmark Perception Assessment Management System – Queen’s pilot project – easy to use online assessment tool – more question types available (than QOL)

Examples from Questionmark

Have a go! In pairs, or on your own, discuss the types of questions that might be suitable in your particular discipline. - true-false - multiple choice / response - matching - ranking - fill in / select a blank - text match - hotspot / drag and drop - Likert Compare the questions you have constructed with your neighbour. Any suggestions for improvement? Any questions that might test higher cognitive ability? Now try constructing a few simple questions in your own discipline(s).

Logic and reasoning Train service suffers when a railroad combines commuter and freight service. By dividing its attention between its freight and commuter customers, a railroad serves neither particularly well. Therefore, if a railroad is going to be a successful business, then it must concentrate exclusively on one of these two markets. For the argument to be logically correct, it must make which one of the following assumptions? a.Commuter and freight services have little in common with each other. b.The first priority of a railroad is to be a successful business. c.Unless a railroad serves its customers well, it will not be a successful business. d.If a railroad concentrates on commuter service, it will be a successful business. e.Railroad commuters rarely want freight services as well. American LSAT (Law School Admissions Test) Answer = c

Logic and reasoning A small software firm has four offices, numbered 1, 2, 3 and 4. Each of its offices has exactly one computer and one printer. Each of these eight machines was bought in either 1987, 1988 or The eight machines were bought in a manner consistent with the following conditions: – the computer in each office was bought either in an earlier year than, or in the same year as, the printer in that office – the computer in office 2 and the printer in office 1 were bought in the same year – the computer in office 3 and the printer in office 4 were bought in the same year – the computer in office 2 and the computer in office 3 were bought in different years – the computer in office 1 and the printer in office 3 were bought in 1988 If the computer in office 3 was bought in an earlier year than the printer in office 3, then which one of the following statements could be true? a.The computer in office 2 was bought in b.The computer in office 2 was bought in c.The computer in office 4 was bought in d.The printer in office 4 was bought in e.The printer in office 4 was bought in American LSAT (Law School Admissions Test) Sample Test, June 1999 Answer = b

Regular assessments Have any of you used regular formative assessment (paper, personal response, Queen’s online, other) with feedback? In groups of two to three, discuss the ways in which you have used, or may be planning to use, regular formative assessment to enhance the learning in your classes. Be prepared to give feedback to the whole workshop. Feedback – any areas of good practice?

Final word the students – uptake – find them useful – ask for more – pressure to do well as teacher sees them – good revision for tests

References Brown G, Bull J and Pendlebury M (1997) Assessing student learning in higher education. London: Routledge. Bull J and McKenna C (2004?) Good practice guide in question and test design. Fry H, Ketteridge S and Marshall S (2003) A handbook for teaching and learning in higher education. London: RoutledgeFalmer. Miller T (2009) Formative computer-based assessment in higher education: the effectiveness of feedback in supporting student learning. Assessment and Evaluation in Higher Education 34: Nicol D (2007) E-assessment by design: using multiple choice tests to good effect. Journal of Further and Higher Education 31: Wiliam and Black (2003) In praise of educational research: formative assessment. British Educational Research Journal 29: