1. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ beliefs and practices on the use of computer-aided assessment to enhance learning

2. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Introduction  Background to CAA at the target institution  Background to this project  Research questions  Methods  Findings at this stage  Discussion and conclusions

3. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Background to CAA at the target institution

4. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson The HELM project Helping Engineers Learn Mathematics  Started in 2002 by a consortium of universities  Includes workbooks and a CAA system  Question bank with thousands of maths questions Concurrently  A separate bank of questions was developed for maths students Harrison et al. (2007), HELM (2006).

5. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Example CAA question

6. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Example CAA solution

7. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Ten years on  How do lecturers and students use CAA now?  What sort of student improvement do lecturers and students desire from CAA?  Does CAA practice affect whether these outcomes are achieved?

8. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Background to this project

9. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Previous work Student focus groups  How students use CAA  The nature of the feedback given  The definition of student improvement  The role of the lecturer Broughton, Hernandez-Martinez and Robinson (2011, 2012a, 2012b).

10. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Research questions

11. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ use of CAA 1. How is CAA implemented in first year mathematics modules for mathematics and engineering students at this institution? 2. Why are lecturers using CAA? 3. What are lecturers’ perceptions of issues arising? 4. How are lecturers dealing with these issues? Roth and Lee (2006).

12. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Methods

13. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ views Questionnaire  Nine completed questionnaires  Questions on practice and opinion Interviews  Six took part in follow-up interviews  Questions expanded on questionnaire responses

14. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Findings at this stage

15. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ implementations of CAA CAA coursework test Paper-based coursework test No test InvigilatedNon-invigilatedInvigilated Practice test access granted more than one week before a test Practice test access granted after the test 0211 Practice test access granted up to one week before a test Practice test access granted after the test 0100 Practice test access not granted after the test 3100

16. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Why lecturers are using CAA Save time by continuing inherited practice Support student learning by encouraging them to practise Assess students more frequently by offering immediate feedback Accepting advice from colleagues following casual conversations

17. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Lecturers’ perceptions of issues arising The questions are formulaic and not challenging enough. The questions do not test conceptual knowledge and understanding. The feedback is not beneficial to every student. Lecturers do not know how students approach the tests.

18. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson How lecturers deal with CAA issues Authoring new CAA questions that challenge conceptual knowledge Replacing the online test with a paper version Introducing other assessments and restructuring module assessment Reducing the weighting of CAA towards the overall module mark

19. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Discussion and conclusions

20. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Conclusions CAA saves time setting tests but it is time-consuming to write new CAA questions The question bank is a treasured resource but it does not test students’ conceptual knowledge CAA encourages students to work between lectures but it is difficult to ascertain how students approach the tests

21. Stephen Broughton Paul Hernandez-Martinez Carol L. Robinson Further information This presentation sbpr.es/sf12a References sbpr.es/sf12b