Strategies for engaging students in computer based assessment – Stage 1, Taking stock Joanne Smailes Northumbria University

Computer Aided Assessment (CAA) First became interested in mid 1990’s Question Mark But.. Large numbers Admin by Academic However… Blackboard VLE introduced in 2000/01

Background – Year 1 Formative use only – revision Well Received Only 25% of class made use of tests Around examination period

Background – Year 2 Used for one summative component Technical problems experienced with – restricting to one attempt only – using a timer Well received

Year /03 Linked to set text. 10 chapters – 10 quizzes. Formative objective but… 3 tests randomly selected for summative mark.

Academic Enquiry Based on… Revival of interest in formative assessment –Students now expect fast and direct feedback –Elearning environment overcomes reduced staff-student contact –Students can address current “deficiencies”

However… CAA is associated with Surface Learning Getting by Fighting against boredom SURFACE Genuine understanding Stimulating Challenge DEEP Good Marks Game Playing ACHIEVEMENT

Reviewing Year 3 Feedback Instruments –Three part questionnaire –Group feedback analysis (GFA) Questionnaire –Response rate 62% –Even gender split –30% students overseas (mainly China) GFA –Three occasions 18 students –Even gender split –Around one-half overseas

Questionnaire Results Three main themes emerged… Technology Feedback Separating formative & summative plus Observation on Approaches to learning

Technology No operational problems found Site Access Requested.. Better access from home

Feedback Design carefully considered –Friendly and simple –Focussed on task (hint and tips) –No feedback if correct –Grades given Very positive comments received throughout Further suggestion made –Page references –Detailed working –Review common mistakes

Formative vs Summative Feedback including correct answers Time Limit Multiple attempts Make all quizzes summative Question/ Quiz weighting

Approaches to learning Achievement 35% Surface 30% Deep 20% Mix 15% 10% 75% CAA General

BM118 Final Results 2000/012001/022002/03 Mean Std. Dev Median52 67 Fail rate20%22%13% 2002/03 results only became available after paper submission

Conclusions Institutions need to consider home access to elearning materials Asynchronous as well as immediate feedback Removal of Summative element? Blended assessment – rethink CAA and learning approaches?

Questions welcomed