Self-assessing with adaptive exercises Chye-Foong Yong & Colin A Higgins University of Nottingham

Introduction Computer-based test –Fixed response question types E.g. multiple choice, multiple response, hotspot –Free response questions types E.g. essay, programming, diagrammatic –Fixed set of questions for all students Either too easy or too difficult

Introduction (cont’d) Computerised-adaptive test –Questions of varying levels of difficulty to suit individual students’ ability –Drawback: ‘fairness’ of assessment Thus, better as formative than as summative assessment

ASAM Stands for “Adaptive Self Assessment Master” An extension to CourseMarker (CM) –CM has the capability to mark programming and diagrammatic questions automatically and give immediate feedback

ASAM (cont’d) Customise the questions to individual students by generating a personalised set of self-assessing questions Multi-modal questionnaires

Components of ASAM Question bank –To store all the questions Authoring tool –With which lecturers use to create questions and questionnaire skeletons Student model –To store the profiles of individual students

Components of ASAM (cont’d) Assessment generating system –Which students are to log on to and to attempt the questions presented Marking tool –To mark students’ submission and pass on the data to the student model

Interaction among various components of ASAM

Evaluation With 2003/4 first year Computer Science students –In the first part/semester of the module Two self assessments (SAs) –of mainly fixed response questions –First SA (after 1 st MCQ test): fixed format –Second SA (after 2 nd MCQ test): adaptive format

Comparison of mean results in 2002/3 and 2003/4 Mean NMCQ1MCQ2MCQ3 2002/ %45.05%55.51% 2003/4 - ASAM %44.18%44.33% 2003/4 – no ASAM %40.80%36.41%

Comparison of mean scores in 2002/3 and 2003/4

Conclusion Self assessment is generally beneficial to students, may it be fixed or adaptive Adaptive self assessment adds value to their learning

Future work Incorporate IMS QTI Making ASAM to be interoperable with other question banks and intelligent tutoring systems