Martin Greenhow & Abdulrahman Kamavi Mathematical Sciences Brunel University maths e.g. – a web assessment application for STEM and beyond

Outline 1. CAA at Brunel University 2. Characteristics of questions and variety of disciplines 3. Mathematics for Economics (Metal Project) Students and CAA: perceptions and reality 4. Students and CAA: perceptions and reality 5. maths e.g. 6. Conclusions

CAA at Brunel University Mathletics – not that Mathletics! Written within Question Mark Perception version 3.4 Javascript, MathML, SVG Questions database spans GCSE, A-level, undergraduate topics Questions database spans MC, NI, RNI, TFU, MR, NI+confidence, Revealed MC etc (no free-form maths input … yet): PROS/CONS? Hundreds of users take circa 30,00 tests pa. Students from Economics, Electrical and Electronic Engineering, Financial Computing, Foundations of Engineering, Foundations of IT, PGCE and Sports Science. Much commonality Low-stakes summative assessment Format: best-ever mark from their first 5 attempts counts towards their module mark; not invigilated and group work is allowed/encouraged. Exam pass required!

TF(U) question decontextualised numbers and variable names (here d and S) randomised Fonts & Colours Low facility – high discrimination Success largely independent of question type

Computer science question Pre-processing of input Table generated on the fly Required element positions randomised

Feedback – can we have too much? Engineering/built env/maths question with schematic numbers change ALL of the formative feedback

Feedback - contextualised economics question part of the formative feedback SVG diagram realised according to the random parameters in the question related material button links to any web resource (via centrally-held lookup array)

Biosciences - decontextualised

Nurses - contextualised The only question with Imperial units! Realistic weights Realistic weight loss!

Adult numeracy? Pictogram changes Scenario changes Notice the key -> mal-rule

Mathematics for Economics (year-by-year changes in assessment) , : no CAA; A and non-A cohorts – results indistinguishable; two class tests, one 3-hour exam set by economics staff (10 days - times 2?) :CAA introduced; 3-hour exam only (2 days - times 2?) , : one cohort, admission AS level mathematics; exam pass mandatory; 2-hour exam only (set by MG) (3 days!!!!!!)

Students perceptions 1 (147 returns) 79% of students use the CAA both for assessment and learning material, 21% for assessment only; 94% use all 5 attempts (first one or two attempts are often used to get feedback); most students say they would use the assessment if not compulsory; 90% of students aim to achieve a mark of at least 80%; 84% plan to use the CAA as part of their revision;

Students perceptions 2 (147 returns) failure to distinguish clearly between their enjoyment of maths and that of CAA per se; the more they enjoy mathematics (or CAA) the more they use the feedback as a learning resource; weaker students benefited relatively more from the CAA: 24% of good students use the CAA only for marks in contrast to weak students where this is 15%; mathematics module is perceived as being of comparable difficulty to other modules maths being: very much easier (4%), easier (33%), about the same (31%), harder (29%), very much harder (3%).

Year-on-year results (% in grade/% mark)

DeSTRESS JISC Project Builds on the Metal question design methodology and experiences to test basic stats for social sciences Released new material end August 2011 Some trials have taken place already Hand calculation with realistically-sized data sets – solution to link with external software such as Excel, SPSS Real data? Data cannot be designed to have certain features or not. Problems with keeping the data and answers current without accessing live web sites (problems of access and communication with the marking scheme). Another challenging area is the interpretation of charts and graphs

A statistics question linked to Excel

A statistics question - tables or formula?

What about literacy? Some things can be done Must disable Word Should enable internet VERY hard!!!

Teacher interface – shop on Amazon?

Conclusions Javascript, MathML and SVG provide a rich environment for setting objective questions Positive effects on students perceptions and on exam performances Widely applicable database of questions MathsE.G. Try it at: works on all browsers, mobile devices(?), link to VLEs