Getting it right: on-line formative assessment of numeracy skills Dr Amanda Cain, Ms Charmian Dawson and Professor Elizabeth Shephard Structural and Molecular Biology University College London

Aims of the project To assess the underlying numerical abilities of students within our second year course To improve these skills To support development of ability and confidence in solving numerical problems Identified a weakness in project students who struggled to make up solutions In year 2 analysis found concepts of percentage solutions and dilutions were often misunderstood Added numerical questions to all pre-practical quizzes, tests and exam paper

Numeracy question from examination paper 75.2% could work out simple molarity (How much NaCl must you weigh out to prepare 300mL of a 0.1M solution of NaCl? The MW of NaCl is 58.44. ), only 52.3% could work out how much salt needed to prepare 1L of a 6% (w/v) solution?

Relationship between A’ level grades and numeracy score Aftr first year of pre-prac quizes, 2006, analysed the scores in numerical exercises to look for trends. Found no link to mathematical or overall academic ability – see above, First plot compared the a’level maths grade to scores in numeracy tests. Second takes their overall a’level point score and compares to numeracy grade. See little correlation with eiother previous maths score or overall ability. Looked at results by degree programme (not shown here)– saw a trend to the more molecular subjects scoring better (Biochem, Biotech, Mol Biology) in comparison to Biol, Genetics, Biomed Sci. Group that performed worst were the Biochemical engineers

Pre-Practical Quizzes Completion required prior to attendance in lab Short series of questions on moodle - theoretical and numerical Pass mark of at least 60% First trialled in second year Molecular Biology Solution was to introduce a series of pre-practical quizes that had to be completed prior to arrival at the labs. Delivered by VLE – in our case moodle. Used multiple choice to ensure they have read and understood the theoretical basis of each session and calculations to give them practice of the maths behind each session. Monitor completion of quiz as we take attendance – link barcode to quiz score so as they scan in we can see immediately if they have completed each quiz – if not they have to go away! Quizes do not count towards assessment of module but are required to be complete and calculations of the same format are included in both in-course tests and exams Second year Mol Biol – large course. Taken by Biochem, Biotech. Mol Biol, Genetics, H Genetics, Neuro and some Biol and Biomeds. This year have 165 registered.

Support for numerical questions Numeracy helplines Practice quizzes Answer files Videos Regular pracice at calculations is important but studens also need to understand the concepts first. Therefore produced a series of numerical helplines as a summer project. Had a team of undergrads in – tell us what they find difficult and come up with ways of explaining it to fellow students. Broke down each step of the calculations. Also set up a practice quiz they could access at any time. Faced with same basic questions but the numers change each time so they can re-inforce their earning. Finally after each lab session we put up a sheet explaining how each of the calculations should be attempted.

Numeracy Helplines Designed by students Simple explanations Equations to memorise Worked examples

Numeracy helplines Designed by a student for students. Range of help lines to cover different subjects. Have theme running through – little hearts for forulae they should know by heart etc. Not necessarily the formatting/design we would have chosen! Also gives worked examples they can follow – take them through step by step (student who designed explained that sometimes academics miss bits out as they should be obvious.

Weight/Volume solutions (w/v)

Help line usage Just under 200 students on the course Pre-prac quizes in weeks1-5, test in week 6 and 12

Practice quizzes Series of questions with randomised values Created within moodle and later in flash Repetition Worked answers available Used regularly

Practice quizzes 193 students on course 302 separate attempts Over 200 views of the answer files

Videos

Video files

Advantages of on-line support resources Used at anytime and repeatedly Regular practice of numerical concepts No fear of embarrassment Immediate results/feedback Helps us to identify areas of weakness which require additional support

Comparison of equivalent questions on molarity and percentage solutions Can see from above results that regular practice at these sorts of calculations improves overall performance. Loked at the percentage of students getting full marks on both a molarity calculation and one involving % solutions. See a good increase in performance between a year when pre-practical quizes were not used and this last academic year where students were practicing regularly.

Analysis of numerical results Analysis of this years scores. Students had three possible attempts at each numeracy quiz. Looked at how their mark improved over the weeks. Graph shows averages of best attempt for each student, by week and the number of attempts required to gain that score. Average scores remain similar each time but the number of attempts drops over the first three weeks. (bit of a blip in week 4 – lower average and slight rise in no. of attempts – getting near test time?, deadlines looming?) Week 4 questions more wordy – relate to Southern buffers – several references to stock solutions 10X etc (only one of these in week 3). May be week 4 was harder – required more thought. By end of period also wise up to the fact that they don’t have to get 100% every week.

Student feedback Talk about free text comments as well.

Summary Continued practice at numerical problems has had a positive effect The support materials are appropriate and well utilised Thanks to UCL for funding the creation of our support materials