1. Getting it right: on-line formative assessment of numeracy skills
Dr Amanda Cain, Ms Charmian Dawson and Professor Elizabeth Shephard
Structural and Molecular Biology
University College London

2. Aims of the project
To assess the underlying numerical abilities of students within our second year course
To improve these skills
To support development of ability and confidence in solving numerical problems
Identified a weakness in project students who struggled to make up solutions
In year 2 analysis found concepts of percentage solutions and dilutions were often misunderstood
Added numerical questions to all pre-practical quizzes, tests and exam paper

3. Numeracy question from examination paper
75.2% could work out simple molarity (How much NaCl must you weigh out to prepare 300mL of a 0.1M solution of NaCl? The MW of NaCl is ), only 52.3% could work out how much salt needed to prepare 1L of a 6% (w/v) solution?

4. Relationship between A’ level grades and numeracy score
Aftr first year of pre-prac quizes, 2006, analysed the scores in numerical exercises to look for trends. Found no link to mathematical or overall academic ability – see above, First plot compared the a’level maths grade to scores in numeracy tests. Second takes their overall a’level point score and compares to numeracy grade. See little correlation with eiother previous maths score or overall ability.
Looked at results by degree programme (not shown here)– saw a trend to the more molecular subjects scoring better (Biochem, Biotech, Mol Biology) in comparison to Biol, Genetics, Biomed Sci. Group that performed worst were the Biochemical engineers

5. Pre-Practical Quizzes
Completion required prior to attendance in lab
Short series of questions on moodle - theoretical and numerical
Pass mark of at least 60%
First trialled in second year Molecular Biology
Solution was to introduce a series of pre-practical quizes that had to be completed prior to arrival at the labs.
Delivered by VLE – in our case moodle. Used multiple choice to ensure they have read and understood the theoretical basis of each session and calculations to give them practice of the maths behind each session.
Monitor completion of quiz as we take attendance – link barcode to quiz score so as they scan in we can see immediately if they have completed each quiz – if not they have to go away!
Quizes do not count towards assessment of module but are required to be complete and calculations of the same format are included in both in-course tests and exams
Second year Mol Biol – large course. Taken by Biochem, Biotech. Mol Biol, Genetics, H Genetics, Neuro and some Biol and Biomeds. This year have 165 registered.

6. Support for numerical questions
Numeracy helplines
Practice quizzes
Answer files
Videos
Regular pracice at calculations is important but studens also need to understand the concepts first. Therefore produced a series of numerical helplines as a summer project. Had a team of undergrads in – tell us what they find difficult and come up with ways of explaining it to fellow students. Broke down each step of the calculations.
Also set up a practice quiz they could access at any time. Faced with same basic questions but the numers change each time so they can re-inforce their earning.
Finally after each lab session we put up a sheet explaining how each of the calculations should be attempted.

7. Numeracy Helplines Designed by students Simple explanations
Equations to memorise
Worked examples

8. Numeracy helplines
Designed by a student for students. Range of help lines to cover different subjects. Have theme running through – little hearts for forulae they should know by heart etc. Not necessarily the formatting/design we would have chosen! Also gives worked examples they can follow – take them through step by step (student who designed explained that sometimes academics miss bits out as they should be obvious.

9. Weight/Volume solutions (w/v)

10. Help line usage Just under 200 students on the course
Pre-prac quizes in weeks1-5, test in week 6 and 12

11. Practice quizzes Series of questions with randomised values
Created within moodle and later in flash
Repetition
Worked answers available
Used regularly

12. Practice quizzes 193 students on course 302 separate attempts
Over 200 views of the answer files

13. Videos

14. Video files

15. Advantages of on-line support resources
Used at anytime and repeatedly
Regular practice of numerical concepts
No fear of embarrassment
Immediate results/feedback
Helps us to identify areas of weakness which require additional support

16. Comparison of equivalent questions on molarity and percentage solutions
Can see from above results that regular practice at these sorts of calculations improves overall performance. Loked at the percentage of students getting full marks on both a molarity calculation and one involving % solutions. See a good increase in performance between a year when pre-practical quizes were not used and this last academic year where students were practicing regularly.

17. Analysis of numerical results
Analysis of this years scores.
Students had three possible attempts at each numeracy quiz. Looked at how their mark improved over the weeks. Graph shows averages of best attempt for each student, by week and the number of attempts required to gain that score. Average scores remain similar each time but the number of attempts drops over the first three weeks. (bit of a blip in week 4 – lower average and slight rise in no. of attempts – getting near test time?, deadlines looming?) Week 4 questions more wordy – relate to Southern buffers – several references to stock solutions 10X etc (only one of these in week 3). May be week 4 was harder – required more thought. By end of period also wise up to the fact that they don’t have to get 100% every week.

18. Student feedback
Talk about free text comments as well.

19. Summary
Continued practice at numerical problems has had a positive effect
The support materials are appropriate and well utilised
Thanks to UCL for funding the creation of our support materials