The role of CAA in Helping Engineering undergraduates to Learn Mathematics David Green, David Pidcock and Aruna Palipana HELM Project Mathematics Education Centre

In This Presentation … HELM Project (Briefly) HELM Learning and Assessment Resources How Engineering Undergrads Use CAA to Learn Mathematics Question Styles Used Conclusions

The HELM Project Helping Engineers Learn Mathematics - HELM Major 3-year curriculum development project, Consortium of five universities, (Loughborough, Hull, Reading, Sunderland, UMIST) £250,000 HEFCE FDTL4 grant (Oct Sept 2005) To enhance the mathematical education of engineering undergraduates by the provision of a range of flexible learning and teaching resources

HELM Environment Mathematics Problem in UK HE (especially, teaching of mathematics to engineering undergraduates) The HELM project produces and disseminates high quality teaching and learning resources supported by a regular CAA testing regime Learning resources –Workbooks –CAL courseware

HELM Assessment Regime HELM Computer Aided Assessment –To reduce burden on staff involved in continuous assessment –To encourage self-assessment (formative) –To drive student learning (to gain the full potential of the other learning resources) A regular pattern with short periods of study followed by assessment drives learning along at a steady pace Question Mark Perception (QMP): web based and CD based implementation

HELM Assessment Regime Currently about 4500 questions; 10,000 questions expected Questions match particular mathematical concepts in support of the topics covered by the HELM workbooks and CAL courseware Questions cloned from a single master question in a library –Ensures the same level of difficulty –Justifies the random selection of questions from a library

CAA: Implementation Integrated web-delivered CAA regime –Self-testing (Formative) –Formal assessment (Summative) CD based CAA regime –Self-testing is straightforward –Formal assessment may be more difficult

Testing Pattern About 2-5 computer based tests a semester 7-10 days of formative testing –Self-testing (Formative) –Users can take the tests unlimited times –No formal record is kept –Detailed feedback (worked solution and hints) is given Detailed feedback (worked solutions) given One-shot summative testing –Structured in exactly the same way as the formative assessment –Students already have sufficient confidence built through similar formative tests

Likes and Dislikes Students like flexibility –Taking tests when they are ready and where thy want –Taking tests as many times as they need, on their own –Getting question specific detailed feedback as worked solutions Students dislike –Unforgiving nature of CAA –No marks for the method and intermediate steps Staff concerns –Question banks for practice & formal testing –Unsupervised testing / cheating

Question Styles Numeric Entry –Simplest response to a math question is a numerical value (either a whole number or a decimal). –Easy to construct and clone –Learner can not easily guess, especially when answers are expected to a certain accuracy –Pre-set tolerances to answers can be allowed –Can give feedback when the answer is not accurate but is within a range –Errors can occur when students enter answers

Question Styles MCQ –Useful when single numeric response is not appropriate –Susceptible to guess work –More difficult to develop and clone

Multiple Numeric Input Involves more than one numeric response per question With mathematics, it may be useful to ask for more than one numeric answer –Roots of a cubic equation –Real and imaginary parts of complex numbers

Multiple Numeric Input Can be used in some circumstances to replace mathematical MCQs reducing chances of guess work Setting conditions for computerised marking becomes difficult

HELM Multi Stage Questions Students dislike their working not being considered when the final numeric answer to a computer based question is wrong In some questions where several stages may have been required before the final result is obtained, the loss of all credit is unfair Longer questions carrying high marks can be broken down into a few individual question stages and partial credit is given for correct responses at each stage

HELM Multi Stage Questions Example Objective: To determine the value of the second derivative of y = x 2 + sin x when x = 1 STAGE - 0 (Preparation stage): Preamble gives the student specific information on answering this type of question and then the whole question is presented

HELM Multi Stage Questions This is a multi-stage question. Credit will be given for each correctly completed stage. If you begin the question you must go on to completion. You may not return to a stage after submitting the answer. You may not return to the question at a later time. Click on the NEXT button to see the question. Determine the value of the second derivative of y = x 2 + sin x when x = 1 Click on the NEXT button to begin stage 1.

HELM Multi Stage Questions STAGE 1: The first part of the question is presented and the learner will be asked to submit to gain the marks allocated for this stage

HELM Multi Stage Questions STAGE 1: Determine the first derivative of y = x 2 + sin x. (A) x + cos x 2 (B) x 3 + cos x 3 (C) 2x + sin x (D) 2x - cos x (E) None of (A), (B), (C) or (D) Select one of the 5 options, then click SUBMIT. This stage is worth 2 mark(s)

HELM Multi Stage Questions After submitting the answer, the student moves to STAGE 2. Feedback for STAGE 1 is given STAGE 2: The correct solution for STAGE 1 is revealed to the student who now has the task of completing the second stage (in this case, determining the second derivative) using answers from STAGE 1

HELM Multi Stage Questions STAGE 2: The correct answer to stage 1 was (2x + cos x). Now determine the second derivative of y = x 2 + sin x. (A) 2 + sin x (B) 2 – cos x (C) 2 – sin x (D) 2 + cos x (E) None of the above Select one of the 5 options below, then click SUBMIT. This stage is worth 1 mark(s)

HELM Multi Stage Questions After submitting the answer, the student moves to STAGE 3 and feedback for STAGE 2 is given STAGE 3: The correct solution for STAGE 2 is revealed to the student who now has the task of completing the second stage (in this case, determining the value of the second derivative when x=1) using answers from STAGE 2

STAGE 3: The correct answer to stage 2 was (2 – sin x). Now determine the value of the second derivative of y = x 2 + sin x when x = 1. Enter your answer, correct to 2 d.p. in the box below, then click SUBMIT. Answer This stage is worth 1 mark(s) HELM Multi Stage Questions This being the last stage, the question is now completed and the student moves on to the next question

Answer Tolerances With a standard single numeric entry question, setting tolerances for an answer is straight-forward When two or more numeric answers are expected, this becomes a complex task as each possible scenario has to be defined within the question marking algorithm Setting conditions for automated marking becomes even more difficult, especially when tolerances to answers are allowed with multiple numeric input questions

Conclusions Increasing diversity of intake standards makes teaching mathematics to engineering undergraduates a challenge and the HELM project addresses this Our CAA regime –Useful for formative and summative tests –Drives student learning through formative testing Good practice for mathematical CAA –Multiple numeric entry questions –Breaking complex mathematical questions into stages –Provision of detailed worked solution as feedback –Allowing answers within specific tolerances with specific feedback and full or partial marks