1. Feb 5 2004 using assessment to motivate learning. QAA Assessment Driven Learning Jean Cook J.cook@gcal.ac.uk www.calmat.gcal.ac.uk

2. Outline  What are we doing when we assess?  What is Assessment Driven Learning?  How do we operate?  What are the benefits to the student?  Are there any benefits to the tutor?  Does it improve pass rates?  Is it cost effective?

3. What should we be doing when we assess?  Enabling students get a measure of their achievement.  Giving feedback to teachers on effectiveness.  Providing statistics for programme boards.  Motivating students to engage in the learning.  Providing feedback so that they can improve and remedy their deficiencies.  Helping them consolidate their learning.

4. What should we be doing when we assess?  Enabling students get a measure of their achievement. often by means of a final exam. often by means of a final exam.  Giving feedback to teachers on effectiveness. usually when it is too late for that group of students usually when it is too late for that group of students  Providing statistics for programme boards. Great! Great!  Providing feedback so that they can improve and remedy their deficiencies. this won’t happen with just a final exam! this won’t happen with just a final exam!  Helping them consolidate their learning. best done as the module progresses rather than at the end. best done as the module progresses rather than at the end.

5. What is Assessment Driven Learning (ADL)?  Reduced lecture time, more tutorials.  Staged assessments, with firm deadlines for completion of assessments.  Immediate feedback to students on their progress.  Assessment as part of the learning-not the final verdict for the student.  Staff get feedback when they can use it!  - and they still get results for Programme Boards.

6. Context  This describes what has been done by the Mathematics Division of Glasgow Caledonian University, in response to the SHEFC Progression and Retention initiative.  The students are mainly first year students, studying a ‘Mathematics’ module.  This implementation relies heavily on CAL software to assist the delivery of the learning material and the assessment.

7. CALMAT Mathematics courseware l Learning materials with integrated tutorials and assessments l Separate tutorial and assessment system l Management system l Records Controller: both student and tutor use this to view progress. l Designer for creating html pages with links to CALMAT components l Player for delivery of the Designer’s html pages l Diagnostic testing facility l For more details visit www.calmat.gcal.ac.uk

8. How we operate  Two lectures and three tutorials each week.  The tutorials are help in computer labs.  Tutor groups are limited to 20 students.  Timetable is carefully structured.  Restricted number of tutors.  10 computer based assessments in the last tutorial in weeks 2-6 and 8-12.  Specified CAL lessons to complete.  Exemption given if an average of 70% is achieved.

9. Why did we switch to ADL?  We are now teaching a different student body Increase in student numbers from 6% to almost 50%. Increase in student numbers from 6% to almost 50%. Most seem to need encouraging to study. Most seem to need encouraging to study. More time in PT work than spent studying. More time in PT work than spent studying. Lectures now satisfy a handful of students and possibly the lecturer. Lectures now satisfy a handful of students and possibly the lecturer.  We are under pressure to improve pass rates and have already pared down syllabuses to essentials already. Does this sound familiar? Does this sound familiar?

10. More (mainly maths) reasons to switch to ADL?  Most students need to be encouraged to enjoy their first year maths module!  We really ought to be helping students study the whole syllabus, particularly in first year mathematics.  We were concerned how little some of our students had to know to pass a module! If you set 8 questions on 100% of syllabus and they have to answer 5(63%) If you set 8 questions on 100% of syllabus and they have to answer 5(63%) Need 40% to pass which implies a student may know only 25% of the syllabus. Need 40% to pass which implies a student may know only 25% of the syllabus. If you set 5 questions on 100% of syllabus and they have to answer 3(60%) If you set 5 questions on 100% of syllabus and they have to answer 3(60%) Need 40% to pass which implies a student may know only 24% of the syllabus.Need 40% to pass which implies a student may know only 24% of the syllabus.

11. Benefits to the student  Quality feedback and help from week 1.  The exemption provides motivation.  The software is widely available, and very supportive.  What the students say: Prefer weekly tests, keeps momentum and requirement for continuous work going. Prefer weekly tests, keeps momentum and requirement for continuous work going. Did not feel they needed the lectures: but staff not willing to give them up! Did not feel they needed the lectures: but staff not willing to give them up! Would have liked complete coverage of content with CAL lessons. Would have liked complete coverage of content with CAL lessons. General satisfaction with module. General satisfaction with module.

12. Benefits to the tutor  Weekly information on student progress.  Ability to target help.  Reduced marking.  Satisfaction when students do well.  What the staff say: If the students had worked as hard in previous years, they would have passed! If the students had worked as hard in previous years, they would have passed! It was hard work, despite the use of computers to deliver and mark tests. It was hard work, despite the use of computers to deliver and mark tests. I am sure the students have learnt more than they did in previous years. I am sure the students have learnt more than they did in previous years.

13. Progression Statistics startrevisedExemptPass %% roll diet1diet2exemptdiet1diet2 99/005047 374 7987 00/014037 294 7889 01/025957421107493 02/03383424507185 99/0010796 665 6974 00/01148125 7414 5971 01/0213311797508387 02/031431281090485 88 99/00 79 483 6165 00/01 78 5410 6982 01/0215114195213678284 02/031081009121 9394

14. Summary of results  In general, 2nd diet pass rates were achieved at the first diet.  The effect of not having to resit a mathematics module would, no doubt, contribute to improving the progression rates for the programmes as a whole.

15. Is ADL cost effective?  Savings are due to More students progressing to year 2. More students progressing to year 2. Staff time saved in marking exam papers. Staff time saved in marking exam papers.  In the first year of operation, the extra fee income and staff time saved in marking examination papers is estimated at over £60K. This assumes 2001-2 intakes and 2000-1 pass rates, and a lecture replaced by several small tutorial groups.

16. To summarise  Assessment Driven Learning makes assessment a part of the learning process.  Students are motivated by the exemption.  Second diet pass rates are achieved at the first diet.  Tutors gain satisfaction from seeing students do well.  It is cost effective.

17. “Assessment methods and requirements probably have a greater influence on how probably have a greater influence on how and what students learn than any other single factor. This influence may well be of greater importance than the impact of the teaching materials.” (Boud 1988)