1. Can a computer-marked exam improve retention?
Jon Rosewell
Communication & Systems, MCT
CALRG Annual Conference, 19th June 2012

2. Does ‘one-off’  poor retention?
An ‘M-world’ module
T180 Living with the Net ( ) as an example
Some studying as ‘one-off’
Students enjoyed and completed course but daunted by EMA
Students who submitted nearly all passed – too late!
Not submitting EMA  not completing module  poor retention statistic (51%)  HEFCE penalty
Setting up students to fail?

3. Can a final CMA improve retention?
Submission of CMA generally high (T184: 87%, n=2158)
iCMA encourages engagement and motivation
Not as daunting as typical EMA
So add final CMA(CME) and reduce EMA
CME is a component of final assessment  submission = course completion
Apologies for the acronym nightmare!

4. Can an iCME improve retention?
Moodle quiz
Same format as mid-course iCMA
Open at start of course; single submission by traditional cut-off date
Questions can be revisited
Question level feedback immediately after cut-off date (cf OU anodyne performance profile 3 months later…)
Even without adaptive feedback, learner can judge their own performance
Builds confidence
Lowers barrier to pass: relatively low score on EMA programming and essay questions sufficient to pass

5. T184 assessment strategy Original 10% – mid-course iCMA
90% – final written EMA
Part I – short answer qns
Part II – prog & essay qns
Threshold:
40 marks on EMA
New
10% – mid-course iCMA
30% – final iCME
60% – final written EMA
prog & essay qns
Thresholds:
0 marks on iCME
0 marks on EMA

14. Hypothesis: Students will attempt iCME first, before EMA
 They will thus complete the module
This will improve their confidence
They will then be motivated to attempt EMA
They will thus pass the module

15. Did more submit iCME? EMA Total Not submitted Submitted CME 44 1 45
13% 19
281
300
87% 63
282
345
18%
82%
T E & 2011J presentations

16. Did students submit iCME first?
CME clearly first
214
76%
EMA clearly first 9 3%
CME & EMA together (<1hr)            58
21%
CME slightly earlier 43
15%
EMA slightly earlier 15 5%
Total
281
100%
91% submitted CME first, 9% submitted EMA first overall, but some submitted ‘together’
T E & 2011J presentation

17. T184 2011J Median 18hrs before deadline 60% in final 24hrs
11% submitted in final hour
T J

18. Median 4 days 15 hrs
T J

23. Interviews with students
I thought the balance of assessment was about the best I’d come across. Interesting computer-based tests … plus the programming elements … an interesting [EMA] with a choice of two essays. … The assessment contributed to the enjoyment of the whole course.
By the time I’d finished the assessment I’d felt I’d learnt more. … I also felt fairly confident that I’d got enough marks on the computer based test.

24. Interviews with students
[Early on] get an achievement and get a bit of success, … it was really easy that first CMA. … So an early success and then you know at the end you’ve got some short answers in the CMA that you think pick up your score a bit, and then you’ve got the EMA where you can do even better. I think it’s spot on.

25. Interviews with students
… as long as they feel it’s rigorous … it didn’t have that kind of ITV quiz thing, you know ‘what’s the capital of England’ or something. … There was a combination with ‘you’ve done this bit now, and now you need to tell us about it, now you need to describe it, now you need to do some programming and show us your work’. It felt like there was real rigour to it so it didn’t feel like ‘this is a quick and dirty kind of interactive quiz’.

26. Interviews with students
Student: I think it was excellent. … The only thing I don’t like is having to wait till September for my result!
Me: So that’s another reason for introducing that computer marked exam … it gives students feedback immediately rather than a long delay.
Student: Yes, so you know where you’ve gone wrong. That’s really good, and it cheers you up as well, not an idiot after all!

27. Interviews with students
“I enjoyed all of it really. I thought the course materials were brilliantly laid out, I thought it was very clear, concise. It gave you enough about the subject to make you interested and wanted you to do more.”
“As I say it was a pleasure doing it. I hope I’ve passed, but I just think it was a joy doing it, it was really fun, you learnt something, it really complemented what I learnt last year and it’s just made me keen to do other stuff, or try and find out more about it.”

28. The end … of T184