Formative Quizzes and their Contribution to the Understanding of Computer Programming by Level One Students Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC ), an exempt charity in England & Wales and a charity registered in Scotland (SC ). Michael Isherwood Associate Teaching Fellow, COLMSCT

Presentation Summary Project Background Indicators of Success? For the Future

Background Students find getting to grips with programming difficult This leads to frustration and dropping out of M150 (and therefore further computing courses)

Background - TMA02/03 Marks

A Solution - iCMAs A series of Interactive Computer Marked Assessments Optional but Accessible through M150 Website But –Also other initiatives – podcasts and video files –Ian Cooke’s Project –Course Team Initiatives on Assessing Programming

Dropout and TMA03 Skipping No of Students engaging % drop out after TMA01 % drop out after TMA02 % skipping TMA B J B All Three (7.6)10.7 (10.5)4.3 (5.3) Bracketed figures for pre iCMA presentations: 2006B – 2007B 2008J and 2009B has iCMAs accessible from M150 Website Fewer skipping TMA03 than previously (2008B aberrant) Marginally fewer dropping out after TMA02 Final figures for completions of 2008J and 2009B not available as resits pending

Logical AND Question Student needs to know form of expression and have the correct Boolean operator (AND) as well as the correct conditions (bracketed parts) OpenMark identifies each of these components, recognises errors and gives targeted feedback on these before the student has a second attempt CP2B A steel bar of nominal length 220mm needs to be less than 1mm different from this length. To be within tolerance, its length must therefore be between 219mm and 221mm. Write a compound condition that evaluates to true for being within tolerance, using len for length (in mm). Free text answer is: (len > 219) AND (len < 221)

Learning on Similar Questions A series of questions has been set using the logical operators AND and OR In Quiz 2 there were two similar questions. The logical operator was AND in the first case and OR in the second case. (The Qs were of the same format) In Quiz 3 there was a similar question requiring selection of either AND or OR There was more than described here to each of the questions /%ANDOROther First question has many still learning the requirement By third question still improvement

Units in Conditional Statements The two Quiz 2 Logical Operator questions incorporated units in the wording of the question. The units depending on variant were g mm nm and o C In the first question 10% of students incorporated the unit and in the second, it was 12% - no improvement!

Learning through Attempts Students have 3 attempts per question After each attempt, targeted feedback is given based on the student’s answer 1 st Try2 nd Try3 rd TryStill Wrong % OR question used earlier It is evident when looking at an individual student’s attempts that most learn on their 2 nd and 3 rd attempts However this is often not true for those who are wrong after the 3 rd attempt

Weak Students – 3 Wrong Attempts 37% with wrong Boolean operator – of these 75% correct on 2 nd attempt BUT 25% don’t despite pretty explicit feedback 13% have all 3 attempts identical and a further 38% have two attempts the same 10% had little or no idea with 3 “garbled” attempts and 2% left completed blank

The Student View Feedback from Students indicates that most find the quizzes both enjoyable and useful Many Students also wanted more There is more on this in Poster Presentation

For the Future Need many more questions available for students who need to develop their skills Needs routes through quizzes that are determined by a combination of that student’s performance and that student’s choice – differentiated learning Tutors need to be able to monitor progress and, if necessary, intervene Much more time needed for analyses of student performance and needs

Thank you for Listening