Exploring Interactive Computer Marked Assessments for Logic Constructs in Level 1 Computing Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) Centre for Excellence in Teaching and Learning The Open University is incorporated by Royal Charter (RC ), an exempt charity in England & Wales and a charity registered in Scotland (SC ). Michael Isherwood Associate Teaching Fellow, COLMSCT

Presentation Summary Project Aims and Background The Quizzes Logical OR Analyses Results and Student Views

Project Aims - summary Support students in their understanding of new and difficult parts of M150 (in effect, Block 2) Confirm understanding prior to tma feedback Investigate extension to other computing courses Through better understanding, aid retention

Background Students find getting to grips with programming difficult This leads to frustration and dropping out of M150 (and therefore further computing courses)

Background - TMA02/03 Marks

On-Line Quizzes 6 Quizzes on 2008J and 2009B presentations – Structured English – Conditions, truth and trace tables – Basic JavaScript – Selection - the if statement – Repetition - the while statement – Repetition - the for statement Results following are from 2008J up to March

Example Question Quiz 4 Q2 71% correct at 1 st attempt Only 4% wrong after 3 rd attempt Note Feedback after 1 st attempt After 2 nd attempt the correct answers remain, so student concentrates on incorrect answer(s)

Logical OR CP5D A person with body temperature outside the range 35 o C to 37 o C is viewed as requiring medical attention. Within this range the temperature is considered normal. Write a compound condition that evaluates to true for requiring medical attention, using temp for temperature (in o C) (temp 37) 24% correct 1 st attempt and 43% after 3 rd attempt Some learning as earlier logical AND only 29% after 3 rd attempt

Analysis of OR On 1 st Attempt AND and && & OR or || ¦¦ unit % 70% correct in principle with 56% absolutely correct 28% incorrectly go for AND etc 12% of responses included unit (g mm nm o C) Again an improvement on earlier logical AND question where 62% and 19% are comparative results No improvement in omitting unit (10% in logical AND) and no difference between unit depending on familiarity or difficulty (temp 37)

Comparison Operator Analysis The 5 variants are : (age =60)(WL =700) (mass =63) (temp 37) (len =221) 49% chose < when correct but only 14% chose <= when correct 69% chose > when correct but only 28% chose >= when correct Some 3% chose = Interpreting boundary value operators (and values) creates an issue when having to apply from text – “with body temperature outside the range 35 o C to 37 o C”

JavaScript Logical OR Example SY6D A shop waives address evidence on credit card sales when the customer's status is that of existing customer or when the purchase amount is under £40. In the following compound statement expressing this situation drag in the symbols required. >= != AND OR & && || ( amount 40 ) ( status 'existing' ) ( amount < 40 ) || ( status == 'existing' ) Note that in JavaScript (as in other programming languages) many symbols differ from the pseudocode symbols eg || for OR and == for =

JavaScript Logical OR Conclusions This was the last question in this JavaScript quiz and all JavaScript symbols had featured in earlier questions 74% chose “OR” with 65% correct with || and 4% with ¦¦ not a great improvement on 70% earlier 25% chose && (and) thus virtually everyone used a logical operator < selected by 94% == correctly selected by 71% but 22% selected = which is the assignment operator

Are the Quizzes valuable for Learning? Student Feedback says yes Many students attempted more than once – Quiz 2: 304 attempts by 211 students – Quiz 4: 195 attempts by 171 students As indicated there is improvement as students progress through question tries and through quizzes Ultimate proof in retention and throughput to further computing courses

Student Responses – Pilot 1 96% of students found the quiz enjoyable 91% found the quiz useful The majority of students didn’t suggest improvements or make any other comment, however some useful suggestions and observations were made and one or two errors were identified

Student Views – feedback sample makes it clear where to use capitals and put spaces in compound statements Excellent..quite short, but long enough to see if I had principles sorted Very useful especially with the trace tables It seemed to work very well Question 7 does not work Make question 3 less ambiguous Should include more difficult questions – stage 1 and stage 2 Questions about repetition and selection could be less ambiguous

if construct – Q2 cont

if constructs – Q3 75% of responses correct at 1 st attempt 3% wrong after 3 rd attempt. After a 2nd attempt, the correct answers were retained, as shown, for the final attempt. The majority of the errors were at the boundary, as here, though several students reversed

if constructs – Q6 On 1st attempt the number of wrong answers is given along with a reminder of the OR (||) operator's effect. The 2nd attempt has feedback as above and the 3rd attempt starts with all the correct responses retained.

A Vision for Student Studies Section Does Quiz Next Section “Pod” + Ppt (IC) Tutor Advice Special Session