Visualising the Code An investigation of student engagement with programming in TU100 Helen Jefferis, Soraya Kouadri & Elaine Thomas Computing & Communication Department

Aim of the project To investigate the impact of using a graphical programming environment on student engagement with programming. Background - There is a strong need for the teaching of introductory programming at level 1 in the Computing and IT degree programme. The majority of new OU students will not have experienced the new National Curriculum. Previous teaching of programming at level 1 (M150) involved a text-based programming, JavaScript. Over half of students  avoided answering the question on programming in the EMA.   TU100 ‘My digital life’ uses a graphical programming environment Sense based on Scratch. What is visual programming? Why is it important? Literature says…. The project will seek to address the fundamental question as to whether the visual programming environment actually engages novice programmers or not in ‘TU100’.

Methodology Identification of the Sense programming questions in each TMA and in the EMA. Identification  and collection of data related to the numbers or students who completed these questions and their overall performance. Analysis of textual comments in a selection of SEaM surveys  of TU100 relating to students’ experience of programming. Three stages to the project.

Comparison of OES Scores Red line shows number of students (right hand axis) – less on B presentations Green is the OES Mean, Yellow the Sense Mean, Blue the ‘rest’ mean. In 13J and 14B there were less marks allocated for Sense but the Average scores for the programming and non-programming elements were about the same. (The marks have been normalised so that these figures are showing the percentage of the available marks for each element) In more recent presentations, students seem to have gained more marks, on average, in the Sense / programming elements of the EMA. NB the J presentation and the following B presentation have very similar questions – which tends to suggest that perhaps Sense was ‘easier’ on 14J / 15B?

Are students passing without passing Sense? <7.5% (460 out of 6,159 students) failed Sense and passed OES This chart shows the students that scored less than 40% The red column is the students that also failed OES Out of 6,159 students in total only 460 (181+161+88+30) passed OES – this is less that 7.5% of students. So the simple answer is no! As can be seen by the line indicating the overall % of students

Small percentage passed module without passing Sense 7.5% Putting it more clearly – a tiny proportion passed the module without passing Sense 92.5%

Analysis of SEaM comments 325 students made comments on one or more of the 3 questions: What aspects of teaching materials, learning activities or assessment did you find particularly helpful to your learning? We would welcome any further suggestions or comments to consider for future editions of the module. Do you have any other comments to add about your study experience on this module? NB Explain what SEaM is So these were general comments not just about programming

Sense & Programming Comments 79 students made some comment about Sense and in addition a further 18 students made a comment about programming. Of these 73% were positive comments (60 students commenting about Sense; 11 about programming) 27% were negative (19 students commenting about Sense; 7 about programming.)

Positive comments examples SENSE was very good too as it enabled me to focus on logic and programming structure, rather than language The Sense board, tasks and associated manual were excellent and this was certainly the high point of the course for me Furthermore, it introduced me to programming in a way that was very easy to understand and to follow with lots of activities and programming examples and which led me to a new hobby. “I particularly liked the Sense Programming learning activities. This is something that I had never done before and found it very engaging and easy to understand. I found my competence levels rising throughout the course, especially for Sense.” “One aspect that was really good was the Sense programming guide and the activities for it through the blocks. It provided a brilliant base for learning programming and advancing skills. It was easy to follow and enjoyable to do. It also stuck in your head for other languages because of the idea of blocks being used for sections of programs”

Negative comments examples I felt the assignment programming tasks were very basic The Programming guide is awful, I have worked thought it more than once and I still cant grasp most of it. I have had to look elsewhere to help develop these skills The Sense programming environment is obviously designed for children, and I cant understand why the OU couldn’t have based the course on a real-world app programming language that is instantly applicable in the real-world... Android for example or C++ two people also commented (and counted as negative comments in this case) that they didn’t want to do it all year eg “.. A 15-point course on Sense would have been nice, followed by another 15- or 30-point course on some proper object-oriented programming.” “Python is very simple and introducing some Python after making students comfortable with Sense would definitely work.”

Summary & future work Summary Future Work More students have engaged with the programming element than on previous modules. There is a strong correlation between the scores that students achieved in the programming and non-programming elements of the EMA. There is little or no difference in the performance of students in the programming elements and the non-programming elements.  Work has informed thinking around new level 1 module Provides a reference point for future studies of level 1 programming Future Work Similar study for TM111 after first presentation Evaluate student performance in text-based programming on TM112