1. Title: Validating a theoretical framework for describing computer programming processes
29 November 2017

2. Research problem
How do we describe the process by which people write computer programs? 
Important in order to understand how teachers might better support the development of computer programming capabilities?
Activity Theory is extensively used framework for analysing how activities are undertaken within social constructivist contexts.
Subject pairs of students working together, Object is the computer program, the tools (instruments) that can be used by subjects to develop the program are computers, integrated development environments, etc.
To better understand teachers might change things to improve the decision process, and give us the way to impact what teachers will do to affect what students will do
Limited studies on how people write computer programs/investigate the computer programming process.
Figure 1. The elements of an activity system and their interrelationships (Engeström, 1987)

3. Computational thinking & notional machine
Computational Thinking: Solving problems, designing systems, and understanding human behaviour, by drawing on the concepts fundamental to computer science (Wing, 2006).
Notional Machine: The notional machine is an abstract version of the computer, “an idealised, conceptual computer whose properties are implied by the constructs in the programming language employed” (du Boulay, et al., 1989).
A model of the computer as it relates to executing programs
These concepts can be used to theoretically ground how people go about performing computer programming process.
Notional machine
OFFICE | FACULTY | DEPARTMENT

4. Methodology
Data was collected from 10 pairs of students completing a scratch programming activity.
Scratch is a visual programming language
Subjects
5 pairs of pre-service teachers with little experience
5 pairs of third year computing students
Program a story game where a hero has to overcome a challenge in order to defeat the villain(s).
Each pair spent approximately 40 minutes to undertake the task.
Both finished the IQ tests; Did the Computational thinking tests (similar CT scores) ; they did not have any prior programming knowledge; decent scratch products

5. Research model
Thematic analysis using an informed grounded theory approach used, starting with an initial framework.
Computational thinking
17 categories and we had the definitions of the categories and we made changes via the entire process
Notional machine

6. Data analysis – Thematic analysis Dual coding by two researchers
Nvivo 11 to conduct the thematic analysis
Two raters coded the first two transcripts independently to identify areas of coding discrepancy. After each of the two videos was coded the pair met to analyse the reasons for the differences and agree upon a category. The inter-rater reliability improved from the first to the second video, leaving the team with a sense of conceptual agreement about the meaning and boundaries of the categories. Following this, the final X videos were coded by one team member, who flagged potential utterances where coding could potentially have been ambiguous. The second coder was then consulted on each of these, and the pair formed a consensus about the most appropriate category.

7. Overall Results (1)- Refinement of coding scheme – Final framework
Three categories were merged into others because they were not observed in majority pairs (Reconstruction Requirements checking; Decoding  Evaluation; Technology  Help seeking)
Percentages of each category observed shown below.
Computational thinking
Task specification (0.2%)
Requirements checking/Reconstruction (0.3%)
Deconstruction (1.4%)
Design (11.1%)
Top 5 categories highlighted in red
Evaluating/Decoding (3.5%)
Encoding
(15.3%)
Help seeking/Technology (0.8%)
Organising (1.0%)
Off topic (1.2%)
Unclassifiable (3.3%)
Reflections on learning (1.4%)
Notional machine
Implementation (47.0%)
Testing (9.2%)
Debugging (4.5%)

8. Over all results (2) – Some significant differences between computing and education students
Chi-square test: = , df = 13, p-value < 2.2e-16
The categorisations are dependent on different disciplines (e.g. education students or computing students)
Significant difference between the expected and observed frequencies

9. Overall results (3)
Computing students made relatively greater focus on deconstructing problem.
Computing students made relatively greater focus on evaluating program.
Education students made relatively greater focus on help seeking.
Computing students made relatively greater focus on implementation.
Education students made relatively greater focus on reflections on learning.
Organising was not significant (p=0.0067) but very low value may indicate that with larger sample education pairs tended to be more collaborative.
1. Think more about the relationship between tasks and design. 2. Computing students are more skilful to evaluate the program. 3. Education students are less knowledgeable on how to use Scratch hence they have to seek help, like watching the tutorials or asking others. 4. Education students did more self-reflections exercises in their discipline.
OFFICE | FACULTY | DEPARTMENT

10. Summary and future work
Project has empirically validated a theoretical framework for describing computer programming processes and shows, using a single contrast, how it can be used to perform educational analysis.
Further research could investigate how programming process differs for:
Different tasks (e.g. more complex specifications)
Different languages (e.g. C++, Python, Blockly)
Different programming environment (e.g. visual vs. text interfaces)
Different Cohorts (e.g. children, experts, gender)
Different teacher interventions (e.g. forms of scaffolding and modelling)

11. Thank you! Q&A