1. Beyond Computational Thinking
A Modern problem solving approach and its application
29. November, 2017

3. Background New world – ICT/STEM HYBRID JOBS
Science, business, designer, …
Not only computer science
Programming
‘Complex problem solver’
SKILLS
ICT jobs grow faster than other
JOBS
ICT jobs: demand > supply
USA – EU – OECD

4. Background
New world – ICT/STEM
Computational Thinking

5. Computational Thinking
features
Problem Solving Approach
Decompose the problem
Abstraction: neglecting information
Abstraction: pattern recognition
Algorithmic design

6. Computational Thinking
The answer?
How is Computational Thinking applied in solving a programming problem? What is the relationship between Computational Thinking ability and the ability to solve programming problems?

7. Method
DESIGN
measurements
Participants

8. Method Coding independent CT measurement – The Bebras task (20)
Phase 1 –
CT measurement
Coding independent CT measurement – The Bebras task (20)
International informatics contest since 2004
Logical problems – without coding elements
8 easy (2p) + 7 medium hard (3p) + 5 hard (4p) = 20 items (57p)
Covariates (gender, age, IQ, …)

9. Method
Phase 2 –
Design & Scratch
In pairs (Bebras score): “Program a story or a game where a hero has to overcome a challenge in order to defeat the villain(s)” in Scratch.
While participants solving programming task:
Videotaped
Voice recorded
Screen captured

10. Method CT feature Example behavior Decompose the problem Abstraction:
Phase 2 –
Computational thinking behaviour scheme
CT feature
Example behavior
(.61 < κ(5 vid; 2 rater) < .69)
Next step to do?
Putting problem into smaller problem
Disc. If-then relation of story/ game
Focusing on x, actively neglecting y
Simplifying anything (problem, codes, tasks, …)
Rephrasing meaning of anything (codes, functions, …)
Identifying similar structure (problems, codes, …)
Aha-moments (must be related to a prior event)
Any usage of copy-paste; copy-paste behavior
Putting code chunks together
Testing and judging code script (i.e. clicking on “run”)
Debugging; adjust code script
Decompose the problem
Abstraction:
neglecting information
Abstraction:
pattern Recognition
Algorithmic design

11. Method
Phase 2 –
Computational thinking behaviour scheme - interact

12. Method Assessment of programming skills
Phase 2 –
assessment programming skills
Assessment of programming skills
Richness: “What and how much is happening their code?”
Variety: “How many different code elements are they using?”
Organisation: “How messy/ clean does their work space look?”
Functionality: “How well is their code working?”
Efficiency: “How well developed is their control flow? Many repetitions?”
= Weighted mean
Reliability: .93 < ICC(CI95%) < .98

13. Method participants Participants Programming pairs Expected: N ≈ 50;
Expected: pairs ≈ 25;
Age: 𝑥 = 𝑆𝐷=5.78
Gender ratio: 62 % female; 36 % male
IQ (paired): 𝑥 = 𝑆𝐷=12.98
Bebras score (paired): 𝑥 =59% 𝑆𝐷=17%
actual: N = 127
actual: pairs = 27

14. Results Explorative results Out of 40 min, participants spent... Mean
How is computational thinking applied in solving a programming problem?
Rather explorative question  explorative data analysis
Out of 40 min,
participants spent...
Mean
Min – Max
Decompose the problem
03 min, 06 sec
00 min, 24 sec –
09 min, 03 sec
Abst.: neglecting information -
Abst.: pattern recognition
00 min, 34 sec
pairs = 17
00 min, 04 sec –
01 min, 30 sec
Algorithmic design
14 min, 59 sec
04 min, 09 sec –
24 min, 25 sec
= CT behavioural (total)
18 min, 28 sec.
06 min, 18 sec –
28 min, 10 sec

15. Results Hypothesis Testing CT (w/o coding elements)
What is the relationship between computational thinking ability and the ability to solve programming problems?
H1: There is a positive correlation between Bebras score and programming ability. CT
(w/o coding elements)
Programming ability r
Bebras score
Richness
.39
Variety
.26
Organisation
.05
Functionality
.29
Efficiency
.24
= weighted mean
.30
Bold: p < .05

16. Results Hypothesis Testing Programming ability CT
What is the relationship between computational thinking ability and the ability to solve programming problems?
H2: There is a positive correlation between the frequency of computational thinking behavioural components and programming ability.
Programming ability CT
(with coding elements) r
Weighted mean
Decompose the problem
.24
Abst.: neglecting information -
Abst.: pattern recognition
.12
Algorithmic design
.63
CT behavioural (total)
.62
Bold: p < .05

17. Results Hypothesis Testing CT (w/o coding elements)
What else?
Control analysis:
H3: There is a positive correlation between the Bebras scores and the frequency of computational thinking behavioural components. CT
(w/o coding elements)
(with coding elements) r
Bebras score
Decompose the problem
.29
Abst.: neglecting information -
Abst.: pattern recognition
.15
Algorithmic design
.34
= CT behavioural (total)
.39
Bold: p < .05

18. Results Hypothesis Testing Outcome = programming ability weighted mean
What else?
Regression analysis:
H4: CT without coding elements and CT with coding elements, can predict programming ability (while controlled for IQ).
Outcome =
programming ability
weighted mean
std. β IQ
0.36
Bebras score
-0.41
CT behavioural
0.74
Model statistics
F(3,20)
6.602
R2 (adj.)
.42
Bold: p < .05

19. Results In summary CT instruments: satisfactorily reliable
CT without programming aspects: Bebras
Already well developed
CT with coding programming aspects: CT behaviour scheme
Reliability estimations “high enough”
Programming assessment: satisfactorily reliable
Participants spent a fair amount of time with CT in general
…but not all features are equally important
In general…
Medium positive relationship between CT (w/o coding) & programming skills
Large positive relationship between CT (w coding) & programming skills
“The best predictor for programming skills are hands on computational thinking elements (if controlled for intelligence and theoretical CT).”
However, it really depends!

20. Obstacles Suggestions?
CT (Bebras) ≠ CT (behavioural); what does this mean?
What to do with abstraction?
Pairing not perfect and