Beyond Computational Thinking A Modern problem solving approach and its application 29. November, 2017

Background New world – ICT/STEM HYBRID JOBS Science, business, designer, … Not only computer science Programming ‘Complex problem solver’ SKILLS ICT jobs grow faster than other JOBS ICT jobs: demand > supply USA – EU – OECD

Background New world – ICT/STEM Computational Thinking

Computational Thinking features Problem Solving Approach Decompose the problem Abstraction: neglecting information Abstraction: pattern recognition Algorithmic design

Computational Thinking The answer? How is Computational Thinking applied in solving a programming problem? What is the relationship between Computational Thinking ability and the ability to solve programming problems?

Method DESIGN measurements Participants

Method Coding independent CT measurement – The Bebras task (20) Phase 1 – CT measurement Coding independent CT measurement – The Bebras task (20) International informatics contest since 2004 Logical problems – without coding elements 8 easy (2p) + 7 medium hard (3p) + 5 hard (4p) = 20 items (57p) Covariates (gender, age, IQ, …)

Method Phase 2 – Design & Scratch In pairs (Bebras score): “Program a story or a game where a hero has to overcome a challenge in order to defeat the villain(s)” in Scratch. While participants solving programming task: Videotaped Voice recorded Screen captured

Method CT feature Example behavior Decompose the problem Abstraction: Phase 2 – Computational thinking behaviour scheme CT feature Example behavior (.61 < κ(5 vid; 2 rater) < .69) Next step to do? Putting problem into smaller problem Disc. If-then relation of story/ game Focusing on x, actively neglecting y Simplifying anything (problem, codes, tasks, …) Rephrasing meaning of anything (codes, functions, …) Identifying similar structure (problems, codes, …) Aha-moments (must be related to a prior event) Any usage of copy-paste; copy-paste behavior Putting code chunks together Testing and judging code script (i.e. clicking on “run”) Debugging; adjust code script Decompose the problem Abstraction: neglecting information Abstraction: pattern Recognition Algorithmic design

Method Phase 2 – Computational thinking behaviour scheme - interact

Method Assessment of programming skills Phase 2 – assessment programming skills Assessment of programming skills Richness: “What and how much is happening their code?” Variety: “How many different code elements are they using?” Organisation: “How messy/ clean does their work space look?” Functionality: “How well is their code working?” Efficiency: “How well developed is their control flow? Many repetitions?” = Weighted mean Reliability: .93 < ICC(CI95%) < .98

Method participants Participants Programming pairs Expected: N ≈ 50; Expected: pairs ≈ 25; Age: 𝑥 =24.29 𝑆𝐷=5.78 Gender ratio: 62 % female; 36 % male IQ (paired): 𝑥 =114.74 𝑆𝐷=12.98 Bebras score (paired): 𝑥 =59% 𝑆𝐷=17% actual: N = 127 actual: pairs = 27

Results Explorative results Out of 40 min, participants spent... Mean How is computational thinking applied in solving a programming problem? Rather explorative question  explorative data analysis Out of 40 min, participants spent... Mean Min – Max Decompose the problem 03 min, 06 sec 00 min, 24 sec – 09 min, 03 sec Abst.: neglecting information - Abst.: pattern recognition 00 min, 34 sec pairs = 17 00 min, 04 sec – 01 min, 30 sec Algorithmic design 14 min, 59 sec 04 min, 09 sec – 24 min, 25 sec = CT behavioural (total) 18 min, 28 sec. 06 min, 18 sec – 28 min, 10 sec

Results Hypothesis Testing CT (w/o coding elements) What is the relationship between computational thinking ability and the ability to solve programming problems? H1: There is a positive correlation between Bebras score and programming ability. CT (w/o coding elements) Programming ability r Bebras score Richness .39 Variety .26 Organisation .05 Functionality .29 Efficiency .24 = weighted mean .30 Bold: p < .05

Results Hypothesis Testing Programming ability CT What is the relationship between computational thinking ability and the ability to solve programming problems? H2: There is a positive correlation between the frequency of computational thinking behavioural components and programming ability. Programming ability CT (with coding elements) r Weighted mean Decompose the problem .24 Abst.: neglecting information - Abst.: pattern recognition .12 Algorithmic design .63 CT behavioural (total) .62 Bold: p < .05

Results Hypothesis Testing CT (w/o coding elements) What else? Control analysis: H3: There is a positive correlation between the Bebras scores and the frequency of computational thinking behavioural components. CT (w/o coding elements) (with coding elements) r Bebras score Decompose the problem .29 Abst.: neglecting information - Abst.: pattern recognition .15 Algorithmic design .34 = CT behavioural (total) .39 Bold: p < .05

Results Hypothesis Testing Outcome = programming ability weighted mean What else? Regression analysis: H4: CT without coding elements and CT with coding elements, can predict programming ability (while controlled for IQ). Outcome = programming ability weighted mean std. β IQ 0.36 Bebras score -0.41 CT behavioural 0.74 Model statistics F(3,20) 6.602 R2 (adj.) .42 Bold: p < .05

Results In summary CT instruments: satisfactorily reliable CT without programming aspects: Bebras Already well developed CT with coding programming aspects: CT behaviour scheme Reliability estimations “high enough” Programming assessment: satisfactorily reliable Participants spent a fair amount of time with CT in general …but not all features are equally important In general… Medium positive relationship between CT (w/o coding) & programming skills Large positive relationship between CT (w coding) & programming skills “The best predictor for programming skills are hands on computational thinking elements (if controlled for intelligence and theoretical CT).” However, it really depends!

Obstacles Suggestions? CT (Bebras) ≠ CT (behavioural); what does this mean? What to do with abstraction? Pairing not perfect and