1. Programming Aptitude Testing Materials, Data analysis
Saeed Dehnadi
Middlesex University

2. I talk about: Test materials Results Collaborators Summary Questions
Mental Models
Answer Sheet
Mark Sheet
Interpretation Algorithms
Results
Middlesex University (initial test)
University of Newcastle (Australia)
York
Strathclyde
Middlesex University (second test)
Collaborators
Summary

3. Questions 1 (single assignment)

4. (M1) Mental model “move value”
a = b;
The value of b is given to a and b changes its value to zero.
a  b // b  0
Ans ( a = 20 , b = 0 ) 8th Answer

5. (M2) Mental model “copy value”
a = b;
The value of b is given to a and b keeps its original value.
a  b // b unchanged
Correct conception of LHS  RHS
Ans ( a = 20 , b = 20 ) 4th Answer

6. (M3) Mental model “move value”
a = b;
The value of a is given to b and a changes its value to zero.
b  a // a  0
Ans ( a = 0 , b = 10 ) 3rd Answer

7. (M4) Mental model “copy value”
a = b;
The value of a is given to b and a keeps its original value.
b  a // a unchanged
Ans ( a = 10 , b = 10 ) 1st Answer

8. Mental models (M5) (copy + add) value
a = b;
The sum of a and b is given to a, and b keeps its original value.
a  (a + b) // b unchanged
Ans ( a = 30 , b = 20 ) 2nd Answer

9. Mental models (M6) (move + add) value
a = b;
The sum of a and b is given to a, and b changes its value to zero.
a  (a + b) // b  0
Ans ( a = 30 , b = 0 ) 10th Answer

10. Mental models (M7) (copy + add) value
a = b;
The sum of a and b is given to b, and a keeps its original value.
b  (a + b) // a unchanged
Ans ( a = 10 , b = 30 ) 9th Answer

11. Mental models (M8) (move + add) value
a = b;
The sum of a and b is given to b, and a changes its value to zero.
b  (a + b) // a  0
Ans ( a = 0 , b = 30 ) 5th Answer

12. Mental models (M9) “no changes”
a = b;
a and b keep their original values.
a unchanged // b unchanged
Ans ( a = 10 , b = 20 ) 6th Answer

13. Mental models (M10) Equality
a = b;
Assignment is a simple equation, and then all equal values of a and b are acceptable.
Ans ( a = 10 , b = 10 ) and
Ans ( a = 20 , b = 20 )
both 1st and 4th answers

14. Mental models (M11) “Swap values”
a = b;
a and b swap their values simultaneously.
a  b a gets b’s value
b  a b gets a’s value
Ans ( a = 20 , b = 10 ) 7th Answer

15. Models for a single assignment int a = 10; int b = 20; a = b;

16. Answer Sheet (Question1)

17. Questions 2 (single assignment)

18. Questions 3 (single assignment)

19. Question 4 (two assignments)

20. Mental model “M1 + Sequence”
a = b;
b = a;
Model is M1 that applies sequentially through both statements:
L1) a  b and b  0 then
a = 20 and b = 0
L2) b  a and a  0 then
b = 20 and a = 0
Single answer:
a = 0 , b = 20

21. Mental model “M1 + Independent (M1 I)”
a = b;
b = a;
Model is M1 that applies independently for each individual line.
L1) a  b and b  0
L2) b  a and a  0
Multiple answers:
a = 20 and b = 0
a = 0 and b = 10

22. Mental model “M1 + Simultaneous-single (M1 Ss)”
a = b;
b = a;
M1 Ss Same as (M1 I), subjects only interested on Left-hand-side values of statements and ignores the right-hand-side values.
L1) a  b and b  ignores
L2) b  a and a  ignores
Single answer:
a = 20 , b = 10

23. Answer Sheet (question4)

24. Question 5 (two assignments)

25. Question 6 (two assignments)

26. Answer Sheet (question6)

27. Question 7 (three assignments)

28. Answer Sheet (question7)

29. Mark Sheet

30. Mark Sheet Interpretation (Tree Structure)

31. Mark Sheet Interpretation (Mode Structure)

32. Data sample

33. Data provided by: Saeed Dehnadi Middlesex University
Barnet College
First Experiment
2005
Data provided by: Saeed Dehnadi Middlesex University

34. Experiment Subjects: Time: Questions
30 students from Middlesex University Introductory to programming (Java)
31 students from Barnet College Further Education Programming Course
Time:
Week 0 of the course
Questions
12 questions

35. Categories Week 0
Consistent (45%)
Inconsistent (39%)
Blank (16%)

36. Categories Week 3
Consistent (67%)
Inconsistent (33%)
Blank (0%)

37. Stable group membership (Week3)
Data provided by: Saeed Dehnadi Middlesex University

38. Pass/Fail statistics (Initial Result)
Data provided by: Saeed Dehnadi Middlesex University

39. Two populations (Initial Result)

40. University of Newcastle
Australian Data
University of Newcastle
2006
Data provided by: Simon School of DCIT (Design, Communication, and Information Technology)

41. Pass/Fail statistics (First glance)
Data provided by: Simon The University of Newcastle

42. Pass/Fail statistics Exclude m2
Data provided by: Simon The University of Newcastle

43. Pass/Fail statistics Exclude m2
Data provided by: Simon The University of Newcastle

44. Data provided by: Linxiao Mu
Strathclyde Data
The University of Strathclyde
2006
Data provided by: Linxiao Mu

45. Data provided by: Linxiao Mu The University of Strathclyde
Mental models in week 20
Value assignment
Reference assignment
Data provided by: Linxiao Mu The University of Strathclyde

46. Value assignment mental models in final exam
P < = 0.05
Data provided by: Linxiao Mu The University of Strathclyde

47. Value assignment mental models in 4 in-course tests
P < = 0.01
Data provided by: Linxiao Mu The University of Strathclyde

48. Reference assignment mental models in final exam
P < = 0.025
Data provided by: Linxiao Mu The University of Strathclyde

49. Reference assignment mental models in 4 in-course tests
P < = 0.05
Data provided by: Linxiao Mu The University of Strathclyde

50. Data provided by: Dimitar Kazakov
York
The University of York
2006
Data provided by: Dimitar Kazakov

51. Data provided by: Dimitar Kazakov The University of York
Pass/Fail statistics
P < = 0.001
Data provided by: Dimitar Kazakov The University of York

52. Data provided by: Saeed Dehnadi Middlesex University
Second Experiment
2006
Data provided by: Saeed Dehnadi Middlesex University

53. Data provided by: Saeed Dehnadi Middlesex University
Pass/Fail (Quiz 1)
P < = 0.20
Data provided by: Saeed Dehnadi Middlesex University

54. Data provided by: Saeed Dehnadi Middlesex University
Pass/Fail (Quiz 2)
P < = 0.01
Data provided by: Saeed Dehnadi Middlesex University

55. Collaborators Ireland - School of Computing in DCU
Australia - Charles Stuart University
Mozambique - Instituto Superior de Transportes e Comunicações, Maputo
UK- Bournemouth University (Royal School of Signals in Blandford)
UK – University of Birmingham
UK – University of Sheffield
UK – King’s College, London
Canada - University of Toronto
Denmark – University of Aarhus
USA – A-level in Computing, High school
UK - A-level in Computing, North West College
Germany – Computing study, teacher training Institution

56. Summary
Originality We are the first to have an aptitude test that works.
Objectivity We have objective test materials that can be used by others.
Speculation Students who can handle meaningless rules learn programming more easily.

57. To be continue …..