1. Modelling early mathematical competencies and misconceptions Erasmus Intensive Seminar Graz 2005 Gisela Dösinger

2. Objective Development of an instrument  Adaptive assessment  Broad range of early mathematical knowledge  Including misunderstandings  Against the background of internalisation  Remedial instruction

3. Motivation Analysis of existing instruments  Neglecting prenumerical knowledge  Only limited sub-domains assessed  Not varying presentation format  Only coarsely defining which competencies assessed  Not covering misunderstandings  Redundant assessment Young children and children with disabilities

4. Methodology For modelling correct knowledge  Competence - Performance Theory  Korossy (1993) For modelling misconceptions  Information System Based Approach  Scott (1982)  Applied by Lukas (1997)

5. Investigation 1 Brief introduction to competence - performance theory  Performance: observable solution behaviour  Competence: underlying knowledge  Latent level explains manifest level  Levels related to each other Competence - Performance Theory

6. Investigation 1 Performance structure (Q,P) Competence structure (E,K) Interpretation function k: Q  (K) k q : {  1,  2,…,  r } Representation function p: K  (Q) Competence - Performance Theory

7. Investigation 1 Step 1. Define solution ways Step 2. Represent solution steps by competencies L =  { f(q)|q  Q} = { {3},{1,2},{2,4},{3,4},..., {2,3,4,6}} Competence - Performance Theory

8. Investigation 1 Step 3. Construct competence space  3.1 Apply surmise function  3.2 Close basis under union Competence - Performance Theory

9. Investigation 1 Step 4. Apply interpretation function Competence - Performance Theory

10. Investigation 1 Step 5. Apply representation function Competence - Performance Theory

11. Investigation 1 Step 6. Construct performance space  6.1 Apply surmise function  6.2 Close basis under union Competence - Performance Theory

12. Investigation 1 State of research on early mathematical knowledge  Proto-quantitative schemata  Enumerative processes  Calculation: addition and subtraction  Internalisation Twenty-six competencies and three internalisation levels Derive dependencies among competencies Early mathematical knowledge

13. Investigation 1 Problem construction: making competencies accessible  Take a competency  Search available instruments for problems  Describe solution way  Represent solution steps by competencies  Supplement competencies by prerequisites  Copy problem to other internalisation levels Forty-nine problems in eighteen problem classes Problem construction

14. Investigation 1 A B FC G R={(A,B),(A,C),(A,F),(A,G),(B,C),(B,F),(B,G),(F,G),(C,G)} P={{A},{A,B},{A,B,F},{A,B,C},{A,B,C,F},{A,B,C,F,G}} Deriving problem order

15. Investigation 1 Hypotheses  For each pair element of the relation it is expected that more difficult problem… …is solved less frequent than or as frequent as less difficult problem …is not solved if the less difficult problem is not solved  The performance states are expected to fit the empirical solution patterns Hypotheses

16. Investigation 1 Method  Ninety-four kindergarteners  Mean 62.64, standard deviation 9.89  Problems partitioned into subsets 13 problems each  Overlapping substructures  Subjects tested individually Method

17. Investigation 1 Solution frequency  Solution frequencies in accordance with hypothesis 1 A.. 74% B.. 79% F.. 42%C.. 89% G.. 37% Results

18. Investigation 1 Gamma-Index  Derived from  -Index (Goodman & Kruskal, 1954)  Measure of association indicating whether two classifications are ordered likely or unlikely Results

19. Investigation 1 Gamma-Index  Gamma-Indices varying from 0.64 to 0.96, significantly differing from 0, thus supporting hypothesis 2 Results

20. Investigation 1 Symmetric distance  Distance distribution, average symmetric distance, and standard deviation are calculated  Distance distributions compared to 'random' ones  ‘Random‘ distributions obtained by using power set as data set Results

21. Investigation 1 Symmetric distance  Symmetric distances ranging from 0.13 to 0.94, with distance distributions significantly differing from 'random‘ ones,supporting hypothesis 3 Results

22. Investigation 1 Distance Agreement Coefficient  Proposed by Schrepp (1993)  For comparing fit of different knowledge structures  The smaller, the better the fit Results

23. Investigation 1 Distance Agreement Coefficient  Adapted for testing relative fit of one knowledge structure  Compare DA to its maximal possible value which is got when d dat is set to its maximal possible value  Distance Agreement Coefficient ranging from 0.05 to 0.32, much smaller than DA max which was about 2, thus supporting hypothesis 3 Results

24. Investigation 1 Reproducibility Coefficient  Proposed by Guttman (1944)  Another measure explaining extent of concordance between data and hypothesised structure  Proportion of cells explained by model  Example: a  b  c  d Results

25. Investigation 1 Reproducibility Coefficient  Reproducibility coefficient ranging from 0.93 to 0.99, thus supporting hypothesis 3 Results

26. Investigation 1 Substructures proved valid  Adaptive assessment Problems on different internalisation levels and on proto- quantitative/quantitative level  not varying in difficulty Use of abstract materials: difficulty to transfer knowledge from concrete, everyday materials Reason for large number of competencies: broad range intended to be covered and fine grained dissolution required Discussion

27. Investigation 2 Brief introduction to information system based approach  Manifest level: correct solutions and bugs  Latent level: competencies and misconceptions  Two main concepts on the latent level Implication Incompatibility Information system based approach

28. Investigation 2 Implication and incompatibility impose an algebraic structure Information system A is a structure ‹ D, Con, › where D is set of data objects X Con is a set of finite consistent subsets of D is a binary relation  Con  D Information system based approach

29. Investigation 2 On manifest level there is a polytomous response format: correct responses, bugs, and slips For every q  Q there is a set of possible responses R q = { q 0, q 1, q 2,…, q n } If R =  R q \ q 0 a response pattern T is a subset of R Information system based approach

30. Investigation 2 Relating latent and manifest level G q : A  R q G : A   ( R ) G ( x ) =  G q ( x )| q 0 Information system based approach

31. Investigation 2 Identify set of knowledge entities D and their structure  Invariance principle a  Additive principle b  Spatial distortion c  b a  c incompatible with a and b Information system based approach

32. Investigation 2 Derive information system A  Build power set  Cancel inconsistent subsets  Cancel subsets incompatible with  Con = {{},{ a },{ c },{ a, b }} Information system based approach

33. Investigation 2 Construct problems and determine responses  Problem A Spatial, number conserving change: row-to-circle A 1 …same number, A 2 …more, A 3 …less  Problem B Spatial, number conserving change: spread-row B 1 …same number, B 2 …more  Problem C Splitting set into subsets: partition-set C 1 …same number, C 2 …more Information system based approach

34. Investigation 2 Relate responses to elements x of information system Determine response patterns Information system based approach

35. Investigation 2 Measure for validating structure: discrepancy between empirical and hypothetical response patterns  Number of problems in which patterns disagree -  2  Comparison to 'random' case: randomly generated response patterns  U-Test for testing statistical significance Measure for validation

36. Investigation 2 Method  Sixty-four kindergarteners  Mean 61.84, standard deviation 9.40  Problems partitioned into subsets 12 problems each  Subjects tested individually Method

37. Investigation 2 Results  Proportions of correct and buggy solutions  Discrepancy Discouraging output  Re-modelling Results

38. Investigation 2 No valid model found, but…  Set of misconceptions identified  Problems able to provoke their application designed  Empirical evidence proven No age effect Application of misconceptions seems to depend from kind of problem Bugs arising from perceptual distraction play important role Discussion

39. Investigation 2 Misconceptions are not stable  better use a probabilistic approach Information system based approach: implications need to be neglected, because only excluding responses can be contained in response pattern Discussion

40. Thank you for your attention!