1. Lifelong Competence Development: On the Advantages of Formal Competence-Performance Modeling Michael D. Kickmeier-Rust, Dietrich Albert, & Christina Steiner Cognitive Science Section, Department of Psychology University of Graz, Austria Learning Networks for Lifelong Competence Development March 30 – 31, 2006, Sofia, Bulgaria

2. Lifelong Competence Development - Lifelong competence development is undoubtedly an important and ambitious aim for the information and knowledge society - This presentation intends to introduce and motivate Knowledge Space Theory and the Competence-Performance Approach as tools to facilitate the development and assessment of competencies over time

3. Introduction: Competence vs. Competency - Competence - vague - broad and intangible - complex vs. simple - hard to be modeled on a formal basis - Competency - small and unique - still intangible - easy to be modeled on a formal basis

4. Introduction: Competence vs. Performance - Often the terms competence/y and performance are mixed-up - Often it is assumed that competencies could directly be observed or assessed - Inflation of competencies because different assessment methods measure different competencies or sets of competencies - e.g., maintain an aircraft, write a scientific article, pass a certain school exam - Incomparability of different assessment methods - e.g., school exam vs. assessment on the job

5. Introduction: Competence vs. Performance - The American Heritage Dictionary of the English Language states: “Competence means the state or quality of being adequately or well qualified; a specific range of skill, knowledge or ability” - This and many other definitions have in common that they describe competence as - abstract - latent - not directly observable

6. Introduction: Competence vs. Performance - Chomsky (1965) distinguished latent competence and observable performance in linguistic theory - Today, this distinction between competence and performance has a much wider application e.g., in the field of knowledge and learning psychology - competence is an unobservable quality or ability - performance is the observable behavior in specific situations (e.g., an exam), which is determined by one specific competency or by a set of competencies

7. Why distinguish competence and performance? - Example 1: Exam in trigonometry Students might be allowed to use a mathematical formulary and a pocket calculator (1) If two students master a certain task of the exam, can we conclude that these students do have the same competencies with regard to the task? - Student 1 might have the necessary competencies to master the task without using the formulary - Student 2 maybe mastered the task only by chance, incidentally choosing the right formula from the formulary

8. Why distinguish competence and performance? - Example 2: Exam in trigonometry Students might be allowed to use a mathematical formulary and a pocket calculator (2) If three students fail in a certain task, can we conclude that these students lack the same competencies? - Student 1 might lack the competence to fully understand the task and its formulation - Student 2 might fully understand the task and also might be able to choose the right formula, but maybe this student is not able to use a required function of the calculator - Student 3 might have the necessary competencies to master the entire task but might have problems to concentrate on the tasks during an exam

9. Lifelong learning? - In terms of tracking and assessing lifelong competence development - we should make sure to measure competencies independent from assessment methods - refer to probably standardized competencies - refer to defined developmental / learning paths

10. Knowledge Space Theory - A well-elaborated theory that may help to achieve these goals is Knowledge Space Theory by Doignon & Falmagne (1985, 1999) and its extensions - KST provides a set-theoretic framework to organize and model the knowledge / competencies in a given domain of knowledge by utilizing Surmise Relations, which establish Knowledge Spaces - KST in its initial form is only a behavoristic approach focusing on problems (e.g., test tasks), which can be mastered or not - From mastering a certain problem, KST allows to assume the mastering of other problems and from failing in a certain problem, KST allows to assume a failing in other problems

11. Knowledge Space Theory - Example: Five problems from the domain “basic algebra”: - a: addition - b: subtraction - c: multiplication - d: division - e: simple linear equations  Q = {a, b, c, d, e}

12. Knowledge Space Theory - Example: Prerequisite Relation for the domain Q = {a, b, c, d, e}

13. Knowledge Space Theory - Example: We can establish a Knowledge Space, which does not contain all of the 2 5 possible Knowledge States but only 8

14. Knowledge Space Theory - Advantages: - Reduction of the number of possible Knowledge States and definition of meaningful learning paths - Mathematical properties: - reflexive - transitive - anti-symmetric

15. Competence-Performance Approach - KST is purely behavioristic focusing on observable performance - CPA (Korrosy, 1997, 1999) is an extension of KST, which distinguishes latent competencies and observable performance - We have a set E of abstract competencies that are relevant for a domain - The Competence State is the collection of a person’s competencies - As in KST, Prerequisite Relations are described on the set of competencies establishing a competence structure C, which contains all possible competence states

16. Competence-Performance Approach - Example: Four competencies from the domain “basic algebra”: - A: addition - B: subtraction - C: multiplication - D: division  E = {A, B, C, D}

17. Competence-Performance Approach - Example: Prerequisite Relation for the competencies in the domain E = {A,B,C,D}

18. Competence-Performance Approach - Example: We can establish a Competence Space, which does not contain all of the 2 4 possible Competence States but only 7

19. Competence-Performance Approach - Unfortunately, we cannot observe this…  Representation and Interpretation Functions enable to map test items / tasks to the competencies  We can determine a person’s Competence State  We can determine the required competencies to master a specific task  No 1-to-1 mapping required

20. Competence-Performance Approach - Example: Two tasks from the domain “basic algebra”: - a: multiplication problem - b: solving linear equations  Q = {a, b}  Representation Function ProblemCompetence State a{A, B, C} b{A, B, C, D}

21. Competence-Performance Approach - Example: - Solved a but not b - Solved a and b - Solved not a and not b - Solved b and not a

22. Advantages - Modeling a domain of knowledge on a formal basis - Referring to clearly defined and unique competencies - Mapping different assessment methods to the same set of competencies - Efficient adaptive testing - Efficient adaptive teaching - Modeling of individual learning path - Computable

23. So what about errors?  Careless errors  Lucky Guesses - Besides the deterministic approach there are also probabilistic approaches

24. Applications - Adaptive, personalized eLearning (RATH, APeLS, EASEL, iCLASS, ELeGI, ELEKTRA) - Organizational competencies / knowledge (Know-Center) - Research tool (e.g., in child development)

25. Applications - Commercial eLearning platform ALEKS (www.aleks.com) Image courtesy of ALEKS Corporation Santa Ana, CA, USA

26. THANK YOU