1. Skill Assignment in Dynamic Work Domains Tobias Ley 13 October 2003 Reliability, Validity and Use of the Structures

2. 2 Overview Data Collection for Study 2 Reliability of individual structures Validity of individual structures Using the structures Data Collection Validity Using the Structures Reliability

3. 3 Purpose of Study II Estimate reliability of results Establish validity measures Establish organizational structure Data Collection

4. 4 Results of Study I Data Collection 113 competencies 47 documents 7 persons

5. 5 Content Analysis 1.Elimination of Similar Competencies N=99 remaining 2.General vs Domain Specific Competencies General (N=56): Skills of a Project Manager e.g. Research Skills, Communication, Project Management etc. Domain (N=43): Technical Knowledge and Skills e.g. Knowledge of elearning, Regional development etc. 3.Categorization of General Competencies 14 Categories Data Collection

6. 6 Results of Content Analysis Data Collection 47 documents 14 Categories

7. 7 Data Collection: Document-Competency Rating Data Collection Reliability Measures

8. 8 Data Collection: Document Similarity Rating „Your task is to introduce a new employee to your work at the Know-Center.“ „How Similar are the two Documents, if... –A:... the new employee has a different background than yourself and goal is to give an introduction about general work requirements?“ –B:... the new employee has the same background as yourself and goal is to introduce your professional expertise?“ Data Collection Situation A: „General Introduction“ Situation B: „Professional Introduction“

9. 9 Results: Reliability of document-competency ratings SubjectsNumber of ratings r 1900.136 21440.500 31120.558 41020.231 51120.125 61600.641 7550.659 Total7750.494 p 11 p 21 p1p1 p 12 p 22 p2p2 p 1 p 2 1 Rating I Rating II Goodman & Kruskal (1954) Reliability

10. 10 Validity: Similarity of Documents s A similarity rated in Situation A [1-5] s B similarity rated in Situation B [1-5] s g similarity in general job competencies required s d similarity in domain specific competencies required Mean Correlation r sgsg sdsd sAsA 0.3600.305 sBsB 0.2750.355 Validity Competency rating for... s g = p 11 + p 22 p 11 p 21 p1p1 p 12 p 22 p2p2 p 1 p 2 1 Doc 1 Doc 2 1 0 0 1 Doc 1:1001101000100011 Doc 2:0101101110101101

11. Competencies 1501 1502 1503 1504 1505 1506 1507 Documents 6 10 11 12 13 14 24 26 33 41 42 45........ 6 42..3.... 10 1234.67 41 11 1..4..7 12 1.34..7 1..4... 13 1.34......4..7 14..34..7 1234567 45 17 12...6. 24 123..6. 12.4.67 12.4.6. 1234.6. 12.4..7 1234..7 12.4... 26 1234........67 33..3..67 1..4.67 1.34.67...4.67..34.6712...67 123..67