Skill Assignment in Dynamic Work Domains Tobias Ley 13 October 2003 Reliability, Validity and Use of the Structures

2 Overview Data Collection for Study 2 Reliability of individual structures Validity of individual structures Using the structures Data Collection Validity Using the Structures Reliability

3 Purpose of Study II Estimate reliability of results Establish validity measures Establish organizational structure Data Collection

4 Results of Study I Data Collection 113 competencies 47 documents 7 persons

5 Content Analysis 1.Elimination of Similar Competencies N=99 remaining 2.General vs Domain Specific Competencies General (N=56): Skills of a Project Manager e.g. Research Skills, Communication, Project Management etc. Domain (N=43): Technical Knowledge and Skills e.g. Knowledge of elearning, Regional development etc. 3.Categorization of General Competencies 14 Categories Data Collection

6 Results of Content Analysis Data Collection 47 documents 14 Categories

7 Data Collection: Document-Competency Rating Data Collection Reliability Measures

8 Data Collection: Document Similarity Rating „Your task is to introduce a new employee to your work at the Know-Center.“ „How Similar are the two Documents, if... –A:... the new employee has a different background than yourself and goal is to give an introduction about general work requirements?“ –B:... the new employee has the same background as yourself and goal is to introduce your professional expertise?“ Data Collection Situation A: „General Introduction“ Situation B: „Professional Introduction“

9 Results: Reliability of document-competency ratings SubjectsNumber of ratings r Total p 11 p 21 p1p1 p 12 p 22 p2p2 p 1 p 2 1 Rating I Rating II Goodman & Kruskal (1954) Reliability

10 Validity: Similarity of Documents s A similarity rated in Situation A [1-5] s B similarity rated in Situation B [1-5] s g similarity in general job competencies required s d similarity in domain specific competencies required Mean Correlation r sgsg sdsd sAsA sBsB Validity Competency rating for... s g = p 11 + p 22 p 11 p 21 p1p1 p 12 p 22 p2p2 p 1 p 2 1 Doc 1 Doc Doc 1: Doc 2:

Competencies Documents