Influence of the Criteria Changes on the Supervisory Rating M á r t a J u h á s z Budapest University of Technology and Economics Dep. of Ergonomics and Psychology E-mail: armima@yahoo.com 15.05.2003. Lisboa

previous study1: Assessment process Assessment process in 1996 at the Public Inquiry Service of a Hungarian Telecom Organization. Motivation of the organization to develop a new selection process was: the introduction of the new information technology to have the more suitable employees who were able to improve and keep their job performance high.

previous study1: Assessment process Job and criteria analysis results of previous job analysis analysis of documents survey among supervisors, employees group discussions working day analysis I. Cognitive ability (Memory, Attention, Emphasing the importance, Comprehensihon) II. Personality traits (Politeness, Self-discipline, Tolerance)

previous study 1: Assessment process Development of the assessment methods We measured the cognitive ability with well-known techniques and with our own instruments. The personality traits were measured by a Big Five personality questionnaire which consists of five major domains (factors) of personality, as well as the six facets (traits) that define each domain. (N=344) I. Neuroticism (N) II. Extroversion (E) III. Openness to experience (O) IV. Agreeableness (A) V. Conscientiousness (C)

previous study 2: Validation process The validation process was in 1997. According to the results from the assessment methods the employees could be divided into 3 groups. Suitable SuitableWithDevelopment Unsuitable Our hypothesis was that among these groups there would be significant differences in the performance measured. This was a validation process of our assessment methods. (N=100)

Methods of the validation process perf.analysis based on IBM system operators rating by each other perf.analysis based on ACD system work-phase analysis Supervisory rating Methods of validation process categorising and analysing the errors speaking style evaluation analysing occupational status

The supervisory rating For the evaluation of the employees’ performance by their supervisors a scoring sheet was used which contained the exact determination of the dimensions to be used. Supervisors used a 4-point scale for the evaluation. The rating categories of the scoring sheet were determined by the criteria of the selection. 1) Achievement, 2) Accuracy, 3) Software Knowledge, 4) Endurance, 5) Quietness, 6) Politeness, 7) Good communication skills, 8) Pleasant voice, 9) Understanding of speech, 10) Memory, 11) Good relationship with the colleagues.

Questions to consider With the changes in the circumstances of the organisation, requirements of fulfiling this job have been changed, or not? The supervisory rating about the employees’ performance is actually followed by criteria changing, or not?. Depending on the criteria changing the different employees’personality will be preferred by supervisors, or not?

Hypothesis and methods In 1996 the whole organisation was in a learning period. We supposed the role of the cognitive abilities was more important than the personality traits. The predictive validity of the Agreeableness (A), Conscientiousness (C) and Extroversion (E) personality factors are crucial in the service job. Supervisory ratings applied in 1997 have been repeated in 2000 and 2002. We wanted to get information about: the changes of criteria and their reflections in the supervisory ratings during the 6 year period. The differences of the supervisory rating were pointed out through the operators’ personality.

Criteria analysis NIT 1996 2002 1. Emphasizing the importance Rating 1. Emphasizing the importance 2. Quickness 3. Attention 4. Software Knowledge 5. Politeness (A) 6. Helpfulness (A) 7. Memory 8. Flexibility (O) 9. Accuracy 10.Tolerance (N) 1. Patience (N) 2. Politeness (A) 3. Accuracy (C) 4. Communication skills 5. Helpfulness (A) 6. Intelligence 7. Emphasizing the importance 8. Quickness 9. Tolerance (N) 10. Dutifulness (C)

Characteristics of the supervisory ratings 1997-2000-2002 Cognitive performance factor Understanding of speech Memory Software knowledge Achievement Good communication skills Accuracy Endurance Affective performance factor Quietness Politeness Pleasant voice Self-discipline Good relationship with colleagues Co-operation

COGNITIVE 2002 1997 2000 A F E CTIVE NAN: Anxiety, NSC: Self consciousness, NAH: Angry hostility, E: Extroversion, EEX: Excitement seeking, EGR: Gregariousness, EPE: Positive emotions, EAC: Activity, O: Openness to experience, OAC: Actions, OVA: Value, OFA: Fantasy, A: Agreeableness, ACO: Compliance, AMO: Modesty, ATR: Trust, ATM: Tender Mindedness, AAL: Altruism.