Download presentation
Presentation is loading. Please wait.
Published byMay Daniel Modified over 9 years ago
1
R ATERS T RAINING AS A S AFETY C ONCERN Sergey MELNICHENKO CompLang Aviation Training Centre (Moscow), Deputy Director Associate Member ICAO LPR Implementation Workshop Russia, St-Petersburg, May 24-26, 2011
2
R ATERS T RAINING AS A S AFETY C ONCERN Associate Member - RATER AS A DECISION MAKER - SELECTION OF PERSONNEL - AB INITIO TRAINING - RATERS CERTIFICATION - INTER-RATER RELIABILITY - INTRA-RATER RELIABILITY - RASCH ANALYSIS - RECURRENT TRAINING - EVALUATION OF RATER’S ADEQUACY ICAO LPRI Workshop, St-Petersburg, Russia, 2011
3
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
4
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Aviation English test results are felt around the globe
5
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern Associate Member Inadequacy in Level Evaluation Level 3 instead of Level 4: Loss of job and/or occupation Financial losses for AO or ANSP Market losses Redistribution of air transportation services Lack of confidence for test-takers
6
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Inadequacy in Level Evaluation Level 4 instead of Level 3: Aviation safety is compromised Degradation to Level 2 Overconfidence for test-takers Raters Training as a Safety Concern
7
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
8
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern ICAO recommendations on interlocutors and raters qualifications
9
Raters Training as a Safety Concern ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member ICAO recommendations on interlocutors and raters qualifications (Doc 9835) Familiarity with ICAO documentation Annex 1; Doc 9835; ICAO Rated Speech Samples CD; ICAO Circ 318
10
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern ICAO recommendations on interlocutors and raters qualifications (Doc 9835) Familiarity with ICAO documentation Operational and/or language specialist expertise Level 5 or Level 6 proficiency Familiarity with Aviation English RATERS CODE OF ETHICS Associate Member
11
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
12
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Theory of language testing ICAO documents: Annex 1, Annex 10, Doc 9432, Doс 9835, Circ 318 The ICAO Proficiency Rating Scale and holistic descriptors Correlating speech samples with the ICAO LPR Scale
13
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Raters training course should be aimed at experienced non-native and native English language teaching professionals and subject-matter experts who are currently teaching Aviation English and who want to be able to rate according to the ICAO Language Proficiency Scale
14
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
15
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Certification by national Civil Aviation Authority or by international aviation organization is mandatory
16
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Is geographical principle necessary?
17
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern Associate Member Are quotas necessary?
18
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
19
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Different raters but same results
20
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
21
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern Associate Member Different times but same results
22
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
23
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Evidence of the validity of the responses
24
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Evidence of the validity of the responses Correctional refresher course
25
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern Associate Member Evidence of the validity of the responses Correctional refresher course Administrative measures to marginal raters
26
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
27
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Annual refresher course for raters
28
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Annual refresher course for raters New materials
29
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern Associate Member Annual refresher course for raters New materials Practical training
30
Associate Member ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Raters Training as a Safety Concern RATER AS A DECISION MAKER SELECTION OF PERSONNEL AB INITIO TRAINING RATERS CERTIFICATION INTER-RATER RELIABILITY INTRA-RATER RELIABILITY RASCH ANALYSIS RECURRENT TRAINING EVALUATION OF RATER’S ADEQUACY
31
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Rater is not a lifetime profession RATER TRAINING IS A SAFETY CONCERN
32
ICAO LPRI Workshop, St-Petersburg, Russia, 2011 Associate Member Raters Training as a Safety Concern Thank you
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.