English language Proficiency for Aeronautical Communication ELPAC English language Proficiency for Aeronautical Communication Adrian Enright EUROCONTROL ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC 1. Introduction 2. Test structure 3. Test development 4. The future ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Test Objectives Design a specific purpose proficiency test to meet ICAO and EC language proficiency requirements Develop test specifications based on a Needs Analysis Continuous review and evaluation Deliver a test that assesses the English language proficiency of the air traffic controller Needs analysis reviewed the report and conclusions produced for the PELA Test and found no change in the language functions associated with aeronautical communication. Test specifications, based on the needs analysis, considered the language requirements of the air traffic controller in the context of the profession. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

There is a need for a specific purpose proficiency test A simple difference between HELP and HELF. But if you cannot make the connection then go back to school (English classes that is). ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Development Team Development team of ATC and English language experts 6 States and Eurocontrol Consultant Project team Essential to combine both disciplines. Bulgaria, Czech Republic, France, Germany, Hungary and Spain. = 11 Dr. Rita Green – language testing expert with experience of many international projects Adrian Enright (ATC), project leader (also for the PELA Test), 20 tears experience in language proficiency test development. Magdalena Vecerova, project assistant - language testing expert from Czech Republic. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC 1. Introduction 2. Test structure 3. Test development 4. The future ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Test Design Evaluating all ICAO language skills Levels 4 and 5 Tasks elicit phraseology and plain language 2 test papers 1. Listening Comprehension (internet based) 2. Oral Interaction Candidates must achieve level 4 as a prerequisite to assessment at level 5. A pass must be achieved in BOTH papers with the lowest level being the actual level accepted. E.g. a 4 in Paper 1 and a 5 in Paper 2 = level 4. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Paper 1 Internet based Testing understanding Controller/pilot Controller/controller Routine & non-routine situations Based on authentic communications Range of accents ensured 6 parts, approximately 60 items, assesses performance at level 4 and level 5. Original RTF and telephone recordings are transcribed, modified to provide suitable items, de-sensitised to preserve anonymity as much as possible whilst remaining realistic and re-recorded to provide a variety of accents. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Test Security High stakes test = high level of security Servers on 3 layers Different rights of access for different users Economic to administer Confidentiality ensured Different users: administrators (National and local/regional); markers; test developers (limited to 4) ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Paper 2 (i) Candidates must demonstrate: Appropriate use of phraseology (RTF) Switching between RTF and plain language Making appropriate responses Resolving misunderstandings Dealing effectively with pilot/controller relationship Negotiating an unusual situation From ELPAC test specifications for tasks 1a and 1b ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

With acknowledgements to Martin Germans It usually takes a few minutes to get this one. Don’t expect laughs from the non-ATC people. Move to next slide… The controller must respond appropriately to pilot messages With acknowledgements to Martin Germans ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Paper 2 (ii) Dealing effectively with pilot/controller relationship Negotiating an unusual situation Making a verbal report in plain English Produce extended speech in an aviation context Give opinions Put forward arguments Evaluate Speculate Hypothesise From ELPAC test specifications for task 2 ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC 1. Introduction 2. Test structure 3. Test development 4. The future ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Trialling Sample of controllers MUST reflect target population Ensure “Face validity” - feedback Already two major trialling exercises 20 countries 237 controllers Next trial – October 2006 +/- 300 controllers Ensure standardisation of administrators/examiners Representative sample of the target population (different countries, ATC functions, gender, work experience, language background etc.) Sample of controllers assisting to the trialling must reflect the target population otherwise the results from trialling could be invalid. 19 countries (+ Eurocontrol): Austria, Belgium, Bulgaria, Croatia, Czech Republic, Finland, France, Germany, Greece, Hungary, Ireland*, Lithuania, Macedonia, The Netherlands, Romania, Serbia & Montenegro*, Slovenia*, Spain, Switzerland and MUAC (Poland, Estonia) ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Test Evaluation Quantative and qualitative analysis Test results Feedback forms Bio data Reports etc. Check how tasks and items perform Determine reliability Evaluate “fairness” of the test Feedback from the test takers is a very important part of data collection and gives insight into the “face validity” of ELPAC. We check how the test tasks and items perform, whether they do not discriminate against a particular group of controllers, how reliable is the test etc. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – Test Validation Internal Periodic checks that ELPAC meets purpose and requirements External Team of language testing experts from Lancaster University Independent “audit” of whole test package Essential that an independent, expert validation of test design and test development processes is carried out. Initial external validation in May 2006 confirmed that ELPAC is following best practice. Final validation will be in May 2007. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC 1. Introduction 2. Test structure 3. Test development 4. The future ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC – 2006/2007 Major trialling exercises Sample tests October 2006 March 2007 May 2007 Sample tests Production of training packages Production of supporting materials We are just over half-way through test development. More items need to be trialled even to produce a minimum number of test versions for July 2007. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC Delivery - July 2007 Evaluation Training Assisting Member States with implementation Ensuring sustainability of the ELPAC Test Between July 2007 and March 2008 ELPAC test evaluation. Assisting States with the training of administrators and examiners. Providing guidance to ANSPs to establish a process that will ensure the continued success of ELPAC. ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

ELPAC The ELPAC Test Development Team applies best practice in language testing Language skills, not ATC skills are assessed ELPAC is a specific purpose language proficiency test ICAO Regional Workshop Language Proficiency Requirements Paris September 2006

Thank you for your attention Any questions? ICAO Regional Workshop Language Proficiency Requirements Paris September 2006