1. English language Proficiency for Aeronautical Communication
ELPAC
English language Proficiency for Aeronautical Communication
Adrian Enright
EUROCONTROL
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

2. ELPAC 1. Introduction 2. Test structure 3. Test development
4. The future
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

3. ELPAC – Test Objectives
Design a specific purpose proficiency test to meet ICAO and EC language proficiency requirements
Develop test specifications based on a Needs Analysis
Continuous review and evaluation
Deliver a test that assesses the English language proficiency of the air traffic controller
Needs analysis reviewed the report and conclusions produced for the PELA Test and found no change in the language functions associated with aeronautical communication.
Test specifications, based on the needs analysis, considered the language requirements of the air traffic controller in the context of the profession.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

4. There is a need for a specific purpose proficiency test
A simple difference between HELP and HELF. But if you cannot make the connection then go back to school (English classes that is).
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

5. ELPAC – Development Team
Development team of ATC and English language experts
6 States and Eurocontrol
Consultant
Project team
Essential to combine both disciplines.
Bulgaria, Czech Republic, France, Germany, Hungary and Spain. = 11
Dr. Rita Green – language testing expert with experience of many international projects
Adrian Enright (ATC), project leader (also for the PELA Test), 20 tears experience in language proficiency test development.
Magdalena Vecerova, project assistant - language testing expert from Czech Republic.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

6. ELPAC 1. Introduction 2. Test structure 3. Test development
4. The future
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

7. ELPAC – Test Design Evaluating all ICAO language skills Levels 4 and 5
Tasks elicit phraseology and plain language
2 test papers
1. Listening Comprehension (internet based)
2. Oral Interaction
Candidates must achieve level 4 as a prerequisite to assessment at level 5.
A pass must be achieved in BOTH papers with the lowest level being the actual level accepted. E.g. a 4 in Paper 1 and a 5 in Paper 2 = level 4.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

8. ELPAC – Paper 1 Internet based Testing understanding Controller/pilot
Controller/controller
Routine & non-routine situations
Based on authentic communications
Range of accents ensured
6 parts, approximately 60 items, assesses performance at level 4 and level 5.
Original RTF and telephone recordings are transcribed, modified to provide suitable items, de-sensitised to preserve anonymity as much as possible whilst remaining realistic and re-recorded to provide a variety of accents.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

9. ELPAC – Test Security High stakes test = high level of security
Servers on 3 layers
Different rights of access for different users
Economic to administer
Confidentiality ensured
Different users: administrators (National and local/regional); markers; test developers (limited to 4)
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

10. ELPAC – Paper 2 (i) Candidates must demonstrate:
Appropriate use of phraseology (RTF)
Switching between RTF and plain language
Making appropriate responses
Resolving misunderstandings
Dealing effectively with pilot/controller relationship
Negotiating an unusual situation
From ELPAC test specifications for tasks 1a and 1b
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

11. With acknowledgements to Martin Germans
It usually takes a few minutes to get this one. Don’t expect laughs from the non-ATC people. Move to next slide…
The controller must respond appropriately to pilot messages
With acknowledgements to Martin Germans
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

12. ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

13. ELPAC – Paper 2 (ii)
Dealing effectively with pilot/controller relationship
Negotiating an unusual situation
Making a verbal report in plain English
Produce extended speech in an aviation context
Give opinions
Put forward arguments
Evaluate
Speculate
Hypothesise
From ELPAC test specifications for task 2
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

14. ELPAC 1. Introduction 2. Test structure 3. Test development
4. The future
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

15. ELPAC – Trialling Sample of controllers MUST reflect target population
Ensure “Face validity” - feedback
Already two major trialling exercises
20 countries
237 controllers
Next trial – October 2006
+/- 300 controllers
Ensure standardisation of administrators/examiners
Representative sample of the target population (different countries, ATC functions, gender, work experience, language background etc.)
Sample of controllers assisting to the trialling must reflect the target population otherwise the results from trialling could be invalid.
19 countries (+ Eurocontrol): Austria, Belgium, Bulgaria, Croatia, Czech Republic, Finland, France, Germany, Greece, Hungary, Ireland*, Lithuania, Macedonia, The Netherlands, Romania, Serbia & Montenegro*, Slovenia*, Spain, Switzerland and MUAC (Poland, Estonia)
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

16. ELPAC – Test Evaluation
Quantative and qualitative analysis
Test results
Feedback forms
Bio data
Reports etc.
Check how tasks and items perform
Determine reliability
Evaluate “fairness” of the test
Feedback from the test takers is a very important part of data collection and gives insight into the “face validity” of ELPAC.
We check how the test tasks and items perform, whether they do not discriminate against a particular group of controllers, how reliable is the test etc.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

17. ELPAC – Test Validation
Internal
Periodic checks that ELPAC meets purpose and requirements
External
Team of language testing experts from Lancaster University
Independent “audit” of whole test package
Essential that an independent, expert validation of test design and test development processes is carried out. Initial external validation in May 2006 confirmed that ELPAC is following best practice. Final validation will be in May 2007.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

18. ELPAC 1. Introduction 2. Test structure 3. Test development
4. The future
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

19. ELPAC – 2006/2007 Major trialling exercises Sample tests
October 2006
March 2007
May 2007
Sample tests
Production of training packages
Production of supporting materials
We are just over half-way through test development. More items need to be trialled even to produce a minimum number of test versions for July 2007.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

20. ELPAC Delivery - July 2007 Evaluation Training
Assisting Member States with implementation
Ensuring sustainability of the ELPAC Test
Between July 2007 and March 2008 ELPAC test evaluation.
Assisting States with the training of administrators and examiners.
Providing guidance to ANSPs to establish a process that will ensure the continued success of ELPAC.
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

21. ELPAC
The ELPAC Test Development Team applies best practice in language testing
Language skills, not ATC skills are assessed
ELPAC is a specific purpose language proficiency test
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006

22. Thank you for your attention
Any questions?
ICAO Regional Workshop Language Proficiency Requirements
Paris September 2006