1. Adrian Enright EUROCONTROL
ICAO Circular 318
Adrian Enright
EUROCONTROL

2. ICAO Circular 318 Gives more detailed guidance
Has recommended criteria for selection of aviation language testing programmes
An indispensable guideline to CAA regulators, aircraft operators and air navigation service providers

3. An indispensable guideline to aircraft operators …
Captain? Captain can you hear me? …have you found my copy of ICAO Circular 318?
An indispensable guideline to aircraft operators …

4. Circular 318 ICAO LPR apply to all languages, but
Circular 318 focuses on English as the language for which most training and testing programmes need to be developed
Circular 318 will not replace the extensive standards, guidelines and principles found in the literature on language testing

5. Background 1998 32nd ICAO Assembly 2003 comprehensive set of SARPs
Annex 1 Personnel Licensing
Annex 6 Operation of Aircraft
Annex 10 Aeronautical Communications
Annex 11 Air Traffic Services
PANS-ATM
March 5th 2011 final deadline for implementation of LPRs

6. Fundamental test principles
Validity
Reliability
Practicability

7. Test purposes Diagnostic Placement Progress Achievement Aptitude
Proficiency

8. Test focus Speaking and listening proficiency High stakes test
Work related context
ICAO Rating Scale, and
Holistic descriptors

9. Use of ICAO phraseology
Testing phraseology is a operational issue, but
Use of phraseology in a work-related context is acceptable
Can be used as a warm-up, or
In setting the context

10. Recommended criteria
Self-contained statements with supplementary information
What it means
Why it is important
Additional information

11. Test Design and Construct
Tests should address the specific and unique requirements of aviation language testing
A separate listening test can provide independent information about comprehension
Tests should be specific to aviation operations –
Visual AND non-visual communications
customize tests for pilots or controllers
Language proficiency tests should not be designed to evaluate technical knowledge
The final score is the LOWEST of the six rated criteria

12. Rating Rating should be carried out by a minimum of two raters
An independent 3rd rater when the 2 raters cannot agree between level 4 and level 3

13. Test familiarisation Complete sample test freely available
Helps to ensure that test takers are not surprised by the format of the test and are properly able to demonstrate their language proficiency
Test description or a marketing brochure is not sufficient

14. Test security
Test security is essential for ensuring reliability of, and confidence in, the testing system
A full description of security measures should be available to all decision makers

15. Testing Team Recommended 2 raters
language specialist expertise
operational expertise
Should demonstrate at least the highest level of proficiency expected of test takers
Familiar with aviation English vocabulary and structures
Complete recurrent training at least once a year

16. Rater reliability Intra-rater reliability Inter-rater reliability
Extent to which a particular rater consistently uses a proficiency scale
Inter-rater reliability
Extent to which a group of independent raters agree on a test taker’s performance

17. Checklist Chapter 3 provides a simple checklist
Cross-referenced to the sub-paragraphs
Assist States in determining:
how far they have gone
how far they have still to go
in implementing valid and reliable language proficiency tests

18. What? That’s not the pre-flight checklist…
Is the rating process documented?

19. Glossary Useful glossary of testing terms and acronyms
“Everything you wanted to know about testing but were afraid to ask”
Not quite, but a good start.

20. Let us not forget just one reason why we set off on this journey to improve aviation English
Tenerife March 1977

21. Thank you A final thought …
We have come a long way in the last 12 years in improving language proficiency in aeronautical communication, but the journey is not over yet. Our commitment today is in laying the foundation for generations to come.
Thank you