1. Training for Proficiency or Training for success in a test?
Philip Shawcross
ICAEA
ICAO LPRI workshop, ENAC, Rome, 4th March 2010

2. Proficiency or success?
Training for Proficiency or
Training for success in a test?
What are the concerns?
Forms of negative washback
Different types of assessment
How can proficiency be trained for?
How do we know we are getting it right?
Conclusion

3. What are the concerns? (1)
Wide variety of test types and quality
No international test accreditation process
A test may have too narrow a focus
A test may not adequately assess all skills
A test may not sufficiently simulate operational conditions
Many tests have well-defined & well known tasks
Encourages use of “easiest” test

4. What are the concerns? (2)
Training to a test does not necessarily prepare for the real world
“Taking our eye off the ball”
Detrimental to learners long-term view of language
Targeted proficiency has wider scope than any test
Proficiency to be defined in operational terms
Need for sustainable levels of proficiency

5. Forms of negative washback (1)
Communication content may be limited by test content
Communication functions may be limited by test format
Particular skills (e.g. interaction) may not be sufficiently trained
Tendency to train to narrow focus
Excessively modular approach to language

6. Forms of negative washback (2)
Tendency to train only to minimum level
Training may “ape” test rather than prepare for real world
Tendency to learn “parrot-fashion”
Lack of separation between training and testing processes and staff
Publication of a bank of test questions

7. Different types of assessment
Benchmark / prognostic tests
Placement tests
Entry tests
Progress tests
Exit tests
Proficiency tests (licensing)

8. How can proficiency be trained for? (1)
Training should have its eye on:
The “big” picture
The long term
Operational realities
Whole range of communicative functions
The acid test of communicating safely
Ability to manage the unexpected
A safety margin
Sustainability

9. How can proficiency be trained for? (2)
How can this be put into practice?
Aviation English training should:
Be modelled on operations, not testing
Provide sufficient practice to achieve a solid level of proficiency
Be approved by SMEs
Explore a wide range of situations
Be applicable to operational situations (lexis) and communicative functions (structure)

10. How can proficiency be trained for? (3)
Take students beyond minimum Level 4
Prioritize expression, fluency & interactions (classroom)
Adopt a holistic approach to language
Make language training a corporate project
Incorporate language maintenance strategies & recurrent training
Receive feedback from operations for adjustment

11. How can proficiency be trained for? (4)
… and more generally
Foster operational effectiveness rather than linguistic correctness
Develop appropriate cognitive processes
Encourage flexibility & adaptability in language use
Respect the students’ different learning styles & speeds
Build self-confidence & self-esteem

12. How do we know we are getting it right?(1)
Integrate language training as a core institutional activity, not a peripheral activity ---> a closed loop for greater awareness & visibility
Do not only rely on progress & exit tests
-- more holistic assessment
Develop students’ own pedagogical self-awareness
Consult SMEs on students’ performance

13. How do we know we are getting it right?(2)
Use feedback from operational monitoring
Pay particular attention to holistic skills (fluency, interactions & discourse management) as criteria of proficiency
Ask “Would I entrust my life to this person in an emergency?”

14. Conclusions Training High stakes Operations-driven
Integrated set of skills
Targets sustainable proficiency
Part of life-long learning

15. Training for Proficiency or Training for success in a test?
Thank you very much
for listening
ICAO LPRI workshop, ENAC, Rome, 4th March 2010