1. Language Implications of NATO’s Expanding Roles Dr. Ray T. Clifford BILC Conference, San Antonio 21 May 2007

2. Welcome to this year’s BILC conference! What is “BILC”?

3. What is BILC? Bureau for International Language Co-ordination A consultative and advisory body for language training matters in NATO.

4. Language Matters “The single most important problem identified by almost all Partners as an impediment to developing interoperability with the Alliance has been shortcomings in communications.” EAPC (PARP-C) D (97) 1, page 10.

5. BILC Within NATO

6. NATO’s only language training organization –Independently chartered 1966 The UK (1966-1981), Germany (1982-1996), and US (1997-2007) have staffed the Secretariat. –Conducts an annual Conference each Summer –Sponsors a PfP Professional Seminar in the Fall –Responsible for STANAG 6001 –Provides services to support the International Military Staff and the International Staff: Language Training Assessments Language Testing Seminars Bureau for International Language Co-ordination

7. Why are we here? Language matters have never been more important to NATO. NATO language needs have increased as NATO’s role has expanded. –The Cold War era. –The post Cold War transformation. –The post September 11, 2001 era. This year’s presentation and study group topics reveal the scope of these expanded language needs.

8. Expanding Needs There are more members and more partners in the Alliance. NATO Response Forces now require both: –English for interoperability. For staff positions. For NCOs during unit training and operations. –Languages Other Than English For conflicts outside of Europe. For counterterrorism.

9. The Challenge is formidable: Language is the most complex of human behaviors.

10. Language is the most complex of human behaviors. “Kuuuurijate töööö jäääärel.”

11. “Kuuuurijate töööö jäääärel.” “A moon researcher’s work- night at the edge of the ice.” Example from a Brochure on the Estonian Language:

12. Language is the most complex of human behaviors. Even professional translators can make mistakes…

14. Sign in Moscow Hotel If this is your first visit to Moscow, you are welcome to it.

15. Hong Kong Tailor Shop Ladies may have a fit upstairs.

16. Instructions for Thermostat in Japanese Hotel Cooles and Heates: If you want just condition of warm in your room, please control yourself.

17. Cocktail Lounge in Norway Ladies are requested not to have children in the bar.

18. Even Simple Language Isn’t Simple

19. Expertise Needed When faced with complex problems… –Novices will always see more options than are seen by the experts. –However, experts will see more solutions ! We are here to gather the expertise needed to find solutions. –Presentations. –Study Groups. –Discussions of the Steering Committee.

20. Some Items on this Week’s Steering Committee Agenda Review of 2006 action items. –Data base of language courses taught in support of NATO operations. –Language training support for the JSSG TEPSO Working Group. –Upgrading the BILC website. –Recommendation that the force goals (EG 0356) are unrealistically high. –Submission of reading items for the BAT.

21. Some Items on this Week’s Steering Committee Agenda (Continued) Progress on the BAT. Review of BILC Constitution and Rules of Procedure. JSSG tasking to create a third edition of STANAG 6001. Transition of BILC Secretariat responsibilities.

22. Some Historical Background

23. “Progress on the BAT” What is the BAT?

24. What is a BAT? BAT is an acronym for… Benchmark Advisory Test(s)

25. What is a Benchmark Advisory Test? Purpose. –The test is designed for NATO military and civilian personnel who are not native speakers of English. –The purpose of the BAT is to assess international English language proficiency, regardless of how it was acquired, by testing reading comprehension of a variety of authentic written materials representing a range of topics, and author purposes. –The test may also be used as an external criterion for validation of nationally-developed tests.

26. BAT History 2005 – BILC –The concept of Benchmark Advisory Tests was approved, without financial support –Reading test specifications drafted. –Working Group Volunteers began submitting items. 2006 – NATO, ACT –Formally asked that tests be developed. –Offered funding, then decided that the funding had to be competitively awarded. –October: Published the bid requirements Deliver international language testing capability in 4 skills. All tests must be finished within a year. 30 days to prepare a proposal. The lowest cost proposal would win.

27. BAT History 2006 – BILC Secretariat –October: Formed a coalition with a professional organization (The American Council on the teaching of Foreign Languages) which was already providing speaking and writing proficiency tests internationally. –November: Submitted a proposal. –December: The BILC/ACTFL proposal won the competition and a contract was awarded. –Financial support for travel to meetings of the working group was at last available.

28. Our Professional Partner The American Council on the Teaching of Foreign Languages (ACTFL) currently has standardized proficiency tests and test delivery capabilities: –Speaking 50 languages –Writing 12 languages –Reading and listening Under construction

29. The Benchmark Advisory Test Built on Voluntary National Contributions –Pool of reading items from 12 BILC member nations –Working Group of Testing experts from BILC member nations Test delivery –Online reading, listening, and writing –Telephonic speaking test with digitized sound files Test scoring –Reading and listening scored online –Speaking and writing scored by trained raters

30. Benchmark Advisory Test Progress to date: –Initial software (alpha version) developed –Test specifications for reading and listening –Review and selection of initial pool of reading items –Conducted Modified Angoff validation of reading test items –Beginning the design specifications for speaking and writing tests –Developing a workshop to train speaking and writing test raters

31. Benchmark Advisory Test Future enhancements –Online instructions and familiarization items. –Computer-adaptive versions of reading and listening tests

32. Creating BAT Tests is a Complex Process Language is the most complex of human behaviors. Testing is also a complex process. Language Testing is complexity squared.

33. 3 Major Types of Tests Achievement Performance Proficiency

34. 3 Major Types of Tests Achievement –Memorized responses using the content of a specific textbook or curriculum. Performance –Rehearsed ability to communicate in specific, familiar settings. Proficiency –Unrehearsed general ability to accomplish real-world communication tasks across a wide range of topics and settings.

35. Different Types of Tests Achievement –Rehearsed, memorized responses –Tests what is taught –Content of a specific textbook or curriculum

36. Sample Achievement Test Item Complete the following with the correct verb form in the past. (go) I _____________ to the United States last year. (be) My seat on the plane _______ in business class. (have) My associates and I _________ meetings each day. (eat) We _________ at typical American restaurants.

37. Different Types of Tests Performance –Semi-rehearsed and rehearsed responses –Ability to communicate in specific, familiar, and predictable settings –What one can do with what has been taught and practiced

38. Sample Performance Test Item Complete the following sentences about an upcoming business trip. Add a minimum of 5 additional words to each sentence. For an upcoming business trip I plan to ____________________________________________. I am certain that the trip will be successful because ____________________________________________.

39. Different Types of Tests Proficiency –Spontaneous, unrehearsed communication –General ability to accomplish communication tasks in a variety of settings –Skill is transferable from one context to another

40. Sample Proficiency Test Item You are planning a business trip abroad. You contact a travel agent. You need to purchase plane tickets, arrange hotel reservations, and get information about local transportation, but you encounter significant hurdles that you had not anticipated, including …

41. A Summary of the Contrasts Achievement, Performance and Proficiency Achievement Memorized Performance Rehearsed Proficiency Unrehearsed Task Repeat, produce, choose Show specific skills Demonstrate a wide range of abilities Context Textbook, Curriculum Focused, restricted Broad, in-depth, variable Accuracy Determined by the teacher Situation dependent Ascending expectations

42. The Benchmark Advisory Tests Are Proficiency Tests

43. In the next series of charts… Each observable increase in ability could: – Represent a week of intensive language study. –Be tested using an achievement test. Multiple observable achievements could be tested with a performance test. Multiple performance gains across a range of topics and tasks could be tested with a general proficiency test.

245. 3 Major Types of Tests Achievement = Memorized responses using the content of a specific textbook or curriculum. Performance = Rehearsed ability to communicate in specific, familiar settings. Proficiency = Unrehearsed general ability to accomplish real-world communication tasks across a wide range of topics and settings.

246. Every Level of the STANAG 6001 Proficiency Scale Describes: Tasks A statement of what is to be done or accomplished. Conditions A description of the conditions under which (or the context in which) the task is to be performed. For language testing, this includes the topics to be addressed. Accuracy A definition of how well the task must be performed under the conditions stated.

247. 5 LEVELTASKSCONTEXT/TOPICSACCURACY 4 3 2 1 0 All expected of an educated NS All subjects Accepted as an educated NS Tailor language, counsel, motivate, persuade, negotiate Wide range of professional needs Extensive, precise, and appropriate Support opinions, hypothesize, explain, deal with unfamiliar topics Practical, abstract, special interests Narrate, describe, give directions Concrete, real- world, factual Intelligible even if not used to dealing with non-NS Errors never interfere with communication & rarely disturb Q & A, create with the languageEveryday survival Intelligible with effort or practice Use memorized phrasesRandom Unintelligible STANAG 6001 - Speaking (Summarized) as a Standard

248. Therefore, Benchmark Advisory Tests are Proficiency Tests STANAG 6001 is a proficiency scale. Tests derived from it assess unrehearsed, general language ability.

249. Of course, BAT development is but one of BILC’s current tasks. We’ll report on all of the Steering Committee agenda Items in our report on Thursday

250. The Steering Committee Report Benchmark Advisory Tests. Review of BILC Constitution and Rules of Procedure. JSSG tasking to create a third edition of STANAG 6001. Transition of BILC Secretariat responsibilities.

251. Thank you for attending this BILC Annual Conference! We look forward to working with you this week to better define and address the: “Language Implications of NATO’s Expanding Roles”.