by Emilija Nesheva, BILC Secretary STANAG 6001 Testing Workshop 4-6 September 2018 Kranjska Gora, Slovenia BILC Update Sept 2017 – Sept 2018 by Emilija Nesheva, BILC Secretary

Bureau for International Language Co-ordination NATO’s consultative and advisory body for language training and testing issues. Main responsibility: custodian of STANAG 6001 Community of language teaching/testing professionals from MoDs and defense-sponsored organizations. Mission: To promote and foster interoperability among NATO and Partner nations by furthering standardization of language training and testing. To support the Alliance's operations through the exchange of knowledge and best practices, IAW established procedures and agreements. Vision: To achieve levels of excellence where progress made by one is shared by all. Purpose: To foster cooperative professional support among BILC member nations and to extend support to NATO within the field of language training and education and language assessment. Active participation in BILC activities from 26 NATO nations and 15 Partner nations. The point of BILC ultimately is INTEROPERABILITY. Our Mission is an important statement because there are plenty of civilian / academic language training and testing professional associations. But the BILC world is the defense-sponsored world and accordingly has some special aspects such as military interoperability, military terms, large numbers of people to train and at frequently higher levels than the purely civilian world of language teaching. BILC is a unique language organization as a result. The BILC vision is to share progress made with member nations, partner nations and any nation who wishes to advance in the field of language training for military purposes. How is this done? Through the secretariat, there are a variety of initiatives that aim to foster these exchanges and promote standardization. Established in 1966 as an advisory body to NATO. Founding members are France, Germany, Italy, the United Kingdom, and the United States. 1967: Belgium, Canada, Netherlands 1975: SHAPE and IMS/NATO (non-voting members) 1978: Portugal 1983: Turkey 1984: Denmark and Greece 1985: Spain 1993: Norway 1999: Czech Republic, Hungary, Poland 2004: Bulgaria, Estonia, Latvia, Lithuania, Romania, Slovakia, Slovenia 2008: Albania, Croatia The major responsibility of BILC is to be the custodian of the STANAG 6001. This STANAG is an important responsibility because it is the guidance underpinning the serious subject of Standardized Language Profiles (SLPs) which are found in many basic personnel requirements not to mention national directives and ministerial orders related to military personnel. I am not going to review the STANAG 6001. It can be found on the BILC web site. The current edition of STANAG 6001 is Edition 5, version 2 .

BILC Steering Committee Chair Col. (ret.) Trayko Stoykov GBR (1966-1981) DEU (1982-1996) USA (1997-2008) CAN (2008-2014) USA (2014-2016) BGR (2016-2019) Senior Advisor Dr. Ray Clifford Secretary Emilija Nesheva Secretary Col. Petko Petkov This is our formal structure. Remember this is not a permanent staff element. To the left you can see the countries which have provided the secretariat over the years. The Presidency of the Secretariat was handed to Bulgaria at the end of the 2016 BILC conference in Riga and that was a great honor for our country. Currently the Secretariat consists of 8 members: BILC Chair, Col. (ret.) Trayko Stoykov, the Senior advisor Dr. Ray Clifford, two secretaries, Col. Petko Petkov and me, and the four associate secretaries, Mr. Keith Wert and Ms. Peggy Garza from the USA and Ms. Julie Dubeau and Jana Vasilj-Begovic form Canada. The BILC Chair presides over the Steering Committee which meets each year at the annual conference. Voting members are NATO national representatives. Partner nations are welcome to attend and observe at Steering Committee meetings but do not have the right to vote. Associate Secretaries Keith Wert Peggy Garza Julie Dubeau Jana Vasilj-Begovic

Major BILC Events BILC Annual Conference in May; BILC STANAG 6001 Testing Workshop in September; BILC Professional Seminar in October. There are 3 major BILC events: conference, workshop and seminar. Who attends BILC events? There is a great diversity of people who attend the BILC events. For example at the conference we meet people ranking from five-stars generals to lieutenants and civilians working for the militaries. This gives us the opportunity to scrutinize the problems of the foreign language training and testing from different aspects. But not only this – we can also implement the advice and absorb the knowledge and experience of all these people. Studying the need for decent English language competence for the NATO member and partner nations from various angles, allows us to have opinions from political, strategical, managerial and practical points of view. Starting from analyzing the needs and tasking by the highest authorities in NATO who are concerned about the interoperability of the NATO forces and the fulfillment of task goal E 1101 N and Partnership goals to going down the chain of command to the managers of education and training and then to the practitioners: the classroom teachers and STANAG 6001 testers who perform the real work with the personnel.

2017 STANAG 6001 Testing Workshop Skopje, 5-7 September, 2017 SETTING THE STAGE FOR TESTER SUCCESS Sub-themes: - Selecting and training new testers - Determining job qualifications for new testers - Norming and re-norming testers to limit “drift” - Integrating new testers into test development teams - Improving testing team dynamics - Designing training for speaking test interviewers and raters - Training proctors and invigilators for standardized test administration - Defining and observing best practices in testing - Improving inter/intra-reliability - Familiarizing teachers and students with the testing policy/directive/test format - Managing stakeholders’ expectations - Developing and documenting professional development plans for testers  

2017 BILC Professional Seminar, Tbilisi, 02-05 October, 2017 PAVING THE ROAD TO THE STANDARDIZATION OF TRAINING AND TESTING: LESSONS-LEARNED Sub-themes: Using STANAG 6001 as the blueprint for language training/testing Making the leap from Level 2 to Level 3 Effective strategies for teaching at L3 Challenges of teaching the skill of writing Designing national tests to correlate with the BAT The benefits and challenges of standardization Setting and meeting the standards Is your Level 3 my Level 3? NATO jobs and language proficiency requirements Dealing with internal pressures to lower the standards

2018 BILC Conference Lisbon, 20-24 May 2018 Theme: “Invigorating Training through Technological Tools” Sub-topics: - Measuring success rates before and after the use of technology - The science behind language learning - Using technology to modernize testing - Technology as a tool of teacher/student empowerment - Does technology provide us with shortcuts to language learning success? - Gaming as a learning tool - Enhancing teacher training to match current expectations - Effective language maintenance tools - Social media and project-oriented, computer assisted language learning - Incorporating translation technology in the classroom Study Groups Topics: - Familiarization with STANAG 6001 for Non-Specialists - STANAG 6001 Handbook for Curriculum Development - Revision of the Tri-Service Word and Terminology List - Enhancing Teaching of the Writing Skill for NATO Staff Officers - Distance Learning Enhanced by Technology The largest BILC event so far – 101 participants from 36 countries from 4 continents and 3 NATO bodies – ACT, IMS and SHAPE!

BILC Cooperative Visits (September 2017 – August 2018) Azerbaijan, September 2017 Jordan, October 2017 BILC conducts non-resident activities wherein BILC teams of experts work with countries on different aspects of their military language training. When BILC Secretariat receives a letter from a country requesting our help, we organize a visit of experts to this country. Teams of experts from different countries voluntarily contribute on many aspects of military language training and testing programs during these visits and collaborations. Funding comes from NATO through various channels.

Cooperation between BILC and Defence Education Enhancement Programme (DEEP) BILC actively supports DEEP activities when asked. Some of the BILC visits have been funded by DEEP. In September there will be such a visit on request by Mauritania and sponsoured by DEEP. The Defence Education Enhancement Programme (DEEP) is providing tailored practical support to individual countries in developing and reforming their professional military education institutions. Expert advice is offered to defence education institutions seeking to become intellectually interoperable with the Alliance. Currently, active tailored DEEP programmes are ongoing in Afghanistan, Armenia, Azerbaijan, Georgia, Kazakhstan, Kyrgyzstan, Mauritania, the Republic of Moldova, Serbia, the former Yugoslav Republic of Macedonia¹, Tunisia and Ukraine. Croatia and Mongolia have completed their programmes and a programme in Iraq is on hold due to the current security situation in country.

BILC’s Professional Development Programme in Partnership Cooperation Menu (PCM. i. e. ACT-sponsoured BILC courses) 1. Language Testing Seminars (conducted at PLTCE): Language Testing Seminar, ACT.647, 2 weeks; Advanced Language Testing Seminar, ACT.658, 3 weeks; Language Standards and Assessment, ACT.648, 2 weeks. 2. BILC Methodology Workshops English Teaching Faculty Development Workshop “Teaching Speaking and Writing for Military Purposes”, ACT.659, 2 weeks. Conducted at 3 PTECs PLTCE, Bulgaria and Slovenia. Facilitators: BILC teaching & testing experts from 24 nations. The seminars BILC offers in conjunction with ACT support have been focused on two objectives: To make STANAG 6001 language tests in each NATO member and partner country comparable to each other; To let the English language training managers, curriculum developers and classroom teachers get acquainted with the principles of assessment IAW STANAG 6001; To offer professional development for the English language teachers who teach military students and subjects

PLTCE sponsored courses IRT Workshop - introduction to IRT concepts and the advantages of using Rasch IRT analysis Applied Consecutive Interpretation Techniques (ACIT) Workshop focuses on the development of skills essential to the task of effective interpreting including: event preparation, memory and comprehension, note taking, and handling linguistic and ethical challenges assertively.

Recent BILC Working Groups’ activities WG Military Terminology and Translation formed in 2011 Level 4 WG formed in 2010 - Paper: “NATO STANAG 6001 Level 4 Language Proficiency – A Conceptual Model and Implications for Testing” (April 2013) - Article “Defining and Assessing STANAG 6001 Level 4 Language Proficiency” (Chapter 10, Language in Uniform, Cambridge Scholars, 2015) - Level 4 Reading Test Specifications Tutorial (principles of text rating, differences btw Level 3 and Level 4 texts, sample texts, sample test development procedure, etc.) Level 4 Test Prototype (Tester and Examinee Booklets for speaking/writing modality; administration and rating procedure) - Test Familiarization Guide Feedback Questionnaire L4 Reading Familiarization Workshop This list was compiled by the members of Study Group that met during the BILC annual conference in Tbilisi, Georgia in May 2013. Under the leadership of Austria their work was completed through online correspondence.

Recent BILC Working Groups’ activities Portability and Recognition of STANAG 6001 Certificates WG formed in 2016 Familiarization with STANAG 6001 for Non-Specialists WG formed in 2018 - Improving recognition of STANAG 6001 certificates both within the militaries as well as by civilian institutions - Exploring more standardization of STANAG 6001 certificates across nations - Rendering to non-specialists the complexities of comparing the STANAG 6001 criterion referenced tests to other types of language tests - Four BILC policy recommendation documents developed and published on the BILC website: BILC Policy Recommendation on Essential Information for STANAG 6001 Test Certificates BILC Policy Recommendation on the Acceptance of Commercial STANAG 6001 Certificates BILC Policy Recommendation on the Longevity of STANAG 6001 Language Certificates BILC Policy Recommendation on the Portability of STANAG 6001 Language Certifications Non-language specialists in NATO staff positions should be familiar with STANAG 6001 language proficiency levels in order to make informed decisions about SLPs and other matters relating to languages such as establishing policy on linguistic interoperability, determining capability targets for languages, and using English as an operational language within NATO, etc. While the STANAG 6001 Overview for Non-Specialists and the full STANAG 6001 descriptors can be used as in promoting a common understanding and interpretation of the descriptors, it has become evident that non-specialists making these decisions remain uninformed. The goals of the WG are to: propose a prototype e-learning module building on the existing BILC documents that will include samples of language at each proficiency level and explanations; explore ways to make the Familiarization with the STANAG 6001 for Non-Specialists module readily available as a practical resource for non-specialists.

Benchmark Advisory Test (BAT) Initiative History of BAT – started in 2003 at the BILC Conference in Harrogate. Purpose: to provide an external measure against which nations can compare their national STANAG 6001 test results and to standardize testing across the nations. 8 nations collaborated on the BAT development. Contract awarded by NATO for the BAT delivery to ACTFL. In 2009, 11 NATO nations participated, with up to 20 tests each . Proposed a possible resurrection of the BAT after the 2016 BILC Conference in Riga - Evidence for the Validity Roadmap (Brno, Sept 2016); - Discussed by the WG on Portability and Recognition of STANAG 6001 Certificates (Bled, Sept 2016); - Survey of interest by BILC Secretary (fall 2017) - 21 nations responded favorably. - This year two BAT Writing&Speaking Forums took place (9-13 July and 6-10 August 2018) at PLTCE, Garmisch Partenkirhen. Participants are encouraged to receive certification as BAT interlocutors/raters. - The certification process will begin a month after the forums. - BAT 2 First a few words for the history of BAT. The idea to create a Benchmark Advisory Test (BAT) was born in 2003 at the BILC Conference in Harrogate, UK after a delegate indicated that the NATO STANAG 6001 interpretations seemed to vary significantly, and that it appeared that PfP nations were expected to adhere to the standard more stringently than some older NATO nations. 13 testing experts from 8 nations participated in the development of the items for the test. To accelerate the development process and to expand the number of skills tested, in December 2006, NATO ACT awarded a contract to the American Council for the Teaching of Foreign Languages (ACTFL). The benchmarking process commenced in the late spring of 2009. The BAT project is a significant step toward language testing standardization and enhancement of interoperability. This year BILC plans to subsidize 10 candidates per nation to sit for the BAT. About 21 nations have expressed interest their candidates to be tested. This summer two BAT Writing&Speaking forums took place at PLTCE and 20 experienced testers from 20 countries took part in them. After the certification of some of them as BAT interlocutors/raters we will expect the second iteration of BAT.

BILC SharP And the last BILC initiative that I will tell you about is the platform for sharing teaching and testing materials BILC SharP which was designed, developed and administered by our Australian colleagues. On the slide, as well as on the BILC site you may find more details how to use it. In order to have access to the platform you have to contact your national POC and he will write to the Australian site administrator to get a password for you.

BILC site

Questions? Contacts: BILC site https://natobilc.org Email chairman@natobilc.org My office telephone +359 2 92 21035 My service mobile phone +359 889 803 732