Tbilisi, 2-5 Oct. 2017 BILC Professional Seminar Paving the Road to the Standardization of Training and Testing: Lessons Learned BILC Update BILC Secretariat

Bureau for International Language Co-ordination Mission: To promote and foster interoperability among NATO and Partner nations by furthering standardization of language training and testing. To support the Alliance's operations through the exchange of knowledge and best practices, IAW established procedures and agreements. Vision: To achieve levels of excellence where progress made by one is shared by all. The point of BILC ultimately is INTEROPERABILITY. Our Mission is an important statement because there are plenty of civilian / academic language training and testing professional associations. But the BILC world is the defense-sponsored world and accordingly has some special aspects such as military interoperability, military terms, large numbers of people to train and at frequently higher levels than the purely civilian world of language teaching. BILC is a unique language organization as a result. The BILC vision is to share progress made with member nations, partner nations and any nation who wishes to advance in the field of language training for military purposes. How is this done? Through the secretariat, there are a variety of initiatives that aim to foster these exchanges and promote standardization.

Bureau for International Language Co-ordination NATO’s consultative and advisory body for language training and testing issues Community of language teaching/testing professionals from MoDs and defense-sponsored organizations Purpose: To foster cooperative professional support among BILC member nations and to extend support to NATO within the field of language training and education and language assessment. Active members from 23 NATO nations and 10+ Partner nations10+ Partner nations Established in 1966 as an advisory body to NATO. Founding members are France, Germany, Italy, the United Kingdom, and the United States. 1967: Belgium, Canada, Netherlands 1975: SHAPE and IMS/NATO (non-voting members) 1978: Portugal 1983: Turkey 1984: Denmark and Greece 1985: Spain 1993: Norway 1999: Czech Republic, Hungary, Poland 2004: Bulgaria, Estonia, Latvia, Lithuania, Romania, Slovakia, Slovenia 2008: Albania, Croatia

BILC Steering Committee Chair Col. (ret.) Trayko Stoykov GBR (1966-1981) DEU (1982-1996) USA (1997-2008) CAN (2008-2014) USA (2014-2016) BGR (26 May 2016 - … ) Senior Advisor Dr. Ray Clifford Secretary Emilia Nesheva Secretary Col. Petko Petkov This is our formal structure. Remember this is not a permanent staff element. To the left you can see the countries which have provided the secretariat over the years. The Presidency of the Secretariat was handed to Bulgaria at the end of the 2016 BILC conference in Riga and that was a great honor for our country. Currently the Secretariat consists of 8 members: BILC Chair, Col. (ret.) Trayko Stoykov, the Senior advisor Dr. Ray Clifford, two secretaries, Col. Petko Petkov and me, and the four associate secretaries, Mr. Keith Wert and Ms. Peggy Garza from the USA and Ms. Julie Dubeau and Jana Vasilj-Begovic form Canada. The BILC Chair presides over the Steering Committee which meets each year at the annual conference. Voting members are NATO national representatives. Partner nations are welcome to attend and observe at Steering Committee meetings. Associate Secretaries Keith Wert Peggy Garza Julie Dubeau Jana Vasilj-Begovic

BILC Activities Language proficiency descriptions developed by BILC BILC is custodian for STANAG 6001 SLPs in job descriptions, prerequisites for training, Force Goals, Partnership Targets Language Needs Analyses “ACT is requested to conduct an assessment of the language requirements for collective groups, NATO posts, along with the review of Job Descriptions” SACEUR SAGE 2014 BILC Team cooperating with NATO ACT/JFT, IMS, ACO/SHAPE reviewed position descriptions against actual critical language skills required. This is a short summary of BILC activities. The major responsibility of BILC is to be the custodian of the STANAG 6001. This STANAG is an important responsibility because it is the guidance underpinning the serious subject of Standardized Language Profiles (SLPs) which are found in many basic personnel requirements not to mention national directives and ministerial orders related to military personnel. I am not going to review the STANAG 6001. It can be found on the BILC web site. The current edition of STANAG 6001 is Edition 5, version 2 . The Language Needs Analysis first part of the study was conducted in March 2015 by the BILC Associate Secretaries (BILC), J. Dubeau and J. Vasilj-Begovic. The preliminary findings confirmed that level 3 was indeed at the required level, and that although some officers were at level 3, most officers did not possess that level of proficiency in all skills, despite the confirmation of level 3 SLPs on national certificates. Even though the original plan had been to conduct the study across the three HQs, in the end, it was only conducted at HQ SACT. Another reason motivating this change of attitude at IMS seemed to have been linked to the Level 4 reading test that BILC piloted at IMS. The test seems to have accentuated the difference between what NATO HQ understood as Level 4 proficiency and the BILC-agreed interpretation. Based on the IMS’s reaction to the pilot reading test, it seems Level 4 is not needed, but solid or high Level 3, possibly 3+ across some or all of the skills, definitely is needed. The IMS said that since they had already adjusted the SLPs by lowering the number of positions requiring Level 4, they were no longer interested in participating in a LNA. The final report of the LNA can be found on the BILC site.

Two ongoing BILC Working Groups Introduction to BILC Two ongoing BILC Working Groups Portability and Recognition of STANAG 6001 certificates Level 4 Working Group - Paper: “NATO STANAG 6001 Level 4 Language Proficiency – A Conceptual Model and Implications for Testing” (April 2013) - Article “Defining and Assessing STANAG 6001 Level 4 Language Proficiency” (Chapter 10, Language in Uniform, Cambridge Scholars, 2015) - Level 4 Reading Test Specifications - Tutorial (principles of text rating, differences btw Level 3 and Level 4 texts, sample texts, sample test development procedure, etc.) - Level 4 Test Prototype (Tester and Examinee Booklets for speaking/writing modality; administration and rating procedure) - Test Familiarization Guide Feedback Questionnaire L4 Reading Familiarization Workshop - Improving recognition of STANAG 6001 certificates both within the militaries as well as by civilian institutions - Exploring more standardization of STANAG 6001 certificates across nations - Rendering to non-specialists the complexities of comparing the STANAG 6001 criterion referenced tests to other types of language tests - Three BILC policy recommendations on the BILC website - - BILC Policy Recommendation on Acceptance of Commercial STANAG 6001 certificates; - BILC Policy Recommendation on Longevity of STANAG 6001 Language Certificates - BILC Policy Recommendation on STANAG Certificate Information - New: BILC Language Policy Considerations on the Portability of STANAG 6001 Language Certifications L4 WG was formed at the 2010 Conference in Istanbul due to the fact that a number of nations had to develop and administer L4 tests but had no profound methodological guidance and help how to do these. You ca sees on the slide that the L4WG has been extremely productive. Read the products. The Portability and Recognition WG was formed after the 2016 Conference in Riga. It was provoked by one of the task set to BILC by the ACT, Joined Force Trainer. Read the slide.

New Benchmark Advisory Test (BAT) Initiative Proposed a possible resurrection of the BAT Evidence for the Validity Roadmap (Brno, Sept 2017) Discussed by the WG on Portability and Recognition of STANAG 6001 Certificates (Bled, Sept 2017) Survey of interest by BILC Secretary (fall 2017) 21 nations responded favorably History of BAT Purpose: to provide an external measure against which nations can compare their national STANAG 6001 test results and to standardize testing across the nations 8 nations collaborated on the BAT development Contract awarded by NATO for the BAT delivery 11 NATO nations participated, up to 20 tests each First a few words for the history of BAT. The idea to create a Benchmark Advisory Test (BAT) was born in 2003 at the BILC Conference in Harrogate, UK after a delegate indicated that the NATO STANAG 6001 interpretations seemed to vary significantly, and that it appeared that PfP nations were expected to adhere to the standard more stringently than some older NATO nations. 13 testing experts from 8 nations participated in the development of the items for the test. To accelerate the development process and to expand the number of skills tested, in December 2006, NATO ACT awarded a contract to the American Council for the Teaching of Foreign Languages (ACTFL). The benchmarking process commenced in the late spring of 2009. The BAT project is a significant step toward language testing standardization and enhancement of interoperability. This year BILC plans to subsidize 10 candidates per nation to sit for the BAT. About 21 nations have expressed interest their candidates to be tested.

BILC Cooperative Visits (October 2016- Oct 2017) Azerbaijan, October 2016 Georgia, October 2016 Ukraine, November 2016 Tunisia, March 2017 BILC conducts non-resident activities wherein BILC teams of experts work with countries on different aspects of their military language training. When BILC Secretariat receives a letter from a country requesting our help, we organize a visit of experts to this country. BILC organizes teams of experts from different countries who voluntarily contribute on many aspects of military language training and testing programs during the BILC visits and collaborations. Funding comes from NATO through various channels. In two weeks we will visit Jordan

Major BILC Events BILC Annual Conference in May; BILC STANAG 6001 Testing Workshop in September; BILC Professional Seminar in October.

Who attends BILC events? Ministry level policy people Commandants School Directors Directors of Studies Department Chairs Testing experts Curriculum developers Classroom teachers As you can see on this slide, there is a great diversity of people who attend the BILC events. For example at the conference we meet people ranking from five-stars generals to lieutenants and civilians working for the militaries. This gives us the opportunity to scrutinize the problems of the foreign language training and testing from different aspects. But not only this – we can also implement the advice and absorb the knowledge and experience of all these people. Studying the need for decent English language competence for the NATO member and partner nations from various angles, allows us to have opinions from political, strategical, managerial and practical point of view. Starting from analyzing the needs and tasking by the highest authorities in NATO who are concerned about the interoperability of the NATO forces and the fulfillment of task goal E 1101 N to going down the chain of command to the managers of education and training and then to the practitioners: the classroom teachers and STANAG 6001 testers who perform the real work with the personnel.

2016 BILC Professional Seminar, Budapest, Hungary, 24-27 October Theme: Authenticity in training and testing: making it real Sub-topics Linking training and testing to real-world target language use Traditional textbooks versus authentic materials Co-operating/partnering with local universities Attaining concordance between linguistic objectives and training outcomes Benefitting from tele-collaboration between training establishments Using social media in training Teaching and learning as two sides of the same coin Language maintenance Workshops Designing L2 reading to write tasks for Student Military Staff Officers Education programmes for Communicatively and Culturally Literate Military Personnel Using Social Media Effectively for Language Students

2017 BILC Conference Vienna, Austria, 14-19 May Theme: Recognizing and Navigating the Specific Dynamics of Military Language Training Sub-topics: - Classroom behavior in the context of military culture - Navigating between mixed cultures in the classroom - Different profiles of language learners and their impact on learning success - Classroom culture - turning over-achievers into role-models - Fostering foreign culture acquisition - Accommodating students with special needs (i.e., learning disabilities, service related injuries, PTSD) - Enhancing student success in mastering higher proficiency levels - Pedagogy employed in building student autonomy - English for Specific Purposes in a military environment Study Groups Topics: - Activities for building cultural mastery blocks - Best practices and challenges in teaching security and military related terminology - Military assistance with the language problems related to the migrant crisis - Best practices in STANAG 6001 testing - Harmonizing the dynamics of military leadership and civilian faculty in language programs

2017 STANAG 6001 Testing Workshop Skopje, 5-7 September SETTING THE STAGE FOR TESTER SUCCESS Sub-themes: - Selecting and training new testers - Determining job qualifications for new testers - Norming and re-norming testers to limit “drift” - Integrating new testers into test development teams - Improving testing team dynamics - Designing training for speaking test interviewers and raters - Training proctors and invigilators for standardized test administration - Defining and observing best practices in testing - Improving inter/intra-reliability - Familiarizing teachers and students with the testing policy/directive/test format - Managing stakeholders’ expectations - Developing and documenting professional development plans for testers  

BILC’s Professional Development Programme in Partnership Cooperation Menu (PCM. i. e. ACT-sponsoured BILC courses) Language Testing Seminars: Language Testing Seminar, ACT.647, 2 weeks; Advanced Language Testing Seminar, ACT.658, 3 weeks; Language Standards and Assessment, ACT.648, 2 weeks The seminars we offer in conjunction with ACT support have been focused on making STANAG 6001 language tests in each NATO and partner country comparable to each other. Meaning that one country’s Level 3 is the same as another country’s Level 3. This is easier said than done, but nevertheless, the Language Testing Seminars and the Advanced Language Testing Seminar are high quality, graduate level professional seminars that try to make language interoperability a reality. The Language Standards and Assessment Seminar is targeted at curriculum developers and classroom teachers who are not involved in building complicated language tests but who still need to understand modern language assessment in regard to STANAG 6001.

IRT Workshop 11 nations attended Objectives Is your nation interested? Introduction to IRT concepts and the advantages of using Rasch IRT analysis Using jMetrik for Rasch IRT analysis Interpretation of the data for informed decisions about tests and items Individualized consultation on national test datasets Is your nation interested? ALTS graduate Held for the first time this summer. Meant for testers.

BILC’s Professional Development Programme in Partnership Cooperation Menu (PCM. i. e. ACT-sponsoured BILC courses) continued BILC Methodology Workshops - English Teaching Faculty Development Workshops, ACT.659 Teaching Speaking and Writing for Military Purposes, 2 weeks Conducted at 3 PTECs Bulgaria Slovenia PLTCE Facilitators: BILC teaching & testing experts from 24 nations (Bosnia- Herzegovina, Bulgaria, Canada, Croatia, Denmark, Estonia, Germany, Hungary, Italy, Lithuania, Netherlands, Poland, Romania, Serbia, Slovakia, Slovenia, United Kingdom, USA, etc.). For years we have been aware of a lack of professional development opportunities for language faculty in military language schools time. For them we offer the Faculty Development Workshop which focuses on the needs of the teachers who teach military students and subjects. This is important because in almost all of our countries, our military language school faculty members are civilian professionals who must transition to military language. For them we are also preparing a two-week course for interpretation techniques which has already been piloted.

BILC SharP And the last BILC initiative that I will tell you about is the platform for sharing teaching and testing materials BILC SharP which was designed, developed and administered by our Australian colleagues. On the slide, as well as on the BILC site you may find more details how to use it.

Successful work during the seminar!!!!

BILC site From 20+ year old HTTP to HTTPS Confidentiality Integrity Identity Google will not index old protocol

BILC site

Questions? BILC site: www.natobilc.org BILC e-mail address: chairman@natobilc.org