2018 BILC Conference 20-24 May, Lisbon BILC Update by BILC Secretariat Old Lisbon Monuments Overview, a painting by E. Gancheva

Bureau for International Language Co-ordination NATO’s consultative and advisory body for language training and testing issues. Main responsibility: custodian of STANAG 6001 Community of language teaching/testing professionals from MoDs and defense-sponsored organizations. Mission: To promote and foster interoperability among NATO and Partner nations by furthering standardization of language training and testing. To support the Alliance's operations through the exchange of knowledge and best practices, IAW established procedures and agreements. Vision: To achieve levels of excellence where progress made by one is shared by all. Purpose: To foster cooperative professional support among BILC member nations and to extend support to NATO within the field of language training and education and language assessment. Active participation in BILC activities from 26 NATO nations and 15 Partner nations.

BILC Steering Committee Chair Col. (ret.) Trayko Stoykov GBR (1966-1981) DEU (1982-1996) USA (1997-2008) CAN (2008-2014) USA (2014-2016) BGR (2016 - … ) Senior Advisor Dr. Ray Clifford Secretary Emilija Nesheva Secretary Col. Petko Petkov Associate Secretaries Keith Wert Peggy Garza Julie Dubeau Jana Vasilj-Begovic

Major BILC Events BILC Annual Conference in May; BILC STANAG 6001 Testing Workshop in September; BILC Professional Seminar in October.

2017 BILC Conference Vienna, 14-19 May, 2017 Theme: Recognizing and Navigating the Specific Dynamics of Military Language Training Sub-topics: - Classroom behavior in the context of military culture - Navigating between mixed cultures in the classroom - Different profiles of language learners and their impact on learning success - Classroom culture - turning over-achievers into role-models - Fostering foreign culture acquisition - Accommodating students with special needs (i.e., learning disabilities, service related injuries, PTSD) - Enhancing student success in mastering higher proficiency levels - Pedagogy employed in building student autonomy - English for Specific Purposes in a military environment Study Groups Topics: - Activities for building cultural mastery blocks - Best practices and challenges in teaching security and military related terminology - Military assistance with the language problems related to the migrant crisis - Best practices in STANAG 6001 testing - Harmonizing the dynamics of military leadership and civilian faculty in language programs

2017 STANAG 6001 Testing Workshop Skopje, 5-7 September, 2017 SETTING THE STAGE FOR TESTER SUCCESS Sub-themes: - Selecting and training new testers - Determining job qualifications for new testers - Norming and re-norming testers to limit “drift” - Integrating new testers into test development teams - Improving testing team dynamics - Designing training for speaking test interviewers and raters - Training proctors and invigilators for standardized test administration - Defining and observing best practices in testing - Improving inter/intra-reliability - Familiarizing teachers and students with the testing policy/directive/test format - Managing stakeholders’ expectations - Developing and documenting professional development plans for testers  

2017 BILC Professional Seminar, Tbilisi, 02-05 October, 2017 PAVING THE ROAD TO THE STANDARDIZATION OF TRAINING AND TESTING: LESSONS-LEARNED Sub-themes: Using STANAG 6001 as the blueprint for language training/testing Making the leap from Level 2 to Level 3 Effective strategies for teaching at L3 Challenges of teaching the skill of writing Designing national tests to correlate with the BAT The benefits and challenges of standardization Setting and meeting the standards Is your Level 3 my Level 3? NATO jobs and language proficiency requirements Dealing with internal pressures to lower the standards

BILC Cooperative Visits (May 2017 – May 2018) Azerbaijan, September 2017 Jordan, October 2017

BILC’s Professional Development Programme in Partnership Cooperation Menu (PCM. i. e. ACT-sponsoured BILC courses) 1. Language Testing Seminars: Language Testing Seminar, ACT.647, 2 weeks; Advanced Language Testing Seminar, ACT.658, 3 weeks; Language Standards and Assessment, ACT.648, 2 weeks. 2. BILC Methodology Workshops - English Teaching Faculty Development Workshop , ACT.659 Teaching Speaking and Writing for Military Purposes, 2 weeks. Conducted at 3 PTECs PLTCE (twice per year, April, September), Bulgaria (July) and Slovenia (November). Facilitators: BILC teaching & testing experts from 24 nations. Kingdom, USA, etc.).

PLTCE sponsored courses IRT Workshop - introduction to IRT concepts and the advantages of using Rasch IRT analysis Applied Consecutive Interpretation Techniques (ACIT) Workshop focuses on the development of skills essential to the task of effective interpreting including: event preparation, memory and comprehension, note taking, and handling linguistic and ethical challenges assertively. Next iteration 21-30 August 2018. Application forms should be sent no later than 10 July 2018 to the PLTCE pltce.programs@marshallcenter.org

Two BILC Working Groups Introduction to BILC Two BILC Working Groups Level 4 Level 4 Working Group Portability and Recognition of STANAG 6001 certificates

New Benchmark Advisory Test (BAT) Initiative History of BAT – started in 2003 at the BILC Conference in Harrogate. Purpose: to provide an external measure against which nations can compare their national STANAG 6001 test results and to standardize testing across the nations. 8 nations collaborated on the BAT development. Contract awarded by NATO for the BAT delivery to ACTFL. In 2009, 11 NATO nations participated, with up to 20 tests each . Proposed a possible resurrection of the BAT after the 2016 BILC Conference in Riga Evidence for the Validity Roadmap (Brno, Sept 2016) Discussed by the WG on Portability and Recognition of STANAG 6001 Certificates (Bled, Sept 2016). Survey of interest by BILC Secretary (fall 2017) - 21 nations responded favorably. This year two BAT Writing&Speaking Forums will take place (9-13 July and 6-10 August 2018) at PLTCE, Garmisch Partenkirhen. Participants are encouraged to receive certification as BAT interlocuters/raters. The certification process will begin a month after the forums. BAT 2

BILC SharP

BILC site

BILC Update on Language Needs Analysis (LNA) Study at ACT BUREAU FOR INTERNATIONAL LANGUAGE COORDINATION BUREAU DE COORDINATION LINGUISTIQUE INTERNATIONALE BILC Update on Language Needs Analysis (LNA) Study at ACT Julie Dubeau Chief Standards, Foreign Languages & Associate BILC Secretary Canadian Defence Academy Julie.Dubeau@forces.gc.ca www.NATOBILC.org

Purpose of study: To examine concordance between SLPs in JDs and minimum language proficiency required to perform tasks Why? SLPs set without informed judgement and “Old” interpretation of STANAG 6001 SLPs are in job descriptions, and are prerequisites for training, Force Goals, Partnership Targets, tied to promotions, incentive pay, etc. Inflated requirements undermine the validity of the job descriptions themselves Language professionals and organizations pressured to meet inflated requirements Need to promote integrity of language standards reporting by nations 17

Participants & Findings 31 officers & 3 NCO interviewed + 26 supervisors Capt to Col, across 5 Directorates, 4 divisions, 14 sections 15 NATO nations + 3 Partner nations 80% of analyzed posts have SLP 3333 Data confirmed it is the correct SLP 14% of analyzed posts had SLP 4 in listening and reading BILC team was not provided with any document that reflects texts of level 4 difficulty 40% found the writing task most challenging Supervisors interviewed stressed the criticality of the skill STANAG 6001 Plus Levels

Conclusions & Recommendations SLP 3333 set appropriately at ACT Writing skill seems to be undertrained Level 4 not needed based on work samples BILC to continue standardization efforts Nations should assess how they train writing and adopt strategies for training to higher levels NATO to review L4 posts STANAG 6001 Plus Levels

Update: BILC Report submitted to BILC SC in May 2017 BILC Report submitted to ACT in June 2017 BILC invited to report findings at NATO HQ to NATO Defence Manpower Committee (NDMC) in November 2017 NDMC approved recommendation to review L4 requirements, nations agreed BILC is requested to provide guidelines for ongoing positional reviews, Feb 2018

Update: Related BILC efforts – Set up two study groups: SG # 1 Familiarization with STANAG 6001 for Non-Specialists Propose a prototype e-learning module building on the existing BILC documents that will include samples of language at each proficiency level and explanations; Explore ways to make the Familiarization with the STANAG 6001 for Non-Specialists module readily available as a practical resource for non-specialists SG # 4 Enhancing Teaching of the Writing Skill for NATO Staff Officer Outline a set of recommendations on how nations should approach the task of increasing students’ writing proficiency, and what strategies they could use to attain this goal

Way Ahead: More to follow… Outcome of Study Group #1 could inform guidelines and recommendations to NATO structures, on assigning appropriate SLPs; Manpower & Personnel Policy rep. to prepare an official tasker to all NATO Military bodies to review SLPs using guidelines that BILC will offer. More to follow…

Jana Begovic, FL Stds O – MPG Canada, and Associate BILC Secretary FAMILIARIZATION WORKSHOP ON LEVEL 4 READING PROFICIENCY Jana Begovic, FL Stds O – MPG Canada, and Associate BILC Secretary Lisbon, May 2018 STANAG 6001 Plus Levels

STANAG 6001 Plus Levels

Introduction and Background Formation of BILC Working Group on Level 4 Istanbul 2010 Aim: to assist nations feeling ill-equipped for testing at Level 4 to amplify STANAG 6001 Level 4 descriptors to discuss testing related implications STANAG 6001 L4 Reading Workshop STANAG 6001 Plus Levels

Working Group Members Jana Vasilj-Begovic – Leader (CAN) Gerard Seinhorst – Co-leader (NLD) Members: Petya Georgieva (BGR) Nancy Powers & May Tan (CAN) Kåre Kildevang & Allan Christiansen (DNK) Dugald Sturges (GER) Michael Adubato (SHAPE) Annette Nolan & Keith Farr (SWE) Peggy Garza, David Oglesby and Ray Clifford (USA) Special Consultants: Mary Jo Di Biase, Martha Herzog (USA) STANAG 6001 L4 Reading Workshop STANAG 6001 Plus Levels

Products Paper: “NATO STANAG 6001 Level 4 Language Proficiency – A Conceptual Model and Implications for Testing” (April 2013) Article “Defining and Assessing STANAG 6001 Level 4 Language Proficiency” (Chapter 11, Language in Uniform, Cambridge Scholars, 2015) Level 4 Reading Test Specifications Tutorial (principles of text rating, differences btw Level 3 and Level 4 texts, sample texts, sample test development procedure, etc.) Level 4 Test Prototype (Tester and Examinee Booklets for speaking/writing modality; administration and rating procedure) Test Familiarization Guide Feedback Questionnaire L4 Reading Proficiency Workshop STANAG 6001 L4 Reading Workshop STANAG 6001 Plus Levels

US MARSHALL CENTER-GARMISCH-PARTENKIRCHEN, GERMANY 23-27 October 2017 Participants from Austria, Bulgaria, Denmark, France, Latvia, Lithuania, Norway, Romania, Sweden and USA Facilitators from Canada, Denmark and USA STANAG 6001 L4 Reading Workshop STANAG 6001 Plus Levels

Workshop Aim/Objectives Aim: To familiarize participants with Level 4 reading proficiency, and in particular, with a reading test prototype   Objectives:   To discuss the nature of the reading skill and the challenges of its assessment at Level 4 To distinguish between Level 3 and Level 4 texts, descriptors, and test items IAW STANAG 6001 To discuss pros and cons of testing techniques and scoring at this level To practice developing Level 4 constructed-response items To rate Level 4 sample tests using a novel approach STANAG 6001 L4 Reading Workshop STANAG 6001 Plus Levels

Comments High expertise of both facilitators/participants Language enhancement environment Learning about a new rating procedure & hands-on rating Understanding differences between L3 and L4 texts Raised awareness of the L4 construct & its complexity Well-balanced activities Dispelling misconceptions about the nature of language Productive work with facilitators and participants STANAG 6001 Level 4 Reading Test STANAG 6001 Plus Levels

Suggestions for Improvement More discussion on how to write L4 items More rating sessions and item development activities Hands-on work (pair/group) on all days STANAG 6001 L4 Reading Workshop STANAG 6001 Plus Levels

NEXT SERIAL??? Prerequisite for attendance: ALTS/LTS graduates Full-time testers Need for Level 4 testing 8-10 participants Jana.Vasilj-Begovic@forces.gc.ca GarzaP@marshallcenter.org STANAG 6001 Plus Levels

Development Methodology Production Planning Process Global Programming It is based on management framework with THREE elements Global Programming Governance Structure Development Methodology Production Planning Process

Bi-SC List of Disciplines as of 2017 Domain   Function Capability Development Mission Execution Enhancement Land Operations Air Operations Maritime Operations Space Support to Operations Cyber Defence Special Operations Nuclear Operations Intelligence Meteorology and Oceanography Joint Targeting Military Policing Logistics Medical Support Operations Planning and Assessment Military Engineering Consultation, Command and Control Education, Training, Exercises & Evaluation (ETEE) Financial Civil Military Cooperation and Civil Military Interaction Counter-Improvised Explosive Devices Weapons of Mass Destruction/ CBRN Defence Strategic Communications Ballistic Missile Defence Federated Mission Networking Building Integrity Counter-Terrorism Energy Security Mil Contribution to Peace Support Gender in Military Operations Ʃ = 7 Ʃ = 15 Ʃ = 2 Ʃ = 5 Air Operations Air Ops Ballistic Missile Defence BMD Building Integrity BI Civil Military Interaction and Civil - Military Cooperation CMI & CIMIC Consultation,Command and Control Comms and Info Int Countering Improvised Explosive Devices C-IED Counter-Terrorism CT Cyber Defence Operations Cyber Def Ops Education, Training, Exercises and Evaluation ETEE Energy Security ENSEC Federated Mission Networking FMN Financial FIN Gender in Military Operations Gender Intelligence Intel Joint Targeting Joint Targeting Land Operations Land Ops Logistics, Support and Resources Log Maritime Operations Mar Ops Medical Support Med Sup Meteorology and Oceanography METOC Military Contribution to Peace Support MC2PS Military Engineering MILENG Military Police MP Space Support to Operations Space Sup Nucleair Operations Nuc Ops Operational Planning and Assessment Ops Plng & Assess Special Operations SO Strategic Communications STRATCOM Weapons of Mass Destruction / Chemical Biological Radioactive Nuclear WMD / CBRN Candidates for 2018 list: Human Resources, Legal, JPR,M&S Total = 29 ETEE includes a mixed bag of topics. One is “Languages”.

Education, Training, Exercises & Evaluation (ETEE) ETEE Department Head (DH) is the NATO School The Training Requirements Analysis (TRA) workshop for “Languages” was held at the NATO School in December 2017 Participants involved were representatives from 3 PTEC* language schools Partner Language Training Center Europe (PLTCE) (US), Garmisch, Gemany Foreign Languages Department (FLD) Shumen, Bulgaria School of Foreign Languages, Poljce, Slovenia * NATO-recognized Partnership Training and Education Centres Also invited: Phil Turner, HQ NATO/IMS and Mike Adubato, NATO/ACO SHAPE

NATO considers “Language Training” a national responsibility

TRA – DAY 1 - Determine performance statements and level for training audiences TRA – DAY 2 – Use course listings from ETOC to match existing training solutions to requirements (performance statements) TRA DAY 3 – Final product - training solutions matched to requirements. Green = requirement fully met / Yellow = requirement partially met / Red = requirement not met

Excerpt from TRA Report Findings *(para 8b - Languages) Leadership at all levels are required to have awareness of the STANAG 6001, Language Proficiency Levels. For Language appointed SMEs in the NCS, participation in the BILC Annual Conference, the BILC Annual Professional Seminar, and the STANAG 6001 Testing Workshop is strongly encouraged as part of their professional development programme. Education in English communication is assumed a national requirement. For further professional development in English communication, several HQs offer additional internal education and training, e.g. HQ SACT’s Strategies for Writing Course.

Conclusions The Language Testing Seminar, the Advanced Language Testing Seminar, the Language Standards & Assessment Seminar and the English Teaching Faculty Development Workshop meet NATO training requirements. Training gaps were identified in these areas: STANAG 6001 familiarization for staff officers Writing IAW NATO conventions