1. 2018 BILC Conference 20-24 May, Lisbon BILC Update by BILC Secretariat
Old Lisbon Monuments Overview, a painting by E. Gancheva

2. Bureau for International Language Co-ordination
NATO’s consultative and advisory body for language training and testing issues.
Main responsibility: custodian of STANAG 6001
Community of language teaching/testing professionals from MoDs and defense-sponsored organizations.
Mission: To promote and foster interoperability among NATO and Partner nations by furthering standardization of language training and testing. To support the Alliance's operations through the exchange of knowledge and best practices, IAW established procedures and agreements.
Vision: To achieve levels of excellence where progress made by one is shared by all.
Purpose: To foster cooperative professional support among BILC member nations and to extend support to NATO within the field of language training and education and language assessment.
Active participation in BILC activities from 26 NATO nations and 15 Partner nations.

3. BILC Steering Committee
Chair
Col. (ret.) Trayko Stoykov
GBR ( )
DEU ( )
USA ( )
CAN ( )
USA ( )
BGR ( … )
Senior Advisor
Dr. Ray Clifford
Secretary
Emilija Nesheva
Secretary
Col. Petko Petkov
Associate Secretaries
Keith Wert
Peggy Garza
Julie Dubeau
Jana Vasilj-Begovic

4. Major BILC Events BILC Annual Conference in May;
BILC STANAG 6001 Testing Workshop in September;
BILC Professional Seminar in October.

5. 2017 BILC Conference Vienna, 14-19 May, 2017
Theme: Recognizing and Navigating the Specific Dynamics of Military Language Training
Sub-topics:
- Classroom behavior in the context of military culture
- Navigating between mixed cultures in the classroom
- Different profiles of language learners and their impact on learning success
- Classroom culture - turning over-achievers into role-models
- Fostering foreign culture acquisition
- Accommodating students with special needs (i.e., learning disabilities, service related injuries, PTSD)
- Enhancing student success in mastering higher proficiency levels
- Pedagogy employed in building student autonomy
- English for Specific Purposes in a military environment
Study Groups Topics:
- Activities for building cultural mastery blocks
- Best practices and challenges in teaching security and
military related terminology
- Military assistance with the language problems related to
the migrant crisis
- Best practices in STANAG 6001 testing
- Harmonizing the dynamics of military leadership and
civilian faculty in language programs

6. 2017 STANAG 6001 Testing Workshop Skopje, 5-7 September, 2017
SETTING THE STAGE FOR TESTER SUCCESS
Sub-themes:
- Selecting and training new testers
- Determining job qualifications for new testers
- Norming and re-norming testers to limit “drift”
- Integrating new testers into test development teams
- Improving testing team dynamics
- Designing training for speaking test interviewers and raters
- Training proctors and invigilators for standardized test administration
- Defining and observing best practices in testing
- Improving inter/intra-reliability
- Familiarizing teachers and students with the testing
policy/directive/test format
- Managing stakeholders’ expectations
- Developing and documenting professional
development plans for testers

7. 2017 BILC Professional Seminar, Tbilisi, 02-05 October, 2017
PAVING THE ROAD TO THE STANDARDIZATION OF TRAINING AND TESTING: LESSONS-LEARNED Sub-themes:
Using STANAG 6001 as the blueprint for language training/testing
Making the leap from Level 2 to Level 3
Effective strategies for teaching at L3
Challenges of teaching the skill of writing
Designing national tests to correlate with the BAT
The benefits and challenges of standardization
Setting and meeting the standards
Is your Level 3 my Level 3?
NATO jobs and language proficiency requirements
Dealing with internal pressures to lower the standards

8. BILC Cooperative Visits (May 2017 – May 2018)
Azerbaijan, September 2017
Jordan, October 2017

9. BILC’s Professional Development Programme in Partnership Cooperation Menu (PCM. i. e. ACT-sponsoured BILC courses)
1. Language Testing Seminars:
Language Testing Seminar, ACT.647, 2 weeks;
Advanced Language Testing Seminar, ACT.658, 3 weeks;
Language Standards and Assessment, ACT.648, 2 weeks.
2. BILC Methodology Workshops
- English Teaching Faculty Development Workshop , ACT.659
Teaching Speaking and Writing for Military Purposes, 2 weeks. Conducted at 3 PTECs PLTCE (twice per year, April, September), Bulgaria (July) and Slovenia (November).
Facilitators: BILC teaching & testing experts from 24 nations. Kingdom, USA, etc.).

10. PLTCE sponsored courses
IRT Workshop - introduction to IRT concepts and the advantages of using Rasch IRT analysis
Applied Consecutive Interpretation Techniques (ACIT) Workshop focuses on the development of skills essential to the task of effective interpreting including: event preparation, memory and comprehension, note taking, and handling linguistic and ethical challenges assertively. Next iteration August Application forms should be sent no later than 10 July 2018 to the PLTCE

11. Two BILC Working Groups
Introduction to BILC
Two BILC Working Groups
Level 4
Level 4 Working Group
Portability and Recognition of STANAG 6001 certificates

12. New Benchmark Advisory Test (BAT) Initiative
History of BAT – started in 2003 at the BILC Conference in Harrogate.
Purpose: to provide an external measure against which nations can compare their national STANAG 6001 test results and to standardize testing across the nations.
8 nations collaborated on the BAT development.
Contract awarded by NATO for the BAT delivery to ACTFL.
In 2009, 11 NATO nations participated, with up to 20 tests each .
Proposed a possible resurrection of the BAT after the 2016 BILC Conference in Riga
Evidence for the Validity Roadmap (Brno, Sept 2016)
Discussed by the WG on Portability and Recognition of STANAG 6001 Certificates (Bled, Sept 2016).
Survey of interest by BILC Secretary (fall 2017) - 21 nations responded favorably.
This year two BAT Writing&Speaking Forums will take place (9-13 July and 6-10 August 2018) at PLTCE, Garmisch Partenkirhen. Participants are encouraged to receive certification as BAT interlocuters/raters.
The certification process will begin a month after the forums.
BAT 2

13. BILC SharP

14. BILC site

16. BILC Update on Language Needs Analysis (LNA) Study at ACT
BUREAU FOR INTERNATIONAL LANGUAGE COORDINATION BUREAU DE COORDINATION LINGUISTIQUE INTERNATIONALE
BILC Update on Language Needs Analysis (LNA) Study at ACT
Julie Dubeau
Chief Standards, Foreign Languages &
Associate BILC Secretary
Canadian Defence Academy

17. Purpose of study:
To examine concordance between SLPs in JDs and minimum language proficiency required to perform tasks
Why?
SLPs set without informed judgement and “Old” interpretation of STANAG 6001
SLPs are in job descriptions, and are prerequisites for training, Force Goals, Partnership Targets, tied to promotions, incentive pay, etc.
Inflated requirements undermine the validity of the job descriptions themselves
Language professionals and organizations pressured to meet inflated requirements
Need to promote integrity of language standards reporting by nations 17

18. Participants & Findings
31 officers & 3 NCO interviewed + 26 supervisors
Capt to Col, across 5 Directorates, 4 divisions, 14 sections
15 NATO nations + 3 Partner nations
80% of analyzed posts have SLP 3333
Data confirmed it is the correct SLP
14% of analyzed posts had SLP 4 in listening and reading
BILC team was not provided with any document that reflects texts of level 4 difficulty
40% found the writing task most challenging
Supervisors interviewed stressed the criticality of the skill
STANAG 6001 Plus Levels

19. Conclusions & Recommendations
SLP 3333 set appropriately at ACT
Writing skill seems to be undertrained
Level 4 not needed based on work samples
BILC to continue standardization efforts
Nations should assess how they train writing and adopt strategies for training to higher levels
NATO to review L4 posts
STANAG 6001 Plus Levels

20. Update: BILC Report submitted to BILC SC in May 2017
BILC Report submitted to ACT in June 2017
BILC invited to report findings at NATO HQ to NATO Defence Manpower Committee (NDMC) in November 2017
NDMC approved recommendation to review L4 requirements, nations agreed
BILC is requested to provide guidelines for ongoing positional reviews, Feb 2018

21. Update: Related BILC efforts – Set up two study groups:
SG # 1 Familiarization with STANAG 6001 for Non-Specialists
Propose a prototype e-learning module building on the existing BILC documents that will include samples of language at each proficiency level and explanations;
Explore ways to make the Familiarization with the STANAG 6001 for Non-Specialists module readily available as a practical resource for non-specialists
SG # 4 Enhancing Teaching of the Writing Skill for NATO Staff Officer
Outline a set of recommendations on how nations should approach the task of increasing students’ writing proficiency, and what strategies they could use to attain this goal

22. Way Ahead: More to follow…
Outcome of Study Group #1 could inform guidelines and recommendations to NATO structures, on assigning appropriate SLPs;
Manpower & Personnel Policy rep. to prepare an official tasker to all NATO Military bodies to review SLPs using guidelines that BILC will offer.
More to follow…

23. Jana Begovic, FL Stds O – MPG Canada, and Associate BILC Secretary
FAMILIARIZATION WORKSHOP ON LEVEL 4 READING PROFICIENCY
Jana Begovic, FL Stds O – MPG Canada, and Associate BILC Secretary
Lisbon, May 2018
STANAG 6001 Plus Levels

24. STANAG 6001 Plus Levels

25. Introduction and Background
Formation of BILC Working Group on Level 4
Istanbul 2010
Aim:
to assist nations feeling ill-equipped for testing at Level 4
to amplify STANAG 6001 Level 4 descriptors
to discuss testing related implications
STANAG L4 Reading Workshop
STANAG 6001 Plus Levels

26. Working Group Members
Jana Vasilj-Begovic – Leader (CAN) Gerard Seinhorst – Co-leader (NLD) Members: Petya Georgieva (BGR) Nancy Powers & May Tan (CAN) Kåre Kildevang & Allan Christiansen (DNK) Dugald Sturges (GER) Michael Adubato (SHAPE) Annette Nolan & Keith Farr (SWE) Peggy Garza, David Oglesby and Ray Clifford (USA) Special Consultants: Mary Jo Di Biase, Martha Herzog (USA)
STANAG 6001 L4 Reading Workshop
STANAG 6001 Plus Levels

27. Products
Paper: “NATO STANAG 6001 Level 4 Language Proficiency – A Conceptual Model and Implications for Testing” (April 2013)
Article “Defining and Assessing STANAG 6001 Level 4 Language Proficiency” (Chapter 11, Language in Uniform, Cambridge Scholars, 2015)
Level 4 Reading Test Specifications
Tutorial (principles of text rating, differences btw Level 3 and Level 4 texts, sample texts, sample test development procedure, etc.)
Level 4 Test Prototype (Tester and Examinee Booklets for speaking/writing modality; administration and rating procedure)
Test Familiarization Guide
Feedback Questionnaire
L4 Reading Proficiency Workshop
STANAG 6001 L4 Reading Workshop
STANAG 6001 Plus Levels

28. US MARSHALL CENTER-GARMISCH-PARTENKIRCHEN, GERMANY October Participants from Austria, Bulgaria, Denmark, France, Latvia, Lithuania, Norway, Romania, Sweden and USA Facilitators from Canada, Denmark and USA
STANAG 6001 L4 Reading Workshop
STANAG 6001 Plus Levels

29. Workshop Aim/Objectives
Aim: To familiarize participants with Level 4 reading proficiency, and in particular, with a reading test prototype
  Objectives:  
To discuss the nature of the reading skill and the challenges of its assessment at Level 4
To distinguish between Level 3 and Level 4 texts, descriptors, and test items IAW STANAG 6001
To discuss pros and cons of testing techniques and scoring at this level
To practice developing Level 4 constructed-response items
To rate Level 4 sample tests using a novel approach
STANAG 6001 L4 Reading Workshop
STANAG 6001 Plus Levels

30. Comments High expertise of both facilitators/participants
Language enhancement environment
Learning about a new rating procedure & hands-on rating
Understanding differences between L3 and L4 texts
Raised awareness of the L4 construct & its complexity
Well-balanced activities
Dispelling misconceptions about the nature of language
Productive work with facilitators and participants
STANAG 6001 Level 4 Reading Test
STANAG 6001 Plus Levels

31. Suggestions for Improvement
More discussion on how to write L4 items
More rating sessions and item development activities
Hands-on work (pair/group) on all days
STANAG 6001 L4 Reading Workshop
STANAG 6001 Plus Levels

32. NEXT SERIAL??? Prerequisite for attendance: ALTS/LTS graduates
Full-time testers
Need for Level 4 testing
8-10 participants
STANAG 6001 Plus Levels

33. Development Methodology Production Planning Process
Global Programming
It is based on management framework with THREE elements
Global Programming
Governance Structure
Development Methodology
Production Planning Process

34. Bi-SC List of Disciplines as of 2017
Domain
Function
Capability Development
Mission Execution Enhancement
Land Operations
Air Operations
Maritime Operations
Space Support to Operations
Cyber Defence
Special Operations
Nuclear Operations
Intelligence
Meteorology and Oceanography
Joint Targeting
Military Policing
Logistics
Medical Support
Operations Planning and Assessment
Military Engineering
Consultation, Command and Control
Education, Training, Exercises & Evaluation (ETEE)
Financial
Civil Military Cooperation and Civil Military Interaction
Counter-Improvised Explosive Devices
Weapons of Mass Destruction/ CBRN Defence
Strategic Communications
Ballistic Missile Defence
Federated Mission Networking
Building Integrity
Counter-Terrorism
Energy Security
Mil Contribution to Peace Support
Gender in Military Operations
Ʃ = 7
Ʃ = 15
Ʃ = 2
Ʃ = 5
Air Operations Air Ops
Ballistic Missile Defence BMD
Building Integrity BI
Civil Military Interaction and Civil - Military Cooperation CMI & CIMIC
Consultation,Command and Control Comms and Info Int
Countering Improvised Explosive Devices C-IED
Counter-Terrorism CT
Cyber Defence Operations Cyber Def Ops
Education, Training, Exercises and Evaluation ETEE
Energy Security ENSEC
Federated Mission Networking FMN
Financial FIN
Gender in Military Operations Gender
Intelligence Intel
Joint Targeting Joint Targeting
Land Operations Land Ops
Logistics, Support and Resources Log
Maritime Operations Mar Ops
Medical Support Med Sup
Meteorology and Oceanography METOC
Military Contribution to Peace Support MC2PS
Military Engineering MILENG
Military Police MP
Space Support to Operations Space Sup
Nucleair Operations Nuc Ops
Operational Planning and Assessment Ops Plng & Assess
Special Operations SO
Strategic Communications STRATCOM
Weapons of Mass Destruction / Chemical Biological Radioactive Nuclear WMD / CBRN
Candidates for 2018 list: Human Resources, Legal, JPR,M&S
Total = 29
ETEE includes a mixed bag of topics. One is “Languages”.

35. Education, Training, Exercises & Evaluation (ETEE)
ETEE Department Head (DH) is the NATO School
The Training Requirements Analysis (TRA) workshop for “Languages” was held at the NATO School in December 2017
Participants involved were representatives from 3 PTEC* language schools
Partner Language Training Center Europe (PLTCE) (US), Garmisch, Gemany
Foreign Languages Department (FLD) Shumen, Bulgaria
School of Foreign Languages, Poljce, Slovenia
* NATO-recognized Partnership Training and Education Centres
Also invited: Phil Turner, HQ NATO/IMS and Mike Adubato, NATO/ACO SHAPE

36. NATO considers “Language Training” a national responsibility

37. TRA – DAY 1 - Determine performance statements and level for training audiences
TRA – DAY 2 – Use course listings from ETOC to match existing training solutions
to requirements (performance statements)
TRA DAY 3 – Final product - training solutions matched to requirements.
Green = requirement fully met / Yellow = requirement partially met /
Red = requirement not met

38. Excerpt from TRA Report
Findings *(para 8b - Languages)
Leadership at all levels are required to have awareness of the STANAG 6001, Language Proficiency Levels.
For Language appointed SMEs in the NCS, participation in the BILC Annual Conference, the BILC Annual Professional Seminar, and the STANAG 6001 Testing Workshop is strongly encouraged as part of their professional development programme.
Education in English communication is assumed a national requirement. For further professional development in English communication, several HQs offer additional internal education and training, e.g. HQ SACT’s Strategies for Writing Course.

39. Conclusions
The Language Testing Seminar, the Advanced Language Testing Seminar, the Language Standards & Assessment Seminar and the English Teaching Faculty Development Workshop meet NATO training requirements.
Training gaps were identified in these areas:
STANAG 6001 familiarization
for staff officers
Writing IAW NATO conventions