Bureau for International Language Coordination Julie J. Dubeau BILC Secretary Istanbul, Turkey May 24, 2010

General information LNA report BAT Report Outline of Presentation

What does BILC do?  Annual Conference in Spring June 2009 hosted by Italy - “Bridging the Gap: Language Requirements vs. Language Reality” May 2010 hosted by Turkey - “Mapping The Road: Success in Language Training”  Professional Seminar in Fall October 2009 hosted by Denmark – "The 21st Century Classroom: Keeping up with the Times!“ October 2010 hosted by Bulgaria – “Aligning Training and Testing in Support of Interoperability”

Ratified by the nations and promulgated by NSA in February 2009 English and French versions can be downloaded from the BILC website Steering Committee will be discussing a proposal for admin change during conference Responsible for STANAG 6001

What else does BILC do? Language Training Assessments Assistance to National Testing Programmes Language Testing Seminars:  LTS & ALTS Special Projects:  Benchmark Advisory Test (BAT)  Language Needs Analysis (LNA)

General Information ACT is tasked by IMS to investigate the feasibility of introducing a computer based learning tool for use by nations to familiarize with NATO terminology… BILC is investigating requirement through JSSG

“ACT should develop ADL solutions that will complement nations’ efforts in language training, understanding of different local government structures and understanding familial, clan and tribal cultures”. ACT has requested BILC’s assistance Please contact the BILC Secretariat if you are interested in providing an ADL product Complementary ADL Solutions IMS Report Dated November , Bratislava Follow-up Tasking on Counter Insurgency (COIN)

November 2008 Chairmen’s Meeting of the NTG in Brussels The aim of this study was to show whether language requirements appeared to be set at appropriate levels to enable military personnel to perform their duties adequately in the NATO OPS context, in this case ISAF. LANGUAGE NEEDS ANALYSIS OF NATO CRISIS ESTABLISHMENT (CE) POSTS

To broadly identify whether the mandatory SLP requested for the post was:  At Level ~ A: meaning that from the tasks described in the JD, the profile appeared to be adequate;  High ~ H: meaning that the SLP requested for the post appeared to be higher than the functions defined in the job description; or  Low ~ L: meaning that the SLP requested for the post appeared to be lower than the functions defined in the job description. LANGUAGE NEEDS ANALYSIS OF NATO CRISIS ESTABLISHMENT (CE) POSTS

Summary of Results RatingsTotal = 609 CE Post JDs A - At Level355 (58.2%) H- High203 (33.3%) L- Low41 (6.7%) H/L- Mixed10 (1.6%)

Job Descriptions requiring SLPs of Level 1: 30 JDs required SLPs of 1111 & all were High. Job Descriptions requiring mixed SLPs of level 1 & 2: 4 required SLPs of 2211 & all were at level. 2 required SLPs of 2221 & both were at level. Job Descriptions requiring SLPs of level 2: 61 JDs required SLPs of /61 (26.2%) were at level. 43/61 (70.4%) were High. Results: Levels 1 & 2

Job Descriptions requiring mixed SLPs of Levels 2 & 3: 156 (25.6%) JDs required mixed SLPs of Levels 2 and 3. Of these 156, 66 (42.3%) were considered as having SLPs that were at level Further Breakdown: 1 required SLP 3221 (High/Low depending on skill) 9 required SLP 3222 (7/9 High) 1 required SLP 3223 (High) 4 required SLP 3232 (3High/1Low) 118 required SLP 3322 (47 at level (39.8%), 56 High (47.4%) & 3 Low (2.5%)) 23 required SLP 3332 (17 at level, 6 Low note that these 6 also req Dari) Results: Levels 2 & 3

Results: Level 3 Job Descriptions requiring SLPs of Level 3: 301 (49.4%) required SLP /301 (79.9%) were considered at level 44/301 (14.6%) were considered High 13 were considered as Low SLPs for functions, (all req expert language knowledge with 3s in Dari and Pashto)

Job Descriptions requiring mixed SLPs of level 3 & 4: 43 profiles of 4343, from which 22/43 were at level (51.1%), and 21(48.8%) were not scored at level (some H some L) 1 profile of 4344 (Low), 1 profile of 4443 (High) Results: Levels 3 & 4

Results: Level 4 Job Descriptions requiring SLPs of level 4: 8 profiles of 4444 (6/8 were considered at level, with comments referring to the functions as requiring expert language ability, diplomatic or sophisticated language use, 2 were H) Out of the 53 JDs requiring some skills at level 4, 10 (18.8%) also req languages other than Eng such as French, Dari, Pashto, etc. Out of these 10, 6 (60%) were at level.

Recommendations NATO CE Post JD SLPs should be reviewed SLPs should be based on an analysis of the tasks performed by incumbents in relation to STANAG 6001 Ed 3. It is strongly recommended that the next analysis of requirements be done with the assistance of BILC language experts. CE posts master sheet should include the SLPs requested for the posts.

IMS-Tasking on Language Proficiency and Education on NATO Standards & Terminology. 12 February 2010  Analysis of CE/PE posts Setting realistic requirements Review of linguistic requirement for Officers for operationally deployed duties‘ has been initiated  Benchmarking What’s next for interoperability?

Benchmark Advisory Test (BAT)- PURPOSE To provide an external measure against which nations can compare their national STANAG test results To promote relative parity of scale interpretation and application across national testing programs To standardize what is tested and how it is tested

Benchmark Advisory Test (BAT) Allocation to 11 Nations (200 ‘free’ tests) Tests administered by LTI, the ACTFL Testing Office via the Internet and Telephone January 2010 – End of Benchmarking Process Positive Feedback from Nations

Benchmark Advisory Test (BAT) Results ListeningSpeakingReadingWriting Black(11)64%(18)72%(10)60%(11)55% White(18)61%(18)56%(18)94%(18)39% Red(18)89%(18)83% (18)50% Blue(20)85%(19)47%(20)55%(20)60% Teal(8)0%--(8)75%(8)38% Maroon(16)69%(15)47%(14)64%(16)44% Pink(18)67%(18)50%(18)78%(18)67% Purple(12)8%--(13)54%(13)62% Orange(13)77%(13)46%(13)54%(13)62% Yellow(17)24%(18)0%(18)33%(18)0% Green Nation did not release national test scores, but indicated that, on average, they were higher than those attained on the BAT.

Test purpose Testing method Use of plus ratings Alignment of author purpose, text type, and reader/listener task Inadequate tester/rater norming (productive skills) Inconsistencies in the interpretation of STANAG 6001 Cut-off score setting, etc. FACTORS CONTRIBUTING TO NON- ALIGNMENT

BAT – Way Ahead Project successful Demand for administrations will dictate future development needs and modes BILC SC to formulate recommendations Nations who require BILC support (post-BAT) can request it

QUESTIONS? Enjoy the Conference!