50 Years of BILC: The Evolution of STANAG 6001 1976 – 2016 and the first Benchmark Advisory Test Ray Clifford 24 May 2016

1976: STANAG 6001, Edition 1 NATO adopted the standards that were drafted by BILC during 1973 through 1975. The lack of consistent specifications limited the STANAG’s usefulness for assessment purposes. The brevity of the descriptions allowed for multiple interpretations. A testing expert later called those descriptions “thin, haphazard, and often off the mark”. To fill the gaps, NATO nations implemented their own interpretations.

Negative Impact of Edition 1 National interpretations were not aligned. National tests and ratings were not equivalent. My level 3 might equal your level 2 or even your level 1. The Standardized Language Profile (SLP) for jobs became inflated. SLP’s were often set by people-comparisons rather than by criterion-based job task analyses. “The last guy was a 3; and his language skills were weak, so we need a 4.” The lack of language skills was identified as NATO’s greatest challenge to interoperability.

Actions Taken 1999: BILC formed a working group on Testing and Assessment. Its initial goal was to elaborate on and interpret the original STANAG 6001 descriptors. Representatives from 11 BILC member-nations participated. 2000: That working group produced the “Interpretation and Elaboration Document”. That “Interpretation” was successfully trialed in the new BILC Language Testing Seminar.

2003: STANAG 6001, Edition 2 2002: The BILC Steering Committee Approved the “Interpretation Document”. Forwarded it to the NATO Standardization Agency for approval and publication. 2003: STANAG 6001, Edition 2 was released. It was the first substantive change in a quarter of a century. The interpretation and elaboration document was included as Annex A.

2009: STANAG 6001, Edition 3 After extensive study and work, Edition 3 was published with several substantive changes. It changed the labels for the levels so they better reflected the content of the level descriptions. 0 = No proficiency. 1 = Elementary Survival. 2 = Fair Functional. 3 = Good Professional. 4 = Very Good Expert. 5 = Excellent Highly-articulate native.

2009: STANAG 6001, Edition 3 (Continued) It defined language proficiency as “unrehearsed, general language communication ability”. It added descriptions of “plus levels” as Appendix 1 to Annex A. “A plus level is understood to be more than halfway between two base levels.” “A plus level substantially exceeds the base level, but does not fully or consistently meet all of the criteria for the next higher base level.”

2010: STANAG 6001, Edition 4 Added two additional purposes for which plus levels may be used. Edition 3: “A plus level may be added to a base level for training or evaluation purposes … “ Edition 4: “A plus level may be added to a base level for training, evaluation, recording or reporting purposes … “

2014: STANAG 6001, Edition 5 2014: Edition 4 reformatted and republished in ATrainP-5 as Edition 5. The format and introductory pages were modified. The standards were not intentionally changed. But typos were introduced in the publication process. 2016: Edition 5, version 2 was just published. Typos have been corrected. Edition 5 will also be available on the BILC website.

Benchmark Advisory Test (BAT) A Parallel Effort that Applied the Improved STANAG 6001

Why Benchmark Testing? To promote consistent interpretation and interpretation of STANAG 6001 across national testing programs. To provide an external measure against which nations can compare their national STANAG test results. To pursue a joint development project as a way of demonstrating how STANAG 6001 tests can be designed, developed, and scored.

Benchmark Advisory Test (BAT) 2003: Possibility raised; options discussed. 2004: Steering committee recommended the concept to the NATO Joint Services Subcommittee Group. 2005: BILC launched a volunteer, collaborative effort to create a Reading and Listening BAT. 2005-6: ACT took accepted the BILC work and Contracted to complete tests in 4 skill modalities. Set an implementation date of 2009.

2009 BAT Administration 200 tests were allocated across 11 Nations. Testing began in May, 2009. Tests were administered by Language Testing International. Results were reported to each participating nation, but only for their own personnel. Overall results were reported without identifying the test takers’ nationality.

Summary The STANAG 6001 revision and the BAT development project were an unparalleled success. You, the BILC family, are the reason they succeeded I have never seen an international team that worked with such a high level of energy, goodwill, cooperation, and dedication.

Congratulations on your accomplishments! Summary The STANAG 6001 revision and the BAT development project were an unparalleled success. You, the BILC family, are the reason they succeeded I have never seen an international team that worked with such a high level of energy, goodwill, cooperation, and dedication. Congratulations on your accomplishments!

Stop! Your time is up!

A Footnote In case you don’t remember the BAT study, … the overall BAT test results will be reviewed in my presentation on Thursday.