1. 1 Cooperative Language Training Programme Assessments “Mapping the Road: Success in Language Training” Keith L. Wert Associate BILC Secretary for Programme Assessment Director, Partner Language Training Center Europe George C. Marshall European Center for Security Studies wertk@marshallcenter.org

2. 2 Cooperative Language Training Programme Assessments History: – 2000: Requested by a NATO Assistant Secretary General to Chair, Nato Training Group who wrote BILC secretariat – 2001 Conducted first assessment – 2001-2010 Before Accession: – Slovakia, Latvia, Slovenia, Estonia, After Accession: – Czech Rep., Bulgaria, Romania PfP and IPAP countries – Macedonia, Georgia, Serbia – Countries that have asked for follow up visits: Czech Republic, Bulgaria, Slovakia, Georgia, Macedonia and Serbia.

3. 3 Language Training Programme Assessments Team members from: – Slovenia, Sweden, Germany, Canada, UK, and US. Level of Interest: – Latvia: Deputy State Secretary – Slovakia: Director of Military Education – Macedonia: Chief of Staff – Bulgaria: Deputy Chief of Staff – Georgia: 1st Deputy Minister of Defense – Romania: Head of MoD Human Resources – Serbia: The J-7 and MoD Personnel Sector Serbia: May 2009

4. 4 Items of Interest  Development of a language policy  Integrating language policy into military personnel policies  Development of a language training structure that meets the objectives of the language policy  Establishing effective and efficient use of language training resources - Appropriate emphasis on and balance between intensive and non intensive programs - Ensuring resources are allocated in a transparent and ‘objective’ manner - Effective and efficient language testing programs to NATO standards - Transparent procedures for faculty professional development - Harmonizing bilateral support for language training - Development of a modern military lexicon, based upon agreed NATO nomenclature - Development of Syllabi at STANAG 6001 Level 3

5. Pre-Visit Process Provide the potential scope of what you are going to be looking over to the appropriate coordinating office so that everyone knows what the objectives are Not a check list. You never really know what aspects of the assessment will require the most attention We provide “Outlines” with points for potential discussion and observation. – Policies – Schools – Testing

6. Pre-Visit Process (Outlines)

7. What is the focus? We are generally not: – Components Book Orders Teacher training Mgt courses Technology – Labs, IMI etc Doing some legwork for the bilateral donors: – Book orders – Course recommendations We are: – Processes Language and Testing Policies Integration with military personnel policies Language Training Management Resource allocation Faculty Development plans

8. Visit Process Extensive orientation briefings by MoD/General Staff – Policy and organization explained Visits to schools Classroom observations Discussions with management Discussions with faculty and students Meetings with bilateral language assistance responsibles, e.g. British Council, Offices of Defense Cooperation. (“Donor countries”)

9. Analytical Process: Overarching Objective To review how the language policy fits with military personnel policies. To see if the language training structure meets the policy objectives. To see if the structure can produce the required numbers of graduates at the required proficiency levels in a somewhat predictable manner.

10. Areas of Interest Language Policy (Personnel policies) Language resources allocation Language laboratories and self-access centers Testing policies and processes Syllabi standardization or how long does a student take to reach Level 1,2,3? (and now4!!!!!) Teachers contra management Management contra teachers Military language instruction Professional Development (transparency thereof) Intensive vs. non intensive language programs

11. Language Policy A language policy must be an inseparable component of personnel policy. If it is not, that is the first sign that there is a problem. Does the language policy exist in a vacuum? – Were the schools involved in making the policy? – Is the policy realistic? – Do the policy makers ever check to see if the policy is successful? – Are personnel assigned to language training for a reason?

12. Language and Personnel Policy MOD/General/Joint Staff Language Schoolhouse Management Are they talking to each other?

13. Language Policy Simple reality test: – Visit several classrooms in different locations and ask a range of teachers and students the same question: “Why are you in this classroom?”

14. Language Resources Allocation Why are some locations well resourced and others not? – Is a language lab at a military unit as important as one at a school house with a full time intensive language program? – Who is in charge of language training resources? – Is the allocation process “transparent”?

15. Testing Policies and Processes Is there a STANAG 6001 Testing Frenzy? – Overused STANAG tests? – Simpler ways to determine Level 1? – Is testing focus on particular assignments? (Personnel policy loop) – Don’t test more than necessary

16. Language Labs and Self-Access Centers They have a life of their own: I know this because they keep reproducing themselves everywhere we go – Great in concept, difficult in execution – Tend to draw resources away from more fundamental needs: like books

17. Syllabi Standardization There should be evidence of language training predictability, especially at the lower levels – If no one knows how long it takes the normal language student to reach Levels 1 and 2, there is a problem – If it is known how long it takes and authorities shorten course lengths anyway, there is a problem – Course lengths determined arbitrarily – Stability and predictability at lower levels is important

18. “ If you can’t measure it, you can’t manage it.” Dr. Ray Clifford, Sweden October, 2001

19. Syllabi Standardization Certificates from different courses do not tell you anything – The students passed the “test” and received credit No comparability across classrooms No comparability across institutions in the same country

20. Teachers contra management Highly educated and professional teachers don’t always grasp the larger scale training management issues and confuse higher level management with the wrong input: – “These books are boring, that is why the students are not succeeding!” – “The test is bad, not a truly professional test.” (I don’t like this test format.)

21. Management contra teachers Senior management has to pay attention to the real results teachers and school houses produce – If it is working, leave it alone and let the language professionals improve it incrementally – Stability is important – Having the power to do something is not synonymous with knowing what to do

22. Military language Primary warning signs are: – Teachers who do not feel it is in their profession to teach it – Courses that introduce highly technical language at low levels of language ability – Leaders who think that just the language and terms of the specialty need to be learned – Intensive basic courses that rely on military language as the primary component

23. Teacher Professional Development There are limited numbers of opportunities for teachers and these opportunities must be handled in a rational and transparent manner Professional institutions everywhere have application procedures and published standards for awarding teachers with research grants and development opportunities There should be a well-thought out process controlled by the country, not the bilateral (donor) providers

24. Intensive vs. Non intensive Programs “Military language training” has unique attributes: – Large numbers of students/High proficiencies to attain Non-intensive basic courses rarely succeed Putting language training resources at local military bases looks good on PowerPoint – The students are frequently pulled from class The issue is diffusion of resources: what percentage is spread thinly to poor effect

25. Line Management Students not grouped /regrouped by ability Classes too large Listening materials not used enough Teachers not changing classes Student attrition policies Working conditions in classrooms poor Evaluations of teacher performance

26. Potential Positive Outcomes Country takes fundamental look at language system Country uses “external experts” to help make hard internal decisions

27. Questions?