ALIGNING TEACHING AND TESTING IN BULGARIA IN SUPPORT OF INTEROPERABILITY

ENGLISH LANGUAGE TRAINING FOR THE BULGARIAN ARMED FORCES Mission: To provide quality English language training to military and civilian personnel for the needs of the Bulgarian Armed Forces.

ENGLISH LANGUAGE TRAINING SYSTEM Strategy for Developing the English Language Training and Testing System for the Period Strategy for Developing the English Language Training and Testing System for the Period

ELT curricula for cadets in National Military University (NMU) and Naval Academy (NA) standardized ELT curricula for cadets in National Military University (NMU) and Naval Academy (NA) standardized ELT curricula for officers in Rakovski National Defence Academy (NDA) ELT curricula for officers in Rakovski National Defence Academy (NDA) content of the General English Intensive Courses (Levels I-IV) standardized content of the General English Intensive Courses (Levels I-IV) standardized course entry and exit requirements and student assessment standardized course entry and exit requirements and student assessment standardized

Foreign Languages Department (FLD) in the town of Shumen established Foreign Languages Department (FLD) in the town of Shumen established 5 Self-Access Centers established (NMU, NA and NDA) 5 Self-Access Centers established (NMU, NA and NDA) Center for Language Training and 4 language labs established in the units with the help of the Kingdom of the Netherlands Center for Language Training and 4 language labs established in the units with the help of the Kingdom of the Netherlands new courses (Specialized English, refresher, distance learning) and materials for SACs developed new courses (Specialized English, refresher, distance learning) and materials for SACs developed

through FMF and IMET programs: through FMF and IMET programs: 421,000 USD worth of ALC materials 421,000 USD worth of ALC materials language labs (incl. upgrade and maintenance) – 14 labs seating a total of 245 students in the military academies, 3 labs in the units language labs (incl. upgrade and maintenance) – 14 labs seating a total of 245 students in the military academies, 3 labs in the units Mobile Training Teams - ELT advisor from DLIELC in the FLD in Shumen Mobile Training Teams - ELT advisor from DLIELC in the FLD in Shumen teacher training in the US and Canada teacher training in the US and Canada

ELT program coordinator in MoD ELT program coordinator in MoD coordinators in the staffs of the three services coordinators in the staffs of the three services commission on ALCPT use (+ test control officers locally) commission on ALCPT use (+ test control officers locally) STANAG 6001 testing team under the MOD STANAG 6001 testing team under the MOD annual seminar attended by all parties involved in the ELT process annual seminar attended by all parties involved in the ELT process

English Language Training for the BAF English language training in the Bulgarian military education institutions As part of the curricula for cadets and NDA officers As qualification courses for military and civilian personnel ELT in the units

ENGLISH LANGUAGE TRAINING in NMU, NA and NDA NMU and NA cadets cadets 700 hours of English language training 700 hours of English language training homogeneous groups based on ALCPT scores homogeneous groups based on ALCPT scores exit requirements - minimum SLP 1111 upon graduation. exit requirements - minimum SLP 1111 upon graduation. Rakovski NDA officers doing an MA degree in Military Science officers doing an MA degree in Military Science 760 hours of English language training 760 hours of English language training homogeneous groups based on ALCPT scores homogeneous groups based on ALCPT scores exit requirements - minimum SLP 2222 upon graduation. exit requirements - minimum SLP 2222 upon graduation.

English Language Courses Course type min SLP to be achieved Course duration Instruction materials Minimum entry ALCPT requirements Minimum exit ALCPT requirements Level I 18 weeksALC Books 1-12 N/A50 points Level II SLP weeksALC Books points70 points Level III SLP weeksALC Books points80 points Level IV SLP weeksALC Books points and SLP points Specialized English courses 2-8 weeks80 points or SLP 2222 set by the training institution Refresher courses 4 weeks65/7570/80

English Language Courses – ctnd. Student screening and selection Student screening and selection Student assessment Student assessment - BQs - BQs - ALCPTs (screening, mid-course (5/8 week), final) - ALCPTs (screening, mid-course (5/8 week), final) Certificates Certificates End-of-course evaluation report End-of-course evaluation report Disenrollment Disenrollment

Course graduates (Jan 2009-July 2010)

SLPs received by students within 6 months of graduation (% students)

STRATEGY : Main Points To Be Considered  cadets – min SLP 2222 upon graduation as of 2012/2013  NDA officers – min SLP 3332 upon graduation as of 2013/2014; admission requirements for language skills - min SLP 2222  mandatory STANAG 6001 exam after completing a course  DL courses and module-based courses designed for the needs of the deployable forces  Level IV course content to be revised  revising course length for all levels

Qualification Courses Taught in English  specialized qualification courses for commanding officers with modules taught in English  academic staff teaching Military Science in military academies and Rakovksi NDA – min SLP 2222

PHASE TWO - TESTING

They are native speakers but are they interoperable? A COLONEL ISSUED THE FOLLOWING DIRECTIVE TO HIS EXECUTIVE OFFICERS: “Tomorrow evening at approximately 2000 hours Halley’s Comet will be visible in this area; an event which occurs only every 75 years. Have the men fall out in the battalion area in fatigues, and I will explain this rare phenomenon to them. In case of rain, we will not be able to see anything, so, assemble the men in the theater and I will show them films of it.” EXECUTIVE OFFICER TO COMPANY COMMANDER: “By order of the Colonel, tomorrow at 2000 hours, Halley’s Comet will appear above the battalion area. If it rains, fall the men out in fatigues, then march to the theater where this rare phenomenon will take place, something which occurs only once every 75 years.” COMPANY COMMANDER TO LIEUTENANT: “By order of the Colonel, be in fatigues at 2000 hours tomorrow evening. The phenomenal Halley’s Comet will appear in the theater. In case of rain in the battalion area, the Colonel will give another order, something which occurs once every 75 years.” LIEUTENANT TO SERGEANT: “Tomorrow at 2000 hours, the Colonel will appear in the theater with Halley’s Comet, something which happens every 75 years. If it rains, the Colonel will order the comet into the battalion area.” SERGEANT TO SQUAD: “When it rains tomorrow at 2000 hours, the phenomenal 75-year-old General Halley, accompanied by the Colonel, will drive his comet through the battalion area in fatigues.”

SUCCESSIVE AND SUCCESSFUL TESTING Successive – comes after language training: Successive – comes after language training: - the cadet graduates of “V. Levski” NMU and “N. Vaptsarov” NA, as well as the officers who graduate “G. S. Rakovski” NDA; - the military and civilian personnel who complete English language courses in military educational institutions.

SUCCESSIVE AND SUCCESSFUL TESTING (cont.) Successful Successful - tests should be reliable and valid.

Reality 1. Finances are limited. 2. There are test candidates per year. 3. There are 5 testing sites. 4. We develop 324 reading items, 324 listening items, and 60 writing prompts each year. 5. The testing team consists of 5 experts.

Bulgarian English Language STANAG 6001 Testing Sector One test team is part of the Ministry of Defense, Human Resources Management Directorate. One test team is part of the Ministry of Defense, Human Resources Management Directorate. The same testing team, develops and administers the STANAG 6001 tests, issues the certificates and is responsible for the data update in the Personnel Automated System. The same testing team, develops and administers the STANAG 6001 tests, issues the certificates and is responsible for the data update in the Personnel Automated System.

Bulgarian STANAG 6001 Testing Team

The Importance of Being Earnest How do we strive to ensure objective testing ?

All STANAG 6001 documents published on the Internet sites of MoD, NMU, NA and NDA. All STANAG 6001 documents published on the Internet sites of MoD, NMU, NA and NDA. STANAG 6001 Specifications widely spread. STANAG 6001 Specifications widely spread. Example tasks for reading, listening and writing on the Internet sites. Example tasks for reading, listening and writing on the Internet sites. Three guidebooks for self preparation. Three guidebooks for self preparation. Clear rating procedures. Clear rating procedures.

Objectivity in Reading The papers are anonymous. Multiple-choice items only. 18 items per each level. 70% pass rate. The three versions of the test are piloted on one and the same group of students and the results should be closely compatible.

Objectivity in Listening The papers are anonymous. Multiple-choice items only. 70% pass rate. 18 items per each level. The three versions of the test are piloted on one and the same group of students and the results should be closely compatible.

Objectivity in Writing The papers are anonymous. Always 3 tasks in each test version. Rating Scale for Writing. Before each STANAG 6001 exam session – norming rater-refresher sessions. Two raters read the papers separately and, if they give different levels, the paper is read by a third rater. After each session – raters reliability analysis.

CAT

Objectivity in Speaking Before each STANAG 6001 exam session – norming rater-refresher sessions. Rating Scale for Speaking. The oral interview is taped. There are two testers in one testing board. There is a strict scenario for the different stages of the interview. After each session – statistical analysis of raters reliability.

What does “our English face” look like? Level 1111 Level 1-2 Level 2222 Level 2-3 Level 3333 Level 3-4 Level 4444 Total number 9%44%18%20%8%1%0.05% 4210 active certificates

Diagram of Levels

Levels 2/2-3/3/3-4/4 Certificates Distribution

STT efforts = success Students + Teachers + Testers = Students + Teachers + Testers =