(Standardizing the Standards of Teaching and Testing in the Military) Ministry of Defense      The Bulgarian Strategy for Developing English Language Training and Testing (2006-2010) IN ACTION (Standardizing the Standards of Teaching and Testing in the Military) Personnel Policy Directorate

Personnel Policy Directorate

Presentation Overview      Legal basis for implementing the ELT and testing strategy for the Bulgarian Armed Forces (BAF), 2006 to 2010 Standardization of STANAG 6001 testing for the BAF Challenges to implementing the Force Goal EG 0356 Personnel Policy Directorate

Legal Basis Documentation      Strategy for ELT and testing for the Bulgarian Armed Forces (BAF), 2006 to 2010 Ministerial Order establishing Steering group Working groups Commission for oversight of ALCPT usage Identification of Test Control Officers Guidance on administering of ALCPT Personnel Policy Directorate

Personnel Policy Directorate Legal Basis (Cont)      Order of the Chief of the GS for relocation of IMET Labs Plan for implementation of the Strategy, 2007 to 2010, endorsed by Deputy Minister Ministerial order on procedures for ALCPT placement, progress and final test and Book Quiz administration Selection criteria Pre-testing Entry, inter-mediate and final ALCPTs Personnel Policy Directorate

Activities of Working Groups Curricula for General Military English courses: developed and trialed Curricula for Specialized English courses: being developed Curricula for Military English courses for Cadets and graduate students: developed Projects for SACs: being developed Distance learning courses for ELT: 24-week 2nd Level course being piloted and two 6-week refresher courses for 3rd and 4th levels being developed. [demo clip]      Personnel Policy Directorate

Trained Military and Civilian Personnel Bulgarian Armed Forces Trainees in ELT in 2006 Training Structures Trained Military and Civilian Personnel Total MoD Bulgarian Armed Forces Officers NCOs Civilians Enlisted Military Schools 41 10 3 401 154 2 29 640 Military Units 217 72 6 36 331 Private Organizations 13 18   54 12 21 621 226 8 65 1007 Personnel Policy Directorate

Personnel Policy Directorate Equipment for ELT Language Laboratories: 33 (20 in the military academies and 13 in the military units) with approximately 500 lab positions IMET: 17 Labs MoD funded: 13 Labs Government of Netherlands: 3 for military units $500,000 from FMF program for upgrade and expansion of IMET labs in military academies Self Access Centers: 4 (3 from the PEP Project and 1 MoD funded) Personnel Policy Directorate

Personnel Policy Directorate Training Materials ALC materials and tests available in all ELT institutions Surplus ALC materials redistributed to military bases $750,000 from FMF program for ALC materials and tests; 1st shipments received in March/April Materials previously provided through the PEP program Materials previously and currently provided from MoD Personnel Policy Directorate

Language and Information Technology Training Center (LITTC) in Shumen Established in September 2005 First courses trialed in April 2006 Opening in September 2006 Formal dedication in September 2007 Number of students (2006): 174 Number of students (2007): 213 (current-103/ projected in Sept 07-110) Personnel Policy Directorate

Personnel Policy Directorate LITTC Center for basic ELT in BAF – Levels1, 2 and 3 courses Staff Director: Land Forces officer Civilian teachers of English: 25 IT teachers: 5 Administrative staff: 5 Personnel Policy Directorate

Personnel Policy Directorate LITTC (cont) Level II 24-position labs: 3 Level IV 15-position labs: 3 SAC with 12 positions ELT classrooms: 15 Instructor offices: 11 Dormitory rooms for 300 students under renovation (2007 MoD funded) Renovated dining facility (2006 MoD funded) Athletic facilities to be renovated as a future MoD funded project in 2008 Personnel Policy Directorate

Standardizing the Standards in STANAG 6001 Testing Perspectives In-country Regionally Internationally Personnel Policy Directorate

In-country Standardization Independent testing body (professional testers) Standardized criteria Standardized test documentation Standardized test format Standardized administration procedure Standardized scoring procedures High test reliability and validity High inter-rater reliability Personnel Policy Directorate

Personnel Policy Directorate Test Reliability Reading Mean Median Mode St.D. Alpha No items No cases A 31.846 33.00 35.00 7.867 .8630 61 26 B 28.24 29.00 29; 33;36 8.576 .9017 29 C 40.91 40.50 39.00 8.11 .8837 22 LISTENING Mean Median Mode St.D. Alpha No items No cases A 25.588 24.00 23.00 10.896 .9326 56 34 B 22.61 22.50 28.00 8.899 .9076 18 C 30.20 33.00 10.56 .9268 25 Personnel Policy Directorate

Latest Statistics for Reading 45,0 40,0 35,0 30,0 25,0 20,0 15,0 Reading A 8 6 4 2 Std. Dev = 7,87 Mean = 31,8 N = 26,00 Personnel Policy Directorate

Latest Statistics for Listening TOTLISTC 55,0 50,0 45,0 40,0 35,0 30,0 25,0 20,0 15,0 Total Listening C 6 5 4 3 2 1 Std. Dev = 10,56 Mean = 30,2 N = 25,00 Personnel Policy Directorate

Inter-rater Reliability RATER1 Pearson Correlation Sig. (2-tailed) N 1 . 10 ,811 ** ,004 **. Correlation is significant at the 0.01 level (2-tailed). Personnel Policy Directorate

Regional Standardization Piloting new Bulgarian test versions in the region: Albania, Croatia, Estonia, Hungary, Slovakia, Slovenia Piloting other countries’ STANAG tests in Bulgaria: Croatia, Serbia, Slovakia, Slovenia Bilateral relations: visit to Romanian MoD/ HRMD in March’07 Opportunity to discuss testing issues and further promote standardization Personnel Policy Directorate

International Standardization Within BILC Consultants from DLIELC (U.S), Canada, the Netherlands The BAT work group: how nations apply the STANAG 6001, opportunity to tune up national efforts Personnel Policy Directorate

International Standardization (Cont) Outside BILC EALTA/ILTA conferences, seminars, colloquia correlating our efforts to the world leading testing experience Personnel Policy Directorate

Personnel Policy Directorate Personnel Tested in 2006 Language Personnel tested failed passed 1111 1-2 2222 2-3 3333 3-4 4444 English 1200 240 960 80 % 41 4% 407 42% 131 17% 286 30% 88 9% 7 1% More than 54% of those tested achieved L2 and higher Personnel Policy Directorate

Language Proficiency IAW STANAG 6001 Total English 466 1849 825 272 86 4 3969 Personnel Policy Directorate

Current STANAG Session April/May 2007 Applicants L 1 L 1-2 L 2 L 2-3 L 3 236 2 76 66 59 33 More than 67% of those tested achieved L2 and higher Personnel Policy Directorate

Personnel Policy Directorate EG 0356 Challenges Ministerial order No OX – 59/25.01.2005, List of key positions in the MoD and BAF: 1804 (4% met the EG 0356 requirements) In 2005: 1140 personnel received ELT of which 753 from the key positions In 2006: 1007 personnel received ELT of which 705 from the key positions Current status of meeting the EG 0356 requirements 15 % HRF; 12 % LRF; 74 % MHQ Personnel Policy Directorate

Trainees from Key Positions (2006 by Category) Officers NCOs Enlisted Civilians High readiness forces 113 78 11 Low readiness forces 131 49 22 Multinational HQ staff 220 56 21 4 Total 464 183 54 Personnel Policy Directorate

Personnel Policy Directorate EG 0356 Challenges (Cont) BILC and NTG concerns Defense Council of the Republic of Bulgaria – discussed the issue of high language requirements; an offer to be forwarded to the NTG JSSG for more realistic requirements Local Commanders and Human Resource Offices Involvement in the process of appointment to positions Personnel Policy Directorate

Responding to the Challenge ELT in the units to enhance language capabilities on-site Redistribution of lab equipment and ALC textbooks to key units Procurement of computers for Distance Learning at key units Personnel Policy Directorate

Thank you for your attention ??? Personnel Policy Directorate