1. (Standardizing the Standards of Teaching and Testing in the Military)
Ministry of Defense
    
The Bulgarian Strategy for Developing English Language Training and Testing ( )
IN ACTION
(Standardizing the Standards of Teaching and Testing in the Military)
Personnel Policy Directorate

2. Personnel Policy Directorate

3. Presentation Overview
    
Legal basis for implementing the ELT and testing strategy for the Bulgarian Armed Forces (BAF), 2006 to 2010
Standardization of STANAG 6001 testing for the BAF
Challenges to implementing the Force Goal EG 0356
Personnel Policy Directorate

4. Legal Basis Documentation
    
Strategy for ELT and testing for the Bulgarian Armed Forces (BAF), 2006 to 2010
Ministerial Order establishing
Steering group
Working groups
Commission for oversight of ALCPT usage
Identification of Test Control Officers
Guidance on administering of ALCPT
Personnel Policy Directorate

5. Personnel Policy Directorate
Legal Basis (Cont)
    
Order of the Chief of the GS for relocation of IMET Labs
Plan for implementation of the Strategy, 2007 to 2010, endorsed by Deputy Minister
Ministerial order on procedures for ALCPT placement, progress and final test and Book Quiz administration
Selection criteria
Pre-testing
Entry, inter-mediate and final ALCPTs
Personnel Policy Directorate

6. Activities of Working Groups
Curricula for General Military English courses: developed and trialed
Curricula for Specialized English courses: being developed
Curricula for Military English courses for Cadets and graduate students: developed
Projects for SACs: being developed
Distance learning courses for ELT: 24-week 2nd Level course being piloted and two 6-week refresher courses for 3rd and 4th levels being developed. [demo clip]
    
Personnel Policy Directorate

7. Trained Military and Civilian Personnel Bulgarian Armed Forces
Trainees in ELT in 2006
Training Structures
Trained Military and Civilian Personnel
Total
MoD
Bulgarian Armed Forces
Officers
NCOs
Civilians
Enlisted
Military Schools 41 10 3
401
154 2 29
640
Military Units
217 72 6 36
331
Private Organizations 13 18 54 12 21
621
226 8 65
1007
Personnel Policy Directorate

8. Personnel Policy Directorate
Equipment for ELT
Language Laboratories: 33 (20 in the military academies and 13 in the military units) with approximately 500 lab positions
IMET: 17 Labs
MoD funded: 13 Labs
Government of Netherlands: 3 for military units
$500,000 from FMF program for upgrade and expansion of IMET labs in military academies
Self Access Centers: 4 (3 from the PEP Project and 1 MoD funded)
Personnel Policy Directorate

9. Personnel Policy Directorate
Training Materials
ALC materials and tests available in all ELT institutions
Surplus ALC materials redistributed to military bases
$750,000 from FMF program for ALC materials and tests; 1st shipments received in March/April
Materials previously provided through the PEP program
Materials previously and currently provided from MoD
Personnel Policy Directorate

10. Language and Information Technology Training Center (LITTC) in Shumen
Established in September 2005
First courses trialed in April 2006
Opening in September 2006
Formal dedication in September 2007
Number of students (2006): 174
Number of students (2007): 213 (current-103/ projected in Sept )
Personnel Policy Directorate

11. Personnel Policy Directorate
LITTC
Center for basic ELT in BAF – Levels1, 2 and 3 courses
Staff
Director: Land Forces officer
Civilian teachers of English: 25
IT teachers: 5
Administrative staff: 5
Personnel Policy Directorate

12. Personnel Policy Directorate
LITTC (cont)
Level II 24-position labs: 3
Level IV 15-position labs: 3
SAC with 12 positions
ELT classrooms: 15
Instructor offices: 11
Dormitory rooms for 300 students under renovation (2007 MoD funded)
Renovated dining facility (2006 MoD funded)
Athletic facilities to be renovated as a future MoD funded project in 2008
Personnel Policy Directorate

13. Standardizing the Standards in STANAG 6001 Testing
Perspectives
In-country
Regionally
Internationally
Personnel Policy Directorate

14. In-country Standardization
Independent testing body (professional testers)
Standardized criteria
Standardized test documentation
Standardized test format
Standardized administration procedure
Standardized scoring procedures
High test reliability and validity
High inter-rater reliability
Personnel Policy Directorate

15. Personnel Policy Directorate
Test Reliability
Reading
Mean
Median
Mode
St.D.
Alpha
No items
No cases A
31.846
33.00
35.00
7.867
.8630 61 26 B
28.24
29.00
29; 33;36
8.576
.9017 29 C
40.91
40.50
39.00
8.11
.8837 22
LISTENING
Mean
Median
Mode
St.D.
Alpha
No items
No cases A
25.588
24.00
23.00
10.896
.9326 56 34 B
22.61
22.50
28.00
8.899
.9076 18 C
30.20
33.00
10.56
.9268 25
Personnel Policy Directorate

16. Latest Statistics for Reading
45,0
40,0
35,0
30,0
25,0
20,0
15,0
Reading A 8 6 4 2
Std. Dev = 7,87
Mean = 31,8
N = 26,00
Personnel Policy Directorate

17. Latest Statistics for Listening
TOTLISTC
55,0
50,0
45,0
40,0
35,0
30,0
25,0
20,0
15,0
Total Listening C 6 5 4 3 2 1
Std. Dev = 10,56
Mean = 30,2
N = 25,00
Personnel Policy Directorate

18. Inter-rater Reliability
RATER1 Pearson Correlation
Sig. (2-tailed) N 1 . 10
,811 **
,004
**. Correlation is significant at the 0.01 level (2-tailed).
Personnel Policy Directorate

19. Regional Standardization
Piloting new Bulgarian test versions in the region: Albania, Croatia, Estonia, Hungary, Slovakia, Slovenia
Piloting other countries’ STANAG tests in Bulgaria: Croatia, Serbia, Slovakia, Slovenia
Bilateral relations: visit to Romanian MoD/ HRMD in March’07
Opportunity to discuss testing issues and further promote standardization
Personnel Policy Directorate

20. International Standardization
Within BILC
Consultants from DLIELC (U.S), Canada, the Netherlands
The BAT work group: how nations apply the STANAG 6001, opportunity to tune up national efforts
Personnel Policy Directorate

21. International Standardization (Cont)
Outside BILC
EALTA/ILTA conferences, seminars, colloquia correlating our efforts to the world leading testing experience
Personnel Policy Directorate

22. Personnel Policy Directorate
Personnel Tested in 2006
Language
Personnel tested
failed
passed
1111
1-2
2222
2-3
3333
3-4
4444
English
1200
240
960
80 % 41 4%
407
42%
131
17%
286
30% 88 9% 7 1%
More than 54% of those tested achieved L2 and higher
Personnel Policy Directorate

23. Language Proficiency IAW STANAG 6001
Total
English
466
1849
825
272 86 4
3969
Personnel Policy Directorate

24. Current STANAG Session April/May 2007
Applicants
L 1
L 1-2
L 2
L 2-3
L 3
236 2 76 66 59 33
More than 67% of those tested achieved L2 and higher
Personnel Policy Directorate

25. Personnel Policy Directorate
EG 0356 Challenges
Ministerial order No OX – 59/ , List of key positions in the MoD and BAF: (4% met the EG 0356 requirements)
In 2005: personnel received ELT of which 753 from the key positions
In 2006: personnel received ELT of which 705 from the key positions
Current status of meeting the EG 0356 requirements
15 % HRF;
12 % LRF;
74 % MHQ
Personnel Policy Directorate

26. Trainees from Key Positions (2006 by Category)
Officers
NCOs
Enlisted
Civilians
High readiness forces
113 78 11
Low readiness forces
131 49 22
Multinational HQ staff
220 56 21 4
Total
464
183 54
Personnel Policy Directorate

27. Personnel Policy Directorate
EG 0356 Challenges (Cont)
BILC and NTG concerns
Defense Council of the Republic of Bulgaria – discussed the issue of high language requirements; an offer to be forwarded to the NTG JSSG for more realistic requirements
Local Commanders and Human Resource Offices Involvement in the process of appointment to positions
Personnel Policy Directorate

28. Responding to the Challenge
ELT in the units to enhance language capabilities on-site
Redistribution of lab equipment and ALC textbooks to key units
Procurement of computers for Distance Learning at key units
Personnel Policy Directorate

29. Thank you for your attention ???
Personnel Policy Directorate