1. BILC Standardization Initiatives and Conference Objectives
Peggy Garza
Associate BILC Secretary
For Testing Programs

2. Outline Standardization Efforts Conference Topics and Goals
- Language Testing Seminars
- Tailored Workshops
- Toolbox for Language Testers
- STANAG 6001 Testing Conferences
Conference Topics and Goals

3. BILC Testing Seminars Language Testing Seminar (LTS)
Advanced Language Testing Seminar (ALTS)
Language Standards and Assessment Seminar (LSAS)

4. LANGUAGE TESTING SEMINAR
(LTS)

5. Language Testing Seminar (LTS)
Developed and conducted by BILC testing experts
Background: A BILC initiative aimed at promoting a common interpretation of STANAG 6001 and consistency in language proficiency testing
Specific objective: To give NATO/PfP language teaching professionals the opportunity to practice developing test items based on STANAG 6001

6. Language Testing Seminar (LTS)
Held since Autumn 2000 at the Marshall Center, Garmisch, Germany
Two-week graduate-level seminar
Offered twice times a year (February and September)
More than 350 graduates from NATO/PfP/MD nations

7. ADVANCED LANGUAGE
TESTING SEMINAR
(ALTS)

8. Advanced Language Testing Seminar (ALTS)
Need for continuing professional development for test developers identified
Stakeholders’ meeting in September 2008
Result: Advanced Language Testing Seminar (ALTS)
Validated in 2009

9. Advanced Language Testing Seminar (ALTS)
Target audience: LTS graduates who continue to work on STANAG 6001 testing
- STANAG 6001 Edition 4 standardization sessions
- Focus on Level 3 for practical exercises
- Opportunity to exchange best practices
Modular approach to seminar

10. Advanced Language Testing Seminar (ALTS)
Pre-seminar assignments
- Approximately 40 hours
Three-week seminar
- Module 1: Receptive Skills: Listening and Reading (1 week)
- Module 2: Productive Skills: Speaking and Writing (1 week)
- Module 3: Test Analysis (3 days)
- Module 4: Managerial Issues (2 days)

11. NEW LANGUAGE STANDARDS AND ASSESSMENT SEMINAR (LSAS)

12. Language Standards and Assessment Seminar (LSAS)
3-11 July 2013
For language teachers and program managers
Objectives
STANAG 6001 familiarization
Guidelines for aligning instruction to standards
Test and measurement principles
Classroom assessment and practical exercises

13. Testing Seminar Schedule in 2014
Language Testing Seminar (LTS)
10-21 February
15-26 September
Advanced Language Testing Seminar (ALTS)
2- 20 June
Language Standards and Assessment Seminar (LSAS)
23 Apr – 1 May
3-11 December

14. STANAG 6001 Toolbox for Testers
Recommended by ALTS participants
Samples of listening and reading items at levels 1-3
Samples of speaking and writing tests at levels 1-3, including plus levels
Frequently Asked Questions (FAQs)
Self-Assessment Outline for STANAG Testing Programs

15. BILC Website

18. Tailored Workshops for National Testing Programs

19. Tailored Workshops Ukraine Moldova Russia November 2011, 2012 May 2012
June 2013 19

20. STANAG 6001 Testing Conferences
Zagreb, 2009
Focus on standardization, moderation, and validation
Sarajevo, 2010
Dates: 31 August – 2 September
Focus on testing speaking
Stockholm, 2011
Dates: 6-8 September
Focus on testing writing
Copenhagen, 2012
Dates: 3-6 September, 2012
Focus on testing listening

21. 2013 STANAG 6001 Testing Conference
Some of your questions:
Our stakeholders have questions. Is there advice for how best to communicate with the stakeholders?
Our stakeholders are asking us to test more and more people. What should we tell them?
We have developed items. What’s next?
How do we know if our tests are good?
We need to hire new testers. Is there a position description for testers? How can we recruit for testers?
How can we keep professionally current ?

22. 2013 STANAG 6001 Testing Conference
Conference format
Presentations
Guest speakers from NATO and SHAPE
Panel discussions
Resolving challenges
Study Groups meetings and reports
Making a validity argument
Washback
Action plans
Renorming and standardizing testers
Piloting items and prompts

23. Specific Conference Goals
Review and clarify the entire test construction process
Identify action plans for test development and administration
Provide ideas for selecting the appropriate testing techniques
Discuss piloting and pretesting
Learn more about determining test validity
Identify practicality issues and discuss lessons learned
Discuss best practices in renorming and standardizing testers
Learn about innovations in testing in the BILC community
Computer-delivered testing
Proficiency and specialist testing
Testing at Level 4

24. Test Life Cycle Diagram

25. Communicating with the Stakeholders
Identify the stakeholders
Coordinate with the stakeholders during the needs analysis and planning stages
Stakeholders approve personnel and resources
Close the loop with the stakeholders during the Test Life Cycle
Action plans
Technical reports
Updates
Washback

26. Test Construction Needs Analysis Planning Design Development
Needs Analysis
Planning
Design
Development
Administration
Monitoring
Test Specifications
Pre-testing / Analysis
Standard Setting
Evaluation/review
Evaluation
Revision
Stage 2
Stage 3
Stage 4
Stage 1
Stage 5
Stage 6
TKC/OCEde

27. Development Stage Part 1
Text selection
Item writing
Item review I (moderating)
Piloting on a small scale
Item review II (editing)
Assembly of trial test
Retain
Revise
Reject
More items needed?
TKC/OCEde

28. Development Stage Part 2
Pre-testing/trialling on a larger scale
Statistical Analyses
Standard Setting
Item banking
Assembly of final version of test
Item revision
Retain
Revise
Reject
TKC/OCEde

29. 2013 STANAG 6001 Testing Conference
Expectations
Active participation and interaction
Frank and open discussions
Provide answers to the difficult questions
Create a forum for future networking and cooperation
BILC appreciates your participation and we look forward to a very productive conference!