BILC Standardization Initiatives and Conference Objectives Peggy Garza Associate BILC Secretary For Testing Programs

Outline Standardization Efforts Conference Topics and Goals - Language Testing Seminars - Tailored Workshops - Toolbox for Language Testers - STANAG 6001 Testing Conferences Conference Topics and Goals

BILC Testing Seminars Language Testing Seminar (LTS) Advanced Language Testing Seminar (ALTS) Language Standards and Assessment Seminar (LSAS)

LANGUAGE TESTING SEMINAR (LTS)

Language Testing Seminar (LTS) Developed and conducted by BILC testing experts Background: A BILC initiative aimed at promoting a common interpretation of STANAG 6001 and consistency in language proficiency testing Specific objective: To give NATO/PfP language teaching professionals the opportunity to practice developing test items based on STANAG 6001

Language Testing Seminar (LTS) Held since Autumn 2000 at the Marshall Center, Garmisch, Germany Two-week graduate-level seminar Offered twice times a year (February and September) More than 350 graduates from NATO/PfP/MD nations

ADVANCED LANGUAGE TESTING SEMINAR (ALTS)

Advanced Language Testing Seminar (ALTS) Need for continuing professional development for test developers identified Stakeholders’ meeting in September 2008 Result: Advanced Language Testing Seminar (ALTS) Validated in 2009

Advanced Language Testing Seminar (ALTS) Target audience: LTS graduates who continue to work on STANAG 6001 testing - STANAG 6001 Edition 4 standardization sessions - Focus on Level 3 for practical exercises - Opportunity to exchange best practices Modular approach to seminar

Advanced Language Testing Seminar (ALTS) Pre-seminar assignments - Approximately 40 hours Three-week seminar - Module 1: Receptive Skills: Listening and Reading (1 week) - Module 2: Productive Skills: Speaking and Writing (1 week) - Module 3: Test Analysis (3 days) - Module 4: Managerial Issues (2 days)

NEW LANGUAGE STANDARDS AND ASSESSMENT SEMINAR (LSAS)

Language Standards and Assessment Seminar (LSAS) 3-11 July 2013 For language teachers and program managers Objectives STANAG 6001 familiarization Guidelines for aligning instruction to standards Test and measurement principles Classroom assessment and practical exercises

Testing Seminar Schedule in 2014 Language Testing Seminar (LTS) 10-21 February 15-26 September Advanced Language Testing Seminar (ALTS) 2- 20 June Language Standards and Assessment Seminar (LSAS) 23 Apr – 1 May 3-11 December

STANAG 6001 Toolbox for Testers Recommended by ALTS participants Samples of listening and reading items at levels 1-3 Samples of speaking and writing tests at levels 1-3, including plus levels Frequently Asked Questions (FAQs) Self-Assessment Outline for STANAG 6001 Testing Programs

BILC Website www.natobilc.org

Tailored Workshops for National Testing Programs

Tailored Workshops Ukraine Moldova Russia November 2011, 2012 May 2012 June 2013 19

STANAG 6001 Testing Conferences Zagreb, 2009 Focus on standardization, moderation, and validation Sarajevo, 2010 Dates: 31 August – 2 September Focus on testing speaking Stockholm, 2011 Dates: 6-8 September Focus on testing writing Copenhagen, 2012 Dates: 3-6 September, 2012 Focus on testing listening

2013 STANAG 6001 Testing Conference Some of your questions: Our stakeholders have questions. Is there advice for how best to communicate with the stakeholders? Our stakeholders are asking us to test more and more people. What should we tell them? We have developed items. What’s next? How do we know if our tests are good? We need to hire new testers. Is there a position description for testers? How can we recruit for testers? How can we keep professionally current ?

2013 STANAG 6001 Testing Conference Conference format Presentations Guest speakers from NATO and SHAPE Panel discussions Resolving challenges Study Groups meetings and reports Making a validity argument Washback Action plans Renorming and standardizing testers Piloting items and prompts

Specific Conference Goals Review and clarify the entire test construction process Identify action plans for test development and administration Provide ideas for selecting the appropriate testing techniques Discuss piloting and pretesting Learn more about determining test validity Identify practicality issues and discuss lessons learned Discuss best practices in renorming and standardizing testers Learn about innovations in testing in the BILC community Computer-delivered testing Proficiency and specialist testing Testing at Level 4

Test Life Cycle Diagram

Communicating with the Stakeholders Identify the stakeholders Coordinate with the stakeholders during the needs analysis and planning stages Stakeholders approve personnel and resources Close the loop with the stakeholders during the Test Life Cycle Action plans Technical reports Updates Washback

Test Construction Needs Analysis Planning Design Development 19-4-2017 Needs Analysis Planning Design Development Administration Monitoring Test Specifications Pre-testing / Analysis Standard Setting Evaluation/review Evaluation Revision Stage 2 Stage 3 Stage 4 Stage 1 Stage 5 Stage 6 TKC/OCEde

Development Stage Part 1 19-4-2017 Text selection Item writing Item review I (moderating) Piloting on a small scale Item review II (editing) Assembly of trial test Retain Revise Reject More items needed? TKC/OCEde

Development Stage Part 2 19-4-2017 Pre-testing/trialling on a larger scale Statistical Analyses Standard Setting Item banking Assembly of final version of test Item revision Retain Revise Reject TKC/OCEde

2013 STANAG 6001 Testing Conference Expectations Active participation and interaction Frank and open discussions Provide answers to the difficult questions Create a forum for future networking and cooperation BILC appreciates your participation and we look forward to a very productive conference!