1. Introduction to the Workshop
Peggy Garza
Associate BILC Secretary

2. Testing the Test – How Valid is Our Validity Argument?
Topics
Validating criterion-referenced tests
Qualitative versus quantitative analysis
Piloting challenges
Test validation partnerships
Inter- and intra-rater reliability
Data analysis tools
Consequential validity
Standard setting
Modified Angoff procedure

3. 23 Nations + 1 NATO HQ Armenia Austria Bosnia and Herzegovina Bulgaria
Canada
Czech Republic
Denmark
Estonia
France
Georgia
Hungary
Latvia
Lithuania
Macedonia
Norway
Poland
Portugal
Serbia
SHAPE
Slovenia
Spain
Sweden
Tunisia
United States of America

4. Facilitators Ray Clifford Mary Jo DiBiase Peggy Garza Roxane Harrison
David Oglesby
Branka Petek
Nancy Powers
Ulle Turk
Jana Vasilj-Begovic

5. Workshop Objectives Share ideas and best practices on test validation
Get answers to questions on test validation
Develop an understanding of how to build a validity argument for national STANAG 6001 tests
Learn about methods and tools for collecting evidence to support a validity argument

6. Workshop Format Plenary sessions Workshop activities Keynote speech
Presentations
Panel of national testing organizations
Workshop activities

7. Workshop Activities Workshops 1&2: 1 ½ hours Workshops 3&4: 2 hours
Workshop 5: 1 hour
Small groups and practical hands-on work
Topics
#1 Reading Item Moderation: Alignment, led by Mary Jo DiBiase, Peggy Garza, Gerd Brendel, and David Oglesby
#2 Piloting: Important Part of Test Validation, led by Branka Petek
#3 Modified Angoff, led by Jana Vasilj-Begovic and Nancy Powers
#4 Statistical Analysis Using jMetrik, led by Ulle Turk
#5 Roadmap toward a Validity Argument led by Mary Jo DiBiase and Peggy Garza

8. What is a Validity Argument?
Glossary definition
Interpretive argument that presents evidence to make a case justifying the score-based inferences and the intended uses of the test.
Is there a better definition?

9. Evidence What do we mean by evidence?
Quantitative
Qualitative

10. My Objective
Obtain input for making improvements to the testing seminars on the topic of test validation