Action Plans for Test Development and Administration STANAG 6001 BILC Conference, Mons, SHAPE, Belgium, September 2013 Ludmila Ianovici-Pascal Head, Language Center, Republic of Moldova

AGENDA 1. ACTION PLAN 2. DEVELOPMENT STAGE 3. ADMINISTRATION STAGE 4. CONCLUSION

Why develop an AP to lend credibility to be sure not to forget any details to understand the boundaries for accountability for efficiency

What is an AP …the way to use strategies to meet its objectives … and consists of a number of steps: what actions/changes will occur who will carry out these changes by when they will take place & how long what resources communication

General Criteria for an AP COMPLETE? CLEAR? CURRENT?

When to create an AP within 6 months to one year VISION (the dream) MISSION (the what and why) OBJECTIVES (how much will be accomplished by when) STRATEGIES (the how) ACTION PLANS (what changes will happen; who will do what by when to make it happen) VMOSA grounds our plans REMEMBER AP is always a WORK IN PROGRESS !!!

The resources needed to carry out an Action Plan: PEOPLE TIME SPACE EQUIPMENT Resources

planning to do too much in a short time not planning activities in enough detail not working out time-lines to make sure that scheduling makes sense not making it absolutely clear who has responsibility and authority for making sure that all the steps get done not thinking rigorously through the resourcing implications Some Issues to consider

the ongoing assessment of how an organization is performing against its action plans Monitoring addresses questions such as: Are outputs being achieved within the timeframes set? Are resources being efficiently and effectively used? Are we doing what we said we would do and if not, why not? Are work units meeting their objectives? Are individuals meeting their objectives? Monitoring

Test Construction Very detailed and important phase STAGES: Needs Analysis Planning Test Specifications Development Test Production Administration

Test Specifications Vital segment of the test development life cycle Should explain "how" to implement the test cases TEST SPECIFICATIONS - GUIDE for writing items Skills tested Levels measured Number of items Type of items Format Timing

Test development procedures Table of specification Test construct (content, task, accuracy) clear o everyone Iterative development process Authenticity Texts & tasks appropriate for the level being tested Level of the task equal to level of the text Limited number of tasks/items per text/level Schedule of updating & maintaining an item pool Reusing items

Development Stage Text selection Item writing Item review I (moderation) Small-scale pre-testing Item review II (editing) Assembly of trial test Large-scale pre-testing Statistical analysis Item revision Item banking Assembly of final version of test

Item Development The item is the most basic building block of any test The most extensive and time-consuming step is ID Iterative development process Result is an item bank Items are worthy The best item is the one that provides the most direct means of measuring the desired learning outcome

Test Administration Things to Know About Test Administration very important source of validity evidence test validity evidence can be seriously compromised by faulty administration threats to test security maximum exposure of items numerous details must be rigorously attended by competent, experienced test-administration staff detailed documents

Formal testing policy Policy approved by MoD or higher authority Mission statement with purpose of testing Standards to be used (STANAG 6001) Formal testing schedule List of employees & their duties Job qualifications Certification of testing staff

Written TA Procedures Test security policy Code of Fair Testing or Code of Ethics Students’ Guide Tester’s Guide STANAG 6001 Briefing Formal schedule published Clear instructions Questionnaires Candidate Comment Form

Conclusions increases efficiency facilitates proper coordination aids in organizing all available resources gives right direction maintains a good control helps to achieve objectives motivates the personnel encourages creativity and innovation helps in decision making