1. Analysing test properties using the EEF data archive
Becky Allen

2. Major commercial tests in the database
Number of schools
Number of pupils tested
PiE GL Assessment (English)
419
23254
NGRT GL Assessment (Reading)
413
17046
PiM GL Assessment (Maths)
208
18208
InCAS CEM
120
11243
BPVS GL Assessment (Vocabulary)
118
3002
WRMT GL Assessment (Reading) 67
762
ACE GL Assessment (Spoken language) 65
3316
Basic Number Screening Test Hodder (Maths) 54
318
PIRA Hodder (Reading) 51
2240
CAT4 GL Assessment (Cognitive Ability) 48
3455

3. Research questions
Which tests (government and commercial) should EEF favour for RCT designs because they have good predictive properties of outcomes in another major test (frequently the KS2 or GCSE)?
How do our estimated properties of the tests relate to information provided by the commercial test providers?
Which tests are particularly well suited to key pupil groups, such as low attainers, pupil premium or EAL students, younger students?
How should we design trials that can most efficiently exploit the properties of the available commercial tests?

4. Limitations of the study
The individual test item data are not held so we are limited to looking at the validity of measurement at the level of the pupil in the test’s subject domain
Schools in database highly non-random sample of all English schools
Half the schools in the database have been ‘treated’ in RCTs, which will affect the behaviour of the test scores

5. Planned outputs
What is the predictive validity of each of these commercial tests? Output – correlation grids for unweighted and weighted sample of pupils and for major sub-groups
What is within versus between school variation in test outcomes? Output – intra-class correlation grids
Can we observe the test behaving in an usual way with respect to pupil demographic characteristics (FSM, gender, EAL and month of birth)? Output – test statistic distributions by demographic characteristic