1. Introduction to Assessment and Monitoring

2. Centre for Evaluation and Monitoring (CEM)
Part of Durham University
Monitoring Systems
1.1 million assessments delivered each year
Pupils aged 3 – 18 years
CEM systems used in 44 countries

3. Why assess?
Newton (2007) 18 purposes for which educational judgements may be used
Not exhaustive
Expanding all the time
Interesting perspective on the historical debate about formative and summative assessment

4. Why assess? From birth ~ weight, hearing etc.
Qualifications ~ driving test, GCSEs, degree
Profile of strengths and weaknesses for planning appropriate learning experiences
Indicator of special educational needs
Monitor progress and attitudes of pupils and cohorts over time

5. Comparisons Progress over time Research Children within a class
Groups such as boys/girls
Classes within a year-group
Current cohorts with previous ones
Other schools within a consortium and nationally
Progress over time
Research
Within school
Nationally and internationally

6. Layers of information (different levels of detail):
Diagnostic at pupil-level
Group and class trends
School-level information (including trends over time)
Consortium/Authority-level
Curriculum-based

7. Ways to Assess and Cautions
Do you know the psychometric properties of the assessments that you use?
Are these methods always reliable and valid?
Early Years Foundation Stage Profile?
Judgements
Observations
E.g. Behaviour
Objective tests
E.g. Attainment
Practical
Pencil and paper
Computer-delivered
Curriculum-based

8. Standardised Assessment
Administration procedure
Comparison against representative norms
Curriculum-based

9. Example of a high-level use of standardised assessment
Standards over time
Consistent content
Consistent sample of schools
Merrell, C. and Tymms, P. (2011) Changes in Children’s Cognitive Development at the Start of School in England 2001 – 2008, Oxford Review of Education, Vol. 37 (3), June 2011, p333 – 345.
Curriculum-based

10. Initiatives in the Early Years
Foundation Stage
Age 3 – 5 years
Increased nursery provision
Curriculum
Assessment
Sure Start
Education Action Zones
Etc. Etc.
Have these initiatives changed
children’s cognitive development
at the start of school?

11. PIPS On-entry Baseline Assessment
Computer delivered assessment of:
Vocabulary
Phonological awareness & early reading
Early mathematics
Additional data:
Date of birth
Sex
First Language

12. Test/Re-test Reliability
= 0.98
Internal Reliability
(Cronbach’s alpha)
= 0.94
Correlation with
attainment at age 11
= 0.68

13. Sample
472 State Schools in England

14. Background Variables

15. Mean PIPS T Scores Academic Year Mean total PIPS BLA T score
(SD in brackets)
2001
48.67 (9.66)
2002
48.50 (9.77)
2003
48.31 (9.43)
2004
48.34 (9.49)
2005
48.23 (9.47)
2006
48.25 (9.42)
2007
47.84 (9.41)
2008
47.59 (9.22)

16. Mean PIPS Raw Scores Effect Sizes 0.11 -0.07 Early Reading Early Maths
Picture Vocabulary
Total
Academic Year
Mean
(max = 170) SD
(Max = 69)
(Max = 23)
Max = 262)
2001
14.32
13.90
22.71
9.21
13.66
4.48
50.69
23.64
2002
13.96
13.44
22.66
9.17
13.53
4.56
50.14
23.29
2003
13.32
13.02
22.80
9.16
5.05
49.64
23.15
2004
13.28
13.03
22.84
9.25
13.70
5.04
49.82
23.27
2005
13.20
13.07
22.38
8.79
13.52
49.10
22.88
2006
13.59
13.93
23.16
9.15
5.11
50.40
23.82
2007
13.24
13.67
23.62
9.22
13.51
50.37
23.57
2008
13.14
13.37
23.74
9.18
13.34
5.08
50.25
23.28
Effect Sizes
0.11
-0.07

17. Main Effects & Interactions
GLMs to analyse changes in BLA scores in relation to:
Year
EAL
Sex
Age at test was entered as covariate

18. Conclusions
Statistically significant decrease from 2001 to 2008 for early reading and picture vocabulary. However, the effect sizes of the differences were small.
Significant increase in early maths scores although again the effect size was small.

19. Why Should the BLA Scores Remain So Stable Over Time?
Success from early interventions aimed at influencing the development of young children is difficult to achieve
Reduction in efficacy of small-scale programmes when rolled out
Limitations with data analysed but nevertheless this is a large dataset that adds to current studies of trends over time

20. Thank you