Introduction to Assessment and Monitoring Christine.Merrell@cem.dur.ac.uk www.cemcentre.org

Centre for Evaluation and Monitoring (CEM) Part of Durham University Monitoring Systems 1.1 million assessments delivered each year Pupils aged 3 – 18 years CEM systems used in 44 countries

Why assess? Newton (2007) 18 purposes for which educational judgements may be used Not exhaustive Expanding all the time Interesting perspective on the historical debate about formative and summative assessment

Why assess? From birth ~ weight, hearing etc. Qualifications ~ driving test, GCSEs, degree Profile of strengths and weaknesses for planning appropriate learning experiences Indicator of special educational needs Monitor progress and attitudes of pupils and cohorts over time

Comparisons Progress over time Research Children within a class Groups such as boys/girls Classes within a year-group Current cohorts with previous ones Other schools within a consortium and nationally Progress over time Research Within school Nationally and internationally

Layers of information (different levels of detail): Diagnostic at pupil-level Group and class trends School-level information (including trends over time) Consortium/Authority-level Curriculum-based

Ways to Assess and Cautions Do you know the psychometric properties of the assessments that you use? Are these methods always reliable and valid? Early Years Foundation Stage Profile? Judgements Observations E.g. Behaviour Objective tests E.g. Attainment Practical Pencil and paper Computer-delivered Curriculum-based

Standardised Assessment Administration procedure Comparison against representative norms Curriculum-based

Example of a high-level use of standardised assessment Standards over time Consistent content Consistent sample of schools Merrell, C. and Tymms, P. (2011) Changes in Children’s Cognitive Development at the Start of School in England 2001 – 2008, Oxford Review of Education, Vol. 37 (3), June 2011, p333 – 345. Curriculum-based

Initiatives in the Early Years Foundation Stage Age 3 – 5 years Increased nursery provision Curriculum Assessment Sure Start Education Action Zones Etc. Etc. Have these initiatives changed children’s cognitive development at the start of school?

PIPS On-entry Baseline Assessment Computer delivered assessment of: Vocabulary Phonological awareness & early reading Early mathematics Additional data: Date of birth Sex First Language

Test/Re-test Reliability = 0.98 Internal Reliability (Cronbach’s alpha) = 0.94 Correlation with attainment at age 11 = 0.68

Sample 472 State Schools in England

Background Variables

Mean PIPS T Scores Academic Year Mean total PIPS BLA T score (SD in brackets) 2001 48.67 (9.66) 2002 48.50 (9.77) 2003 48.31 (9.43) 2004 48.34 (9.49) 2005 48.23 (9.47) 2006 48.25 (9.42) 2007 47.84 (9.41) 2008 47.59 (9.22)

Mean PIPS Raw Scores Effect Sizes 0.11 -0.07 Early Reading Early Maths Picture Vocabulary Total Academic Year Mean (max = 170) SD (Max = 69) (Max = 23) Max = 262) 2001 14.32 13.90 22.71 9.21 13.66 4.48 50.69 23.64 2002 13.96 13.44 22.66 9.17 13.53 4.56 50.14 23.29 2003 13.32 13.02 22.80 9.16 5.05 49.64 23.15 2004 13.28 13.03 22.84 9.25 13.70 5.04 49.82 23.27 2005 13.20 13.07 22.38 8.79 13.52 49.10 22.88 2006 13.59 13.93 23.16 9.15 5.11 50.40 23.82 2007 13.24 13.67 23.62 9.22 13.51 50.37 23.57 2008 13.14 13.37 23.74 9.18 13.34 5.08 50.25 23.28 Effect Sizes 0.11 -0.07

Main Effects & Interactions GLMs to analyse changes in BLA scores in relation to: Year EAL Sex Age at test was entered as covariate

Conclusions Statistically significant decrease from 2001 to 2008 for early reading and picture vocabulary. However, the effect sizes of the differences were small. Significant increase in early maths scores although again the effect size was small.

Why Should the BLA Scores Remain So Stable Over Time? Success from early interventions aimed at influencing the development of young children is difficult to achieve Reduction in efficacy of small-scale programmes when rolled out Limitations with data analysed but nevertheless this is a large dataset that adds to current studies of trends over time

Thank you Christine.Merrell@cem.dur.ac.uk