Using CEM Data in Practice: Feedback from CEM Secondary Assessments Peter Hendry: CEM Consultant Using CEM Data in Practice: Feedback from CEM Secondary Assessments Glasgow Conference 13th Feb 2013 Peter.Hendry@cem.dur.ac.uk Secondary@cem.dur.ac.uk

Assessment for Excellence: 2 systems S1/S2 Baseline Assessment (MidYIS): A Test of Developed Ability at start of S1 or S2 4 sections: Vocabulary, Maths, Non-Verbal and Skills (Proof Reading, and Perceptual speed and accuracy Computer adaptive* or paper-based Any time Term 1 + catch ups Value Added measures to SOSCA and to Scottish Qualifications S2 Curriculum-based Assessments (SOSCA): A curriculum assessment of knowledge, skills and understanding at end of S2 Assesses what has been taught in the classroom Covers, Maths, Science, Reading (English) Computer adaptive* Four week assessment window mid April to mid May + catch ups Value Added measures from PIPs, InCAS and MidYIS and to Scottish Qualifications Assessment for Excellence – 2 systems – MidYIS in S1 or S2 SOSCA at end S2 – this is the focus of this presentation Reading is a proxy for English – can’t manage the writing execise or the marking but reading is an excellent proxy for English and correlates very well with performance in English 2

AfE: key elements School and students’ test performances are compared to a nationally representative sample Assessment data from the tests provides school and pupil data to inform teaching and learning ‘Predictive’ data indicates future potential performance to SQ, and will to National 4 and 5, to inform target setting and subsequent monitoring ‘Value-added data’ provides measures of relative performance at SQ (and will for National 4 and 5) and SOSCA from earlier baselines to inform self-evaluation

AfE Feedback: MidYIS and SOSCA Assessment Feedback: Standardised score system: 50-150, mean 100, SD 15 School data including component averages and band profiles (historical and current) Pupil data in the form of spreadsheets and Individual Pupil Records (IPRs)

AfE Feedback: MidYIS and SOSCA ‘Predictions’ Feedback: Currently to S2 SOSCA from S1/S2 baseline (MidYIS) and to SQ from MidYIS and SOSCA ‘Value-Added’ Feedback: To SQ from curriculum baseline (SOSCA), and from S1/S2 baseline (MidYIS) To S2 curriculum assessment (SOSCA) from P7 baseline (PIPS or InCAS) and/or S1/S2 baseline (MidYIS)

www.cemcentre.org

Detailed Information about the assessment. Well worth a look. Data for your school.

Mean 100, Standard Deviation of 15 Bands, percentiles, standardised scores… 25% 25% C B 25% 25% D A Standardised scores Percentiles: 10 5 20 30 1 40 90 95 80 70 99 60 50 98 Mean 100, Standard Deviation of 15

SCHOOL A: S1 baseline data

SCHOOL A: S1 baseline data Intake profiles: overall and test component bands A school with a ‘completely average’ intake would have 25% of pupils within each band.

SCHOOL A: S1 Baseline

SCHOOL A

SCHOOL A: S2 curriculum (SOSCA) data

School no.1 School no.2

STANINES

S1 baseline IPR

S2 curriculum (SOSCA) IPR

S2 curriculum (SOSCA) IPR

Predictions Feedback Currently available to Standard Grade from AfE baselines In the future to National 4 & 5 Qualifications from AfE baselines Also to AfE S2 curriculum (SOSCA) from S1 and/or S2 baseline (MidYIS) Includes Chances Graphs

How is a ‘prediction’ generated? 50% on or above the trend line 1 Trend line (regression line) *********************** 2 ******************************* 3 ******************************************* ‘PREDICTION’ (expected grade) 4 GRADE ************************************* GRADE ************************************** ******************************** ***************************** 50% on or below the trend line ***************** BASELINE SCORE

Data for 2012 examinations Grade

Predictions: Chances Graphs 50% chance of a grade 2 – the most likely single grade. 50% chance of a different grade Point Prediction =1.8 Grade Prediction = 2 Chances Graphs based on Pupil’s SOSCA Test Score 25

Not a label for life ...just another piece of information The Chances graphs show that, from almost any baseline score, students come up with almost any grade - - -there are just different probabilities for each grade depending on the baseline score. In working with students these graphs are more useful than a single predicted or target grade Chances graphs show what can be achieved: By students of similar ability By students with lower baseline scores

Latest information as at 13 Feb 2102 Replacing SG with Nat 4 & 5 predictions: CEM statisticians have put together a proposal. This has been looked at by colleagues in Scottish Authorities. The CEM Technical team have now published the new predictive data. This data is now available on the CEM secondary+ website alongside the existing SG predictive data

New slide!!

Measuring Value-Added – Terminology . Measuring Value-Added – Terminology Exam grade +ve VA -ve VA 0 2 4 6 8 BASELINE SCORE Raw Residual Trend Line/Regression Line

SCHOOL A: S2 curriculum (SOSCA) data

SCHOOL C: AfE data

Standardised Residual Bar Chart SCHOOL C: AfE data Standardised Residual Bar Chart 99.7% confidence limit 95% confidence limit ANY VALUE IN THE INNER SHADED AREA SHOWS THE EXPECTED RESULTS i.e. AVERAGE VALUE ADDED

SCHOOL C: SQ data