Introduction to Value-Added Data Dr Robert Clark

Theory and Stats bits…

. Trend Line/Regression Line Measuring Value-Added – Terminology BASELINE SCORE -ve VA +ve VA Residuals VA Exam grade

Measuring Value-Added – An Example Low AbilityAverage AbilityHigh Ability Baseline Score A* U B C D E F G Result Aldwulf Beowulf Cuthbert +ve (+ 2 grades) -ve (- 2 grades) National Trend ‘Average’ Student The position of the national trend line is of critical importance Subject A Subject B

Some Subjects are More Equal than Others…. A-Level >1 grade A*ABC A B C D E

Burning Question : What is my Value-Added Score ? Better Question : Is it Important ?

Value Added Charts Pre 16

Performance inline with expectation VA Score Performance below expectation Problem with Teaching & Learning ? Performance above expectation Good Practice to Share ?

Which Subjects Cause Most Concern ? Danger of Relying on Raw Residuals Without Confidence Limits

Which subjects now cause most concern ?

Business Studies

Religious Studies

Value Added Charts Post 16

SPC Chart Year Performance inline with expectation VA Score Performance below expectation Problem with Teaching & Learning ? Performance above expectation Good Practice to Share ?

Subject Summary - 3 Year Average Subject Summary - Current Year

A2-English Literature Statistical Process Control (SPC) Chart Year

Student Level Residuals (SLR) Report Scatter Plot A2 – English Literature General Underachievement ?

Student Level Residuals (SLR) Report Scatter Plot A2 – English Literature Too many U’s ?

Other things to look for… Why did these students do so badly ? Why did this student do so well ? How did they do in their other subjects ?

Summary of Process Examine Subject Summary Determine ‘interesting’ (i.e. statistically significant) subjects Look at 3 year average as well as single year if available Look at trends in ‘Interesting Subjects’ Examine student data –Scatter graphs Identify students over / under achieving (student list or Paris) Any known issues ? Don’t forget to look at over achieving subjects as well as under achieving Phone / CEM when you need help understanding / interpreting the data / statistics !

Baseline Choice

Do students with the same GCSE score from feeder schools with differing value-added have the same ability ? How can you tell if a student has underachieved at GCSE and thus can you maximise their potential ? Has a student got very good GCSE scores through the school effort rather than their ability alone ? Does school GCSE Value-Added limit the ability to add value at KS5 ? Can you add value at every Key Stage ? How can you check for this ? GCSE or Baseline Test ?

The Effect of Prior Value Added Beyond Expectation +ve Value-Added In line with Expectation 0 Value-Added Below Expectation -ve Value-Added Average GCSE = 6 Do these 3 students all have the same ability ?

Same School - Spot the Difference ? GCSE as Baseline Test as Baseline

National or School Type Specific ?

Comparison to all schools Comparison to Independent Schools Only

Comparison to all schools Comparison to FE Colleges Only

Questions: → How does the unit of comparison used affect the Value-Added data and what implications does this have on your understanding of performance ? → Does this have implications for Self Evaluation ?

Thank You Robert Clark –

Definitions: Residual – difference between the points the student attains and points attained on average by students from the CEM cohort with a similar ability Standardised Residual – the residual adjusted to remove differences between qualification points scales and for statistical purposes Average Standardised Residual – this is the ‘Value Added Score’ for any group of results Subject VA – average of standardised residuals for all students’ results in the particular subject School VA – average of standardised residuals for all students’ results in all subjects for a school / college Confidence Limit – area of statistical uncertainity within which any variation from 0 is deemed ‘acceptable’ and outside of which could be deemed ‘important’