The Learner Achievement Tracker (LAT) Analysing the Reports for School Improvement Ben Fuller Assessment Adviser, Standards & School Effectiveness January 2006

What is the Learner Achievement Tracker? Developed to coincide with the New Measures of Success – part of ‘Success for All’ Four partners: LSC, DfES, ALI, OFSTED ‘Basket’ of measures to contextualise performance of all providers of post-16 education Application across the post-16 sector No additional bureaucratic burden LAT will be accessed through the Provider Quality Gateway at https://gateway.lsc.gov.uk

LAT Output reports Type of LAT report What the report shows Summary Charts (VA and DT) Overview of performance across different qualifications and across different subjects within a given qualification type compared to the national average National comparisons graph Performance in a given qualification/subject compared to the national average across the full range of prior attainment National chances charts Chances of a learner with a given prior attainment achieving a particular outcome

Value Added and Distance Travelled for 16 – 19 learners Measures the progression in qualifications of 16-19 learners based on their prior attainment ie, how well do learners achieve post-16, based on their achievement at Key Stage 4? Covers the majority of larger, approved qualifications taken by 16-19 learners, including vocational and ungraded qualifications

Value Added and Distance Travelled for 16 – 19 learners VA for 16-19 learners will measure achievement in Level 3 graded qualifications, for example: GCE A and AS level AVCE BTEC National Diploma DT for 16-19 learners will include other approved qualifications at levels 1, 2 and 3 for example: NVQ Foundation and Intermediate GNVQ GCSE

VA and DT data Data used to calculate the VA and DT for 16 – 19 learners is gained directly from the the awarding bodies VA score is average difference (in QCA points) between students’ results and national averages for each level of prior attainment Providers will not be involved in collecting data for the measure Providers check the accuracy of the data in the usual way (LAT first view based on unvalidated data, then amended later)

Summary Charts of VA and DT (institution-level data) LAT Output Reports Summary Charts of VA and DT (institution-level data)

Summary Charts VA Shows the VA scores for each qualification type and an overall institutional VA score For each qualification type, a chart shows the VA scores for each subject (provided >4 students) Page 17 of the Quality Improvement Pack DT Shows the DT scores for each qualification type For each qualification type, a chart shows the DT scores for each subject group (provided >4 students) Page 30 of the Quality Improvement Pack

Example of a Value Added Summary Across Qualifications Significant when shaded grey.

Example of a Value Added Summary within Qualification Type 95%

Application of the Value Added Summary Charts First indication of performance across the full range of prior attainment Identifies strengths and areas for improvement (LAs and Inspectors) - a view of which schools/ institutions are performing better/worse than national average (given prior attainment)

Key Questions Are there types of course/qualification in which performance has been particularly strong or weak? Are there particular subjects in which performance has been particularly strong or weak? Given their prior attainment, are learners being matched appropriately to programmes of learning?

2. National Comparison Graphs (VA) LAT Output Reports 2. National Comparison Graphs (VA) (pupil level data)

National Comparison Graphs Show the performance of the provider in a given qualification compared to the national performance in the same qualification across the full range of prior attainment For example, NVQ Level 3 in construction or A Level History Pages 21 and 34 of the Quality Improvement Pack

Example of a Value Added National Comparisons Graph School’s line of best-fit and shaded area (confidence interval) is entirely below national line – therefore significantly below at all levels of prior attainment. 95% Fail Pass

Interpretation of VA lines Provider sig. above national.

Interpretation of VA lines Provider not sig above naional.

Interpretation of VA lines Provider sig. below national.

Interpretation of VA lines Provider sig. below national. Provider not sig below.

Interpretation of VA lines Provider sig above national for students with higher prior attainment, but sig below for lower PA.

Interpretation of VA lines Not sig.

National Comparison Graph (Distance Travelled)

Application of the Value Added National Comparisons Graph Are learners at this institution making better or worse progress than would be expected when compared to the national picture? Do learners with a given prior attainment (for example a low or high prior attainment) make better or worse progress than expected? Have any learners achieved higher or lower than expected? What levels of achievement/target grades can be expected from current cohort of learners given prior attainment?

3. National Chances Charts LAT Output Reports 3. National Chances Charts

National Chances Charts Show the chances of a learner with a given prior attainment achieving a particular outcome in a given qualification at a nationally average institution For example, the chances of a learner with a given prior attainment achieving grades A – E in A Level History or achieving a Level 1, 2 or 3 NVQ in construction Pages 27 and 40 of the Quality Improvement Pack

Example of a Value Added National Chances Chart (Fails not included) % based on nat. data for the chosen PA band.

Application of the Value Added National Chances Chart VA chances charts can be used to support learner progress by helping: Target setting To monitor learner progress To identify where support is required To identify appropriate courses for individual learners

Issues for further exploration Learners with missing or no prior attainment Context variables Customising the LAT Treatment of AS levels in the VA measure Treatment of fails in the VA measure Use of success rate for DT

Further Information The Quality Improvement Pack contains detailed information about the interpretation and application of each type of LAT reports www.lsc.gov.uk/nms - ‘Measuring Success’ newsletters downloadable www.successforall.gov.uk VADT@lsc.gov.uk