1. Hubble Bubble Some Qualifications Bring Trouble
June 2012
Debbie Grossman
Emma Williams
Planning and Statistics

2. Aim:
To understand the way qualification data is collected, processed and used both within the University and by outside organisations.

3. Qualification Levels GCSE grades G - D GCSE grades A* - C
A Levels, BTECs, International Baccalaureate etc.
HNC
HND, Foundation Degree etc.
Honours degree
Masters Degree
PhD

4. Qualifications recorded
Currently have 231 codes in BANNER
142 Level 3
126 sent to HESA

5. 2010-11 UG New Entrants (Excluding Foundation Degrees)
Task
Highest Qualification
Full time
Part time
Level 4 and above
A Levels
BTECs
Access Course (accredited)
Activity
Think about what you know about the student population and in groups estimate the percentage of new entrants that would be in each of these categories. Note: The qualifications listed are not a complete list therefore the total will not equal 100%

6. 2010-11 New Entrants (Excluding Foundation Degrees)
Values
Highest Qualification
Full time
Part time
Level 4 and above
22.05%
67.74%
A Levels
38.39%
5.97%
BTECs
8.74%
1.83%
Access Course (accredited)
3.94%
0.88%
Activity answers
Opportunity to discuss the answers and how close the teams answers were to the actual values.
Did they come close to peoples perceptions?

7. Applicant Qualifications submitted as part of the application process
L3 and/or above
GCSEs & equivalents from
Professional qualifications/experience
Foundation degrees studied at Greenwich
UCAS
Electronic Applications System
ABL
Non UCAS Applicants
Entered manually by Admissions
Note for Debbie: BOTH go into the Electronic Applications System (EAS) which is an internal system and are then ‘SOATEST-ed’.
Some will go in via ABL, but only when the qual comes through UCAS I think i.e. this year’s results. Qualifications held before ABL in August go in manually to SOATEST.
Non UCAS applications come in through OLAF (On-Line Application Form) and go into EAS, to then be entered manually by admissions into SOATEST, as stated.

8. Student OLR Qualifications checked at Stage 2 registration.
Review of qualifications held in BANNER
Update or comment
Qualifications checked at Stage 2 registration.
Original Certificates
SOATEST (BANNER) updated if required

9. ELQ
Status determined by qualifications held prior to starting the new programme
Generated within BANNER
Rules updated by PAS

10. AAB+ and Equivalents Unlimited recruitment for 2012-13
Not based on a tariff score
Determined by HEFCE
Specific list of qualifications with certain grades
Best 3 A Levels, BTECs, Best 5 Scottish Highers, 35 or more International Baccalaureate
ABB for

11. Tariff Tariffable Qualifications UCAS determine tariff scores
Generally Level 3
Not all level 3 have tariff scores
UCAS determine tariff scores
Currently based solely on GLH

12. Tariff Challenge 1
Place the qualifications in order (lowest to highest) according to their tariff score.

13. Tariff Challenge 1 Position Qualification & Grade Points 1
International Baccalaureate - 31
413 2
BTEC Diploma QCF - D*D*
280 3
CACHE Diploma in Childcare & Education - C
240 4
GCE A Level with additional AS – AB
170 5
AAT NVQ level 3 in Accounting - Pass
160 6
GCE A Level – A*
140 7
Cambridge Pre–U – D3
130 8
Irish leaving Certificate – A1 90 9
Diploma in Fashion Retail – Pass 80 10
Sports Leaders UK - Pass 30

14. HESA Tariff Most qualifications that have UCAS tariff
No qualifications that don’t have UCAS tariff
Provided to institutions
Provided to league table providers
Dependent on league tables request

15. Internal Tariff Score HESA tariff methodology In session calculation
Available for student and applicant
Applicant
Predicted
Actual
Student

16. Tariff Challenge 2
Which of the qualifications listed have tariff points?
UCAS
HESA

17. Tariff Challenge 2 Qualification UCAS HESA Access course
British Horse Society Horse knowledge & care
BTEC: Subsidiary Diploma QCF
Foundation Degree
Hong Kong Diploma of Secondary Education
iMedia Diploma
International Baccalaureate Diploma or Certificate
Certificate in Mathematics for Engineers
Music Theory Examinations
Professional Qualifications at Level 3
Vocational (VCE) A-levels
City & Guilds Land Based Services Level 3 Qualification: Extended Diploma
Key Skills level 2

18. Tariff Challenge 2 Answers
Qualification
UCAS
HESA
Access course N
British Horse Society Horse knowledge & care Y
BTEC: Subsidiary Diploma QCF
Foundation Degree
Hong Kong Diploma of Secondary Education
iMedia Diploma
International Baccalaureate Diploma or Certificate
Certificate in Mathematics for Engineers
Music Theory Examinations
Professional Qualifications at Level 3
Vocational (VCE) A-levels
City & Guilds Land Based Services Level 3 Qualification: Extended Diploma
Key Skills level 2

19. Four Main League Tables
Times
Complete Guide
Guardian
Sunday Times
There are other league tables as well:
Thomson Reuters World University Rankings published by Times Higher Educational
Shanghai Jiao Tong University Academic Rankings of World Universities

20. Measures in League Tables
Times
Complete
Guardian
Sunday Times
Student satisfaction √
Research quality
Student – staff ratio
Spend per student on academic services and/or facilities
Tariff scores on entry
Good honours (1st & 2:1)
Completion rates
Value added
Dropout compared with benchmark
Graduate prospects
Peer assessments
They all use slightly different versions of these measures and the weighting they use and how they are compiled into an overall measure varies. The newspapers vary on how much information they give on this – and this is where some scope for newspapers to add their own interpretative overlay over the HESA data
Going to unpack these in detail shortly
In brief:
All include student satisfaction – from the NSS
Research quality (Times & Complete)
Student staff ration (not Sunday Times)
Other spending measures (Complete & Guardian)
All look at tariff on entry
Good honours & completion rates (Times & Complete)
Guardian & Sunday Times have more sophisticated measures – Guardian value added & Sunday Times dropout compared with expected dropout
All look at graduate prospects – from DLHE
Sunday Times also uses peer assessment
NOTE
Times: student satisfaction & research quality 1.5, others 1
Complete Guide: Z-transformation of each measure, then 1.5 weight for Student Satisfaction & Research Assessment, 1.0 for rest
Built up from subject level tables: teaching 10%, assessment & feedback 10%, overall satisfaction 5%, value added 15%, student-staff ratio 15%, expenditure 15%, entry score 15%, career prospects 15%
Sunday Times: teaching excellence 250 points, student satisfaction +50 to -55 points, peer assessment 100 points, research quality 200 points, tariff 250 points, unemployment 200 points, good degrees 100 points, dropout rate +57 to -74 points

21. Tariff in League Tables
Average tariff
First degree, new entrants
Level 3 Tariffable qualifications
Includes partner college students
Excludes those on year 0
Tariff on entry is an important measure in all the league tables.
Average UCAS tariff (as counted by HESA)
Of first degree new entrants
With A-levels or equivalent
Includes partner colleges but excludes year 0
Thought to be seen by applicants and their parents as a measure of quality of student population
The data comes from the HESA student return – some of you will be aware that we have been making major efforts to improve the quality of the information we provide to HESA about qualifications on entry – and this is one of the reasons.

22. Tariff on entry 2012 Guides Times Complete Guardian Sunday Times
First year, first degree
All modes
Full-time & sandwich
Aged under 21
No age limit
Excluding those on foundation years
Excluding those with 0 tariff points
Based on those with A-levels or equivalent, SQA Higher or Advanced Higher,
ONC and OND including BTEC, Baccalaureate
-----
Ceiling set at 500 points & %-age of available points calculated – scores then halved to give score out of 250
Data from 2009/10 HESA student return i.e entrants
2009 entrants
Either
Full-time and sandwich only Or
Those under 21 only.
Excludes those with no tariff points at all
And
Based on A-levels and close equivalent
Sunday Times do a transformation that disregards very high scores.

23. Tariff on entry 2012 Guide (2009/10 data)
Greenwich score
Approx equivalent to
Greenwich rank
Times
215
CDD-CCD
109 of 114
Complete
111 of 116
Guardian
209
115 of 119
Sunday Times
104 (=208)
118 of 122
Based on 2009 entrants we were somewhere like CDD-CCD
Target of 50th ranked university is BBB (assuming they don’t improve as well)
Two things we can do:
Up the tariff we require before we admit students
Improve the way we record students’ qualifications on entry so that we really get every tariffable qualification on record
Matters more to us than, say, Russell group universities, because most of their entrants would have A-levels only so not so complicated
CDD=200
CCD=220
BBB=300
ABB=320

24. Times and Complete University Guide: Tariff on Entry Changes from 2013
Up to 2012
In 2013
First year, first degree, all modes
Aged under 21
Times: aged under 21
Complete: dropped age cut-off
Excluding those with 0 tariff points
Excluding those on foundation years
Based on those with A-levels or equivalent, SQA Higher or Advanced Higher, ONC and OND including BTEC, Baccalaureate
Based on wider group than previously: A-levels, Scottish & Welsh Baccalaureate, IB, various qualifications at level 3 and, crucially, ‘Level 3 qualifications of which some or all are subject to UCAS tariff’
Data from 2009/10 HESA student return i.e entrants
Data from 2010/11 HESA student return, i.e entrants
Because of problems with the coding of qualifications on entry at some universities (not here), the Times and Complete Guide are changing the way they calculate tariff on entry.
Crucially they are proposing to include a group of students who have a mixture of tariffable and non-tariffable qualifications.
This is unhelpful to us: if a student has one A-level and then other qualifications that got them the entry but are not tariffable, they’ll look as if they have a very low tariff.

25. Expected values for 2013 Guides compared with 2012
Times
Complete
2012 Greenwich score
215
A Level Grade equivalent
CDD-CCD
Expected 2013 score including mixed qualifications
254
245
CCC-BCC
CCC+
Expected 2013 score excluding mixed qualifications
257
251
Almost BCC
We have asked for these students to be excluded from our data –which makes a difference of 6 tariff points in the Complete Guide and 3 tariff points in the Times.
Either way, the 2013 Guides (based on 2010 new entrants) should show a substantial improvement. Now in the CCC to BCC range.
Remember target was BBB.
CDD=200
CCD=220
CCC=240
BCC=260