Presentation is loading. Please wait.

Presentation is loading. Please wait.

Professor Diana Kornbrot University of Hertfordshire

Published byAnderson Salvador Neiva Chaplin Modified over 6 years ago

Similar presentations

Presentation on theme: "Professor Diana Kornbrot University of Hertfordshire"— Presentation transcript:

1 Professor Diana Kornbrot University of Hertfordshire
'Quality'? Indicators from combining Likert items and other measures from surveys Professor Diana Kornbrot University of Hertfordshire

2 Scene Setting Background Experience
Evaluating Higher Education Benchmarking for HEA, Student Experience Psychological Measurement and Mathematical Modelling Public Indicator Statistics are Composite Components all same type 1-5 Ordinal: agreement or quality(Likert items); Money Components mixed Economic Development: 1-5 Ordinal + Money + Mortality + Literacy Higher Education: 1-5 Ordinal + Money + Failure + Employment Agregation Method from Components Changes Results Important Decisions made on Indicator Ranks 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

3 Getting the Data Higher Education Economic Development
HESA By institution By subject Pay to get by insitution by subject HOLIS Search by Subject Area, University, Gender, Degree Class THES PUSH £14.95 UCAS Economic Development 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

4 Times Higher Education Supplement
League Tables 2006 Entry standards A level points ST 100 Student-to-staff ratio ST 100 Student satisfaction (4min-20max) Likert 13.7 – 16.1 ST 150 Research assessment exercise Pubs 0 -7, bizarre ST 200 Library and computing spending £s 3 year average Facilities spending £s 3 year average Good honours % of graduates ST 100 Completion % entrants Graduate destinations % known ST 100 Salaries 04-05 VC Pay 2004/2005 £104k-£305k Average FT Salary 2004/2005 £21.7k-£43.2k 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

5 National Student Satisfaction
Form For each statement, show extent of agreement or disagreement 5 Definitely agree, 4 Mostly agree, 3 Neither agree nor disagree, 2 Mostly disagree, 1 Definitely disagree, N/A Not applicable The quality from my University for eatch feature was: 5 Excellent, 4 Good, 3 Satisfactory, 2 Weak, 1 Poor, N/A Not applicable Examples 8. I have received detailed comments on my work 19. The course has helped me to present myself with confidence Topics The teaching on my course Q1-4 Assessment and feedback Q5-9 Academic support Q10-12 Organisation and management Q12-15 Learning resources Q16-18 Personal development Q19-22 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

6 Making the League Table
Actual Agregation Procedure Normalize so maximum is 100 Decide on weighting Aggregate Do weightings matter? Desirable? Normalize to z-scores Distinguish input and output measure Exploratory principal components? Univariate multiple regrssion > canoncial input measure? MANOVA? > canoncial input, output measure? Rank Categorize League? Where are big differences 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

7 Principal Components 2 Components 29-Nov-18
Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

8 Principal Components Oblimin Varimax Component correlation .199
Component 1: 41.0 %variance Component 2: 41.0 %variance 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

9 Input and Output 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

10 League Measures Raw ZGood Hons ZNSS Oxford Thames Valley Cambridge
Loughborough Bristol Lampeter Warwick Leicester Nottingham East Anglia Bath Chichester Imperial College Chester Durham York LSE Royal Holloway University CL Bath Spa 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

11 League Measures Agregate
29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

12 On Evaluating HE Misleading if not by discipline
Shows ideas League Tables Uninformative Need predictors Weighting Matters? Benchmarking E-Learning Untested target: Embed E-Learning Implications Better tools from public statistics 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

13 On Indicators Generally
Economics Very Similar Can show almost anything Needs sound basis Predictors Weighting? 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

14 Thank You 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

15 Original Abstract Single quality indicators are often derived from weighted sums of survey items, including ordinal Likert items. It is widely believed that the weights and combining procedures don't make much difference to the final rankings or scores. This presentation challenges this assumption using examples from the National Student Survey of UK higher education and country development indices. 29-Nov-18 Kornbrot, Radical Statistics Meet 24 Feb 2007: Quality of Publice Survey Statistics

Download ppt "Professor Diana Kornbrot University of Hertfordshire"

Similar presentations

THROUGH FAITH AND LEARNING BISHOP JUSTUS 6TH FORM Applying to AB Universities KS5 Conference 2013-14 Ms Linton Director of 6 th Form.

THROUGH FAITH AND LEARNING BISHOP JUSTUS 6TH FORM Applying to AB Universities KS5 Conference Ms Linton Director of 6 th Form.
GLOBAL RANKINGS OF UNIVERSITIES John O’Leary I Editor I Times Higher Education Supplement.

GLOBAL RANKINGS OF UNIVERSITIES John O’Leary I Editor I Times Higher Education Supplement.
Hubble Bubble Some Qualifications Bring Trouble

Hubble Bubble Some Qualifications Bring Trouble
Gifted, talented, ambitious and able. How to make the most of your potential Richard Kennett Director of UK Student Recruitment, Outreach and Admissions.

Gifted, talented, ambitious and able. How to make the most of your potential Richard Kennett Director of UK Student Recruitment, Outreach and Admissions.
Year 11 IAG session. Aims: To understand what different qualifications mean To understand what you need for different courses/an intro to what Uni’s look.

Year 11 IAG session. Aims: To understand what different qualifications mean To understand what you need for different courses/an intro to what Uni’s look.
Reflecting on the results of the 2013 National Student Survey Professor Kelvin Everest PVC for Student Experience November 2013.

Reflecting on the results of the 2013 National Student Survey Professor Kelvin Everest PVC for Student Experience November 2013.
See (http://www.oecd.org/publications/). OECD-JRC handbook on CI The ‘pros’: Can summarise complex or multi-dimensional issues in view of supporting decision-makers.

See ( OECD-JRC handbook on CI The ‘pros’: Can summarise complex or multi-dimensional issues in view of supporting decision-makers.
1 Single Indicator & Composite Measures UAPP 702: Research Design for Urban & Public Policy Based on notes by Steven W. Peuquet. Ph.D.

1 Single Indicator & Composite Measures UAPP 702: Research Design for Urban & Public Policy Based on notes by Steven W. Peuquet. Ph.D.
Leeds University Business School Management Alistair Norman Management Division.

Leeds University Business School Management Alistair Norman Management Division.
Regulating the engineering profession 1 EC UK Experience in Accreditation of Engineering Programmes Professor Ian Freeston University of Sheffield, UK.

Regulating the engineering profession 1 EC UK Experience in Accreditation of Engineering Programmes Professor Ian Freeston University of Sheffield, UK.
1 Herschel Grammar School Academic Success and Personal Responsibility AS YOU ARRIVE … PLEASE COMPLETE THE PSHCE SURVEY Place a tick in one of the BEFORE.

1 Herschel Grammar School Academic Success and Personal Responsibility AS YOU ARRIVE … PLEASE COMPLETE THE PSHCE SURVEY Place a tick in one of the BEFORE.
1 Herschel Grammar School Academic Success and Personal Responsibility Is there life after Herschel?

1 Herschel Grammar School Academic Success and Personal Responsibility Is there life after Herschel?
1 Herschel Grammar School Academic Success and Personal Responsibility Is there life after Herschel?

1 Herschel Grammar School Academic Success and Personal Responsibility Is there life after Herschel?
LEAGUE TABLES AND NEW KPIs VC’s OPEN MEETING Anita Wright Head of Planning November 2013.

LEAGUE TABLES AND NEW KPIs VC’s OPEN MEETING Anita Wright Head of Planning November 2013.
Benchmarks and Benchmarking in the UK - Lessons Learned Catherine Connor Quality Enhancement Unit London Metropolitan University.

Benchmarks and Benchmarking in the UK - Lessons Learned Catherine Connor Quality Enhancement Unit London Metropolitan University.
Searching for Globally Feasible Indicators from Domestic Rankings Ya Lan Xie and Ying Cheng Graduate School of Education, Shanghai Jiao Tong University,

Searching for Globally Feasible Indicators from Domestic Rankings Ya Lan Xie and Ying Cheng Graduate School of Education, Shanghai Jiao Tong University,
Learning and Teaching Conference 2014 Remembering the Student in Student Satisfaction Dr David Bowles.

Learning and Teaching Conference 2014 Remembering the Student in Student Satisfaction Dr David Bowles.
UCAS 2013. Guest Speakers Camille Evans: Bullers Wood School Careers and alternative routes to University Amy Staniforth: University of East Anglia Student.

UCAS Guest Speakers Camille Evans: Bullers Wood School Careers and alternative routes to University Amy Staniforth: University of East Anglia Student.
St. Joseph’s Grammar School “providing a line of sight to employment”

St. Joseph’s Grammar School “providing a line of sight to employment”
University Strategy Marcus Williams. Contents 0.1 Strategic goals 0.2 Re-positioning 0.3 Campus infrastructure 0.4 Sussex off campus 0.5 Malaysia Office.

University Strategy Marcus Williams. Contents 0.1 Strategic goals 0.2 Re-positioning 0.3 Campus infrastructure 0.4 Sussex off campus 0.5 Malaysia Office.

Similar presentations

Ads by Google