1. #NewDLHE – Design Phase
Dan Cook
2017/01/05

2. What has the review covered so far?
Outline
What has the review covered so far?
What is the mandate for change?
What will replace DLHE?

3. #NewDLHE – a brief history

4. Timeline Review Launch (summer ‘15) Strategic Group 3x Working Groups
Consultation Launch (spring ‘16)
Engagement events & blogs
2x Research Projects
Consultation summary
Strategic & Working Groups
Design Launch (early ‘17)
Engagement & Consultation
Implementation (from summer ‘17)

5. THANK
YOU!

6. Mandate for change

7. Broad agreement:
Outcomes are about more than salary – we need to expose a richer set of data on outcomes publicly and raise profile.
LEO and TEF raises the bar.
We need a universal survey of all graduates
We should make extensive use of linked data – earnings and further studies
The survey date should be pushed-out to months
We should have a suite of questions specifically for graduate entrepreneurs
We should collect data on work placements in-year (not through a survey)

8. Complexities we are resolving:
Timescales
Survey points
Quality assurance of data/methodology

9. Complexity 1: Quality assurance

10. Bitter Times Higher Education (March 2015)

11. Anecdotes that influence perception
focussing on or prioritising calls on courses which generally get positive results, and de-prioritising those that tend to get less positive results
focussing calls on students with better academic outcomes, and de- prioritising calls to those who attained worse outcomes
not reporting, or mis-reporting, low level salary data or unemployment
mis-coding SOC to achieve a ‘better’ classification
coding those working away from home as travelling
counting someone who has slammed the phone down, after hearing the call is from the university, as an explicit refusal
amending responses that are considered to ‘not make sense’ without re-contacting the respondent.

12. Centralisation

13. Centralisation (2)

14. Approach
Accept that perceptions matter – and we must address them meaningfully
Formal assessment of current and desired level of quality assurance
Appraisal of official statistics requirements for NewDLHE (and DLHE)
Utilise the UK Statistics Authority’s administrative data and assessment toolkit.
Determine what controls need to be put in place to ensure perceptions of confidence

15. Two possible approaches
Centralisation
What characteristics would a centralised approach need to promote confidence?
How would the cost model work and what opportunities are there for savings/efficiencies?
What controls will we need to ensure data is fair and consistent?
What relationships need to be in place to make this work?
What extra value is created?
Is it enough?
Audit
What characteristics would an audit regime need to promote confidence?
Resurveying in what quantity? With what cost?
What happens if an HEP fails audit? What happens if no HEP fails audit?
Is this the relationship HEPs and HESA want to have with each other?
What extra value is created?
Is it enough?

16. What could centralisation look like?
Single national contract (procured by HESA on sector’s behalf)
Governance through a HESA-managed mechanism
HEPs are committed to a reduced core of questions
There will be national optional banks of questions
There will be an option for directly-contracted services (additional questions, additional surveys, referral of students etc.) and a working relationship with the contractor, data exchange (both ways).
Supply of linked salary/study data to HEPs from HESA at agreed points

17. Complexity 2: Timescales and survey points

18. Can a single survey point work?

19. Can a single survey point work? (2)

20. Preferences for survey points

21. Timing
12 – 18 months is overall preferred timescale for most respondents
The Australian GOS has made 12 months work
LEO data won’t be available until approximately 21 months after summer graduations
Still looking in detail but 12 months or slightly longer (say, 14/15 months) looks feasible
Still need at least 2x contact periods

22. One or many surveys? Definitely one centrepiece, but…
A third of respondents (and about a quarter of HEP responses) want additional survey capability in #NewDLHE
Specific use-cases: teachers’ NQT survey – should we accommodate?
First destinations still of interest to some
Longitudinal outcomes a major interest (LEO data likely to be produced up to 10 years out from graduation)
Samples, cohorts, waves, academic research projects - additionality

23. Design for the replacement for DLHE – work-in-progress headlines

24. Timescales to implementation

25. Census survey structure
Core Questions
National agreed minimum standards
Controlled methodology to National Statistics standard
Optional Banks
Additional banks of questions available for benchmarking and special interests
Centrally-defined methodology
Bespoke Customisation
Additional services for HEPs

26. Survey approach Native to web/telephone
Good survey design - only ask one question at a time
Exploit potential of routing for web and call-centre
Learn from experience – a more natural, conversational flow
Cognitive testing
Optional banks of questions
HEPs have their own bespoke elements as now

27. What’s going? Salary question
Hours of work – simplified to part-time/full-time
Further study section

28. What’s new? New measures of outcomes (DRAFT):
How well does your current activity fit with your future plans?
To what extent do you feel that the work you are doing is meaningful and important?
To what extent are you utilising the skills you developed at university in your work?
More questions for entrepreneurs
Will be some questions for those not working paid employment.
Optional banks of questions to include:
Research students
International students
Subjective Wellbeing
Net Promoter
Newly Qualified Teachers? NHS employees?

29. Implications

30. Possible implications for HEPs
Internal coordination
Maintaining contact details for alumni for months
Placements data collected in-year as part of the student record
Legal
Data-sharing agreements
Fair processing notices
Survey coordination
Relationship management
Management information
Additional services / customisation
Finance
Survey costs incurred centrally – funding?
Savings on co-ordination in HEPs
Economies of scale

31. Possible implications for HESA
Governance
Survey Design
Operations
Finance
Publication & Use
Legal
Procurement

32. What next?

33. Design phase
Consult & engage
Finalise & Implement

34. Thank you /
2017/01/05