Review of Destinations of Leavers from Higher Education (EDLHE) Survey Review outcomes Sarah Grimes Senior Business Development Analyst

The review… Fundamental review of the survey and supporting systems in advance of 2011/12 implementation… …with particular reference to: coverage the content of the questionnaire methodology functionality of the DLHE Online Survey System possible technical update interaction between the EDLHE and Longitudinal (LDLHE) surveys

The review… Commenced March April 2011 – published coding manual and questionnaire Review Group of HESA Statutory Customers, institutional representatives and technical experts have been working on the development of proposals for change Sector consultation September/October responses from a range of institution types A small number of other organisations and groups including software suppliers, Vitae, AGCAS, GLAM and BIS also submitted responses

The review… Significant interest in destinations data at a political level, in particular the provision of careers information to prospective students, and so the review also had to take into account: Outcomes from external research funded by BIS (International Student Tracking Study) Outcomes from external research funded by HEFCE (Evaluation of Longitudinal Destinations of Leavers from Higher Education) Outcomes from the Browne Report (Securing a Sustainable Future for Higher Education in England) Anticipated outcomes from 2011 Higher Education White Paper Development of the Key Information Set (KIS)

What has been reviewed? The review areas can be categorised as follows: Timing of the survey Changes to the survey coverage Questionnaire content and format Methodology How SIC coding is undertaken Working with SOC 2010 and how coding is undertaken DLHE Online System – how it can work better for institutions Plus the record is being updated to XML format

Survey timing Explored alternative timings – 9, 12, 18 and 24 months Analysis undertaken using EDLHE and LDLHE data No evidence that six months is sufficiently poor at indicating leavers’ outcomes that it must be changed EDLHE must be complemented by the later LDLHE Survey as EDLHE is not an indicator of later outcomes: it is an indicator of what leavers can do six months after leaving If the LDLHE Survey does not continue then the timing of the EDLHE Survey will need to be reconsidered

Coverage Four changes to coverage which means that overall more leavers will be surveyed One change to qualification awarded (QUAL) criteria One change to mode of study (MODE) criteria One change to exchange (EXCHANGE) criteria Extension to all overseas students There will be a wider range of students for whom good contact details will be required (the changes creates approximately 30,000 new instances) Student Record contacts need to work closely with DLHE contacts in order to ensure that the right students will be surveyed

Change to QUAL From 2011/12 the EDLHE Survey population will be extended to include all HE qualification types apart from credits This change is to reflect the increasing numbers of students studying professional qualifications Will still exclude intercalated degrees, and also awards for visiting students Also there is still no requirement for the inclusion of qualifications for serving school teachers and post-registration health and social care students

Change to MODE From 2011/12 the EDLHE Survey population will include all qualifying postgraduate research students (including those who qualify from dormant status) Large numbers of leavers have previously been excluded from the survey population because of their dormant status

Change to EXCHANGE From 2011/12 the EDLHE Survey population will no longer include incoming exchange and visiting students These leavers (approximately 1000) are excluded from other populations and from all analysis undertaken at HESA

Extension to all overseas students From 2011/12 the EDLHE Survey population will include all non-EU international leavers This decision was made having considered the recommendations from the BIS-commissioned International Student Tracking Study and sector feedback on practicalities Feedback showed that there is varying success across institutions who already survey these leavers These leavers must be contacted electronically (use of the paper questionnaire and telephone methods is optional)

Extension to all overseas students (continued) For 2011/12 and 2012/13 surveying these leavers will be seen as a pilot No publication of this data until it has been reviewed If published it will be summary level data No target response to be set Assessment of the two years of surveying these leavers will inform the decision about the continued inclusion of them in the survey population

Questionnaire content A number of changes to the questionnaire content, the most significant changes are: Section A Collection of salary data Use of JACS for coding subject of study, training or research Removal of part-time motivation questions Introduction of employability and entrepreneurship questions The reviewed content will allow for the capture of more complete information about graduate activity – we know that the current questionnaire no longer reflects reality Internships and sandwich placements can now be recorded

Section A Similar approach to LDLHE questionnaire A single list of options including work and study Asks for all activities and then which of them is the most important Will capture more information about the range of activities a leaver may be involved in New set of publication categories for activity

Changes to Section B (employment) Leaver is asked to indicate how many jobs in total they are doing… …and then describe their MAIN employment Additional options for basis of employment Country added to place of work question Reason for taking the job question – same approach as Section A (the leaver can indicate all reasons and then just the main reason) Changes to the list of options for how the leaver found the job Changes to the list of options for how the leaver worked for the employer Better instructions e.g. for company/organisation it is the place of work and not the agency name that is required

Changes to Section B (continued) Salary Three-part salary question: Annual pay for ‘main’ job Part-time hours Estimated total earnings Will allow the capture of total earnings Indicating how many other jobs the leaver is doing on the census date will contextualise the salary data

Salary (continued) Salary information will be used to form part of the KIS – average salary six months after completing course HEPISG has stipulated that at least 50% of students employed full-time must have provided a salary for the course KIS to include average salary HEFCE is planning half a dozen pilot audits to see how audits of DLHE might work

Section C (now Newly qualified teacher status) Moved to follow the employment questions Addition of a question to capture leavers who gained employment through the GTC Scotland teacher Induction Scheme Changes to types of school (state-funded and non-state-funded) to reflect the current funding arrangements Seeking a teaching post question now for those on temporary teaching contracts on the census date as well as for those not employed as a teacher

Section D (now further study, training or research) Moved to allow teaching questions to follow employment Leaver is asked to indicate how many courses in total they are registered on… …and then describe their MAIN study, training or research Reason for undertaking further study, training or research question removed as a data not used Slight changes to funding types

Section D (continued) JACS JACS replaces SIC for coding subject of study JACS 3.0 has recently been developed by HESA and UCAS Will be used to capture subject of further study, training or research for EDLHE from 2011/12 Will be used ahead of implementation for the Student Record and Staff Record (2012/13) to save making changes in two consecutive years

Section E Part-time motivation The three part-time motivation questions have been removed Have long been considered out of place on the EDLHE questionnaire Motivation for undertaking the original course part-time will be considered for collection as part of the Student Record during the next review

Section E (continued) Employability and Entrepreneurship Strong political interest in collecting information about how HE prepares students for work and setting up and running their own business A question each to capture this information, plus a question about preparation for further study will be included as the new Section E

Employability and Entrepreneurship (continued) Your overall higher education experience Finally, please could you reflect on your overall experience in higher education using the following scale: 1 = Very well 2 = Well 3 = Not very well 4 = Not at all Q29 How well did your overall experience in higher education prepare you for employment? Can't tell Q30 How well did your overall experience in higher education prepare you for further study? Can't tell Q31 How well did your overall experience in higher education prepare you for being self-employed/freelance or for starting up your own business? Can't tell

Questionnaire design Improvements to ‘look and feel’ Advice sought from accessibility bodies Coding boxes removed to create more space and so improve the layout Questions and design underwent cognitive testing

Methodology Relaxed methodology allowing each institution to choose the data capture methodology which enables them to achieve the best level of response (depending on quality of types of contact details) Single version of the questionnaire for use for all methods of contact Minimum set of questions which should be answered in order for the response to count No reduced set of questions for telephoning Minimum set for use when speaking with a third party

Methodology (continued) Approach taken for the LDLHE Survey of 2006/07 leavers was to contact as many leavers by first, then to use the telephone and then postal contact The sequence was swapped because of the declining postal response and increasing expense associated with multiple mailings Better quality of response online and by telephone (plus environmentally better) This resulted in nearly 50,000 responses – up approximately 8,500 from the previous (2004/05 leavers) survey

Methodology (continued) timing No longer concept of initial and follow-up periods Attempt to contact all leavers at least once and optionally more than once Remains ideal to contact the leaver directly but a third party or institution records can be used to elicit the information Attempts to contact all non-EU leavers in the DLHE population should be made at least once electronically April survey field work - census date in April to end June January survey field work - census date in November to end March

SIC coding Centralised coding exercise Size of organisation (in terms of employees and finance) will be additional data available Therefore number of employees question removed from questionnaire Institutions will be able to see the coding prior to signing off their data HESA is currently exploring options for how this will work and will tender for this service Needs to be in place by March 2013

SOC SOC coding will not be centralised SOC 2010 will be used from 2011/12 5 th digit required in places –work was undertaken by Charlie Ball at HECSU Will be obtaining an upgrade of CASCOT to include this fifth digit IER will finalise the 5 th digit Offer training and good practice guidance on SOC coding Needs to be in place by April 2012

DLHE Online System No development of a separate input-only version Some enhancements can be made to the online version to facilitate institutions using it as an input tool Storage of the data Accessibility to the system – surveying non-EU international leavers means it will need to be available 24/7 Needs to be ready for institutional use by March 2012

XML EDLHE 2011/12 will be returned in XML Will be the second of the record-based collections to be moved to XML following the Student Record for 2007/08 Some training will be provided as part of wider EDLHE 2011/12 implementation seminars in September/October

Roll out of the revised DLHE Record and related processes April 2011: Publication of manual Final questionnaire available May – December 2011: Incorporating new SIC processes SOC and related coding tool (plus good practice) Any changes to DLHE Online Survey and storage of EDLHE online data Training seminars –28 September Napier University –29 September Edinburgh University –4 October Woburn House, Tavistock Square London

Changes for 2010/11 National Statistics Country Classifications 2006 (NSCC) adopted for the EDLHE Survey Compulsory to ask for salary data when speaking with a leaver on the telephone LDLHE opt-out question should not be asked to third parties

DLHE Good practice This will be updated following the completion of the 2011/12 survey Institutions should let us know how they have undertaken the survey and any tips that they can pass on The updated version will be electronic

Any questions?