1. How do you solve a problem like Impact? Summary of survey findings Paul Redmond Head of the Careers & Employability Service, University of Liverpool

2. Overview 46 completers (82% heads of service with between 4 – 7 years service). Majority (85%) located in student service-related department. 11% in a teaching-related department 3.8% in an external-relations department Spread of universities (post-1992, RG, etc.)

3. Reporting structure 70% of heads report to Head of Student Services 20% report to PVC Academic deans / heads of T&L = 6.8% respectively

4. Defining your Services purpose Providing services to students – nearly all cite employability services to students as the primary goal, e.g. Improving the employability of students and graduates and providing a central point of contact to support CEIAG across the campus …

5. Defining your Services purpose Providing services to students – nearly all cite employability services to students as the primary goal, e.g. Improving the employability of students and graduates and providing a central point of contact to support CEIAG across the campus as well as delivering centrally …

6. Rank your main stakeholders 1 st 2 nd 3 rd 4 th 5 th Students93% (53)7% (4)000 Graduates057.4% (31)9.3% (5)14.8% (8)18.5% (10) Employers019.6% (11)44.6% (25)28.6% (16)7.1% (4) Academics09.1% (5)36.4% (20)29.1% (16)25.5% (14) University managers 7.1%8.9% (5)10.7% (6)28.6% (16)44.7% (25)

7. How do you or your HEI measure the responsiveness, profile and visibility of your service with the above groups? Mixture of internal and external feedback measures Student numbers / record of users Referrals from academics Profile with senior managers DLHE – institutional comparators (ePI) Student satisfaction / user surveys (various) League tables Involvement of CES in key projects /initiatives Access to prized networks Employer evaluations Hits on website Profile and visibility … Matrix

8. How do you record and track usage of your Service? Central recording systems On-line systems (e.g. Interfase) Numbers attending events records Mixture of qualitative and quantitative research – headcounts to mystery shoppers We dont! Basic headcounts Monthly MIS surveys / reports Interview stats Hits on website Twitter traffic …

9. How do you measure your Services impact on students? NumbersSatisfactionImpact Dont measure Curriculum based group work 82.1% (46)85.7% (48)28.6% (16)1.8% (1) Centrally based group work 81.8% (45)90.9% (50)20% (11)1.8% (1) One to one discussions 84.2% (48)89.5% (51)36.8%(21)0 Online / self help72.2% (39)63% (34)18.5% (10)13% (7)

10. How do you measure your impact with employers? NumbersSatisfactionImpactDont measure Employer vacancy handling 86.8% (46)62.3% (33)30.2% (16)7.5% (4) Employer fairs / presentations / events on and off campus 86.8% (46)94.3% (50)18.9% (10)1.9% (1) Employer input into group work with students 66.7% (34)64.7% (33)13.7% (7)19.6% (10) Facilitating employer links with academic departments 41.2% (21)39.2% (2)23.5% (12)43.1% (22) Consultancy and other tailored services for employers 34% (16)36.2% (17)21.3% (10)55.3% (26)

11. Other ways in which you measure your impact on employers None (several) Numbers of vacancies filled / quantity and quality of applications Employer advisory board Repeat business Direct feedback from employers in relation to vacancies and internships

12. Do you or your institution measure the impact of your service or your INSTITUTIONS REPUTATION AND SUCCESS on:

13. How your careers service impacts on your institutions reputation and success Provides comparative data on graduate employment (shows where HEI stands in league tables) Success in generating funded projects Income generation League Tables / DLHE – Seen as our most important function by senior management, sadly.) KPIs in reports to management Head produces full-cost recovery study to gauge VfM Contribution to student retention Return on Investment … not sure about how this might be specifically measured … Nothing too explicit.. … None.

14. How highly are you perceived in your HEI?

15. What KPIs and / or targets does your Service have? DLHE – top listed KPI Employability P.I. Nos of finalists / graduates seen Increased numbers using the Service Student satisfaction surveys Listed in Top 10 surveys Personal recommendation No. of placements achieved Matrix Mixture of externally imposed and internally verified

16. Final comments … I am still at a loss as to how to establish meaningful and real impact measures for most of the work we are engaged in, despite working at the problem for many years! Fortunately, I am rarely required to justify our impact. Be careful of using DLHE as a KPI or measure of impact. Our work does not necessarily have immediate effect – it may take many weeks, months or years for a client to really take on advice. How do we measure the impact of guidance? Collective approach from AGCAS in addressing this is very much welcomed.

17. Reflection

18. The Impact Paradox: By prioritising overwhelmingly services to students, which produce intangible outcomes, careers services find it difficult to measure impact. Those services which can be measured are often not those viewed as key priorities by services. Location, Location, Location: How crucial is institutional positioning? Across the sector, wide variations exist (and varying levels of measurement sophistication). But impact measurement is becoming a widely accepted issue of importance for most careers services. Nevertheless, methodologies remain limited.