Just before we get started… Who am I? How questions will be handled Resources available after the webinar Key QILT Dates & the AGS Item Review 2

Key QILT dates & the AGS Item Review Quality Indicators for Learning and Teaching

About the Social Research Centre We are a private, for-profit company owned by ANU Enterprise Our services include: survey design and execution, qualitative research, survey data management, statistical consulting and analytical reporting. Key QILT Dates & the AGS Item Review 4

Overview 1.Student Experience Survey (SES) timeline 2.Graduate Outcomes Survey (GOS) timeline 3.Employers Experience Survey (ESS) timeline 4.Australian Graduate Survey (AGS) Item Review Key QILT Dates & the AGS Item Review 5

SES timeline Six main components in the SES timeline Sample draw & verification Additional items / populations specification Pre-fieldwork promotion Fieldwork / data collection Data preparation, coding & output production Dissemination & distribution The data collection period is fixed so we have very little flexibility with these dates or the ‘upstream’ dates Key QILT Dates & the AGS Item Review 6

SES: sample draw & verification The sample draw and verification procedure is identical to the procedure used for the 2014 University Experience Survey (UES). The Social Research Centre: Identifies in-scope students from HEIMS Requests institutions append students’ contact information Requests institutions flag students that have withdrawn, deferred or have passed away. Key QILT Dates & the AGS Item Review 7

SES: fixed dates 8th June: Close off for inclusion of additional populations. 22nd – 26th June: Sample files sent to institutions. 3rd July: Close off for inclusion of additional questions. 10th July: Final day to return sample files. 17th July: Final sign-off on scripted additional questions. 20th July: National social media campaign commences. 3rd August: Data collection commences. 31st August: Data collection concludes. Key QILT Dates & the AGS Item Review 8

SES: data cleaning & production Strict data input controls = very clean quantitative data Cleaning of course names is required where the student has updated the course name in the survey Please respond as quickly as possible to coding queries (this is the main delay in producing final files) Draft data files are sent FYI but you are welcome to do a ‘sense check’ particularly if this is your first SES Key QILT Dates & the AGS Item Review 9

SES: (slightly less) fixed dates 7th – 25th September: Coding queries sent to institutions. 14th – 16th October: Draft data files sent for checking. 23rd October: Final day to return data files (if necessary). Before 30th November: Final Institutional Data Files. 31st December: Final Institutional Reports. 31st December: Final National Report. 31st December: Final National Data File. Report and national data release date? Key QILT Dates & the AGS Item Review 10

SES timeline The SES data collection period is 'locked in'. Start planning now! If you have non-standard teaching periods or holidays in August, contact us ASAP If this all looks confusing / overwhelming, contact us ASAP Key QILT Dates & the AGS Item Review 11

GOS timeline The SES and the GOS follow the same process: Sample draw & verification Additional items / populations specification Pre-fieldwork promotion Fieldwork / data collection Data preparation, coding & output production Dissemination & distribution The data collection period is also fixed. Key QILT Dates & the AGS Item Review 12

GOS: sample draw & verification The sample draw and verification procedure is identical to the SES. The Social Research Centre: Identifies in-scope students from HEIMS Requests institutions append students’ contact information Requests institutions flag students that did not graduate or have passed away. (We are still working on the postgrad research issue) Key QILT Dates & the AGS Item Review 13

GOS: courses vs majors Some GOS questions ask students to think about their qualification Institutions can reference GOS responses to: Courses or programs Majors from administrative records, or Majors supplied by the student. We do not recommend the use of student supplied major (reduced data quality) Student supplied majors will attract an additional coding fee Key QILT Dates & the AGS Item Review 14

GOS: fixed dates 7th September: Close off for inclusion of additional populations. 7th September: Final day to indicate that majors will be used. 21st – 25th September: Sample files sent to institutions. 2nd October: Close off for inclusion of additional questions. 9th October: Final day to return sample files. 16th October: Final sign-off on scripted additional questions. 19th October: National social media campaign commences. 2nd November: Data collection commences. 30th November: Data collection concludes. Key QILT Dates & the AGS Item Review 15

GOS: data cleaning & production Strict data input controls = very clean quantitative data Cleaning of course names & majors is required where the student has updated or provided these in the survey Please respond as quickly as possible to coding queries Draft data files are sent FYI – we would suggest a ‘sense check’ given this is the first GOS Key QILT Dates & the AGS Item Review 16

GOS: (slightly less) fixed dates 1st – 11th December: Coding queries sent to institutions. December: Draft data files sent for checking. December: Final day to return data files (if necessary). January: Final mid-round Institutional Data Files. Institutional Reports and the National Report are not produced until the April Round has been finalised (tentatively, August 2016). Key QILT Dates & the AGS Item Review 17

GOS timeline How will you communicate the change from the AGS to the GOS within your institution and to your potential graduates? Do you need to supplement the GOS with questions or graduate populations? Key QILT Dates & the AGS Item Review 18

ESS timeline The ESS is ‘live’ at the same time as the GOS. All employed graduates are asked to provide contact information for their supervisors Supervisors can receive an about the survey as soon as the graduate has provided contact information. There is nothing for you to do with the ESS – just be informed about the survey so you can answer general questions if they arise. Key QILT Dates & the AGS Item Review 19

ESS: dates 19th October: National communications campaign commences. 2nd November: Data collection commences. 22nd December: Data collection concludes. ESS has a CATI non-response follow-up phase February: Mid-round data files sent to institutions. Key QILT Dates & the AGS Item Review 20

ESS timeline The ESS is ‘set-and-forget’ – you don’t need to do anything! Fact sheets and FAQs will be provided to institutions to help with briefings and any questions Key QILT Dates & the AGS Item Review 21

AGS Item Review In December 2014, 35 universities provided feedback about the GDS and the CEQ. Each AGS item was categorised as essential, important, interesting and irrelevant to institutional KPIs and performance management frameworks Many institutions provided useful commentary on their ‘ratings’ and additional feedback about the AGS Key QILT Dates & the AGS Item Review 22

GDS feedback Of the 70 GDS items and 38 were identified as essential or important. Of these 38: 8 demographic items are available from HEIMS 23 are labour force items 5 relate to further study, and 2 are demographic items that may not be available from HEIMS. Key QILT Dates & the AGS Item Review 23

CEQ feedback The Overall Satisfaction Item, the Generic Skills Scale and the Good Teaching Scale were regarded as essential. The Graduate Qualities Scale and the open-ended items (Best Aspects & Needs Improvement) were identified as important. Additional comments suggested that these CEQ concepts are still very relevant to institutions but that the implementation is no longer contemporary. Key QILT Dates & the AGS Item Review 24

Next steps for the GOS GDS items have been aligned with Australian Bureau of Statistics standards (mostly from the Labour Force Survey). The CEQ scales are still under consideration. An initial version of the Graduate Outcomes Questionnaire (GOQ) has been drafted. We will require feedback regarding the ‘non-ABS’ items to ensure that they are still fit for purpose. Feedback guidelines for the draft GOQ are being prepared. Key QILT Dates & the AGS Item Review 25

GOS trial For almost all institutions, the next round of the AGS will be ‘business as usual’ – no institution will be required to participate As soon as a draft questionnaire has been approved by the department we will approach potential trial institutions If you would like to be a trial institution, you are welcome to contact us to discuss options Key QILT Dates & the AGS Item Review 26

ESS trial The GOS and the ESS are linked so institutions participating in the GOS trial will also be part of the second ESS trial. The next ESS trial will: Refine the supervisor recruitment methodology Test a modified and improved questionnaire Key QILT Dates & the AGS Item Review 27

AGS Item Review Essential and important GDS items are being redeveloped. Key CEQ scales are 'moving to the next round'. A draft GOQ will be circulated as soon as possible. Key QILT Dates & the AGS Item Review 28

Working with the QILT team We are always available to  answer questions (no matter how vague)  provide advice about research methodology or data collection  explain technical aspects of the QILT surveys or the data files  assist with the inclusion of additional questions or populations If you have been liaising with a specific person, please contact them directly, otherwise, will always work. Key QILT Dates & the AGS Item Review 29

To summarise The SES data collection period is 'locked in'. Start planning now! Do you need to communicate internally about the GOS? Do you need to supplement the GOS? Be aware of the ESS so you can answer questions. Essential and important GDS items are being redeveloped. Key CEQ scales are 'moving to the next round'. Key QILT Dates & the AGS Item Review 30

And finally… Block out the SES survey period so participation for your students can be maximised. Please start discussions now about any additional populations or questions for the QILT surveys (noting the close-off dates). If you have any concerns or questions, please contact the QILT Team. Key QILT Dates & the AGS Item Review 31