Briefing session on Online Student Evaluation of Teaching and Learning Prepared by Social Sciences Research Centre of The University of Hong Kong 1

SETL process Online process will cover all courses with undergraduate and TPG students (including Common Core courses) on 2 semesters system with teacher data in SIS, unless otherwise agreed. Non-semester courses can be evaluated on-line if the course information with an end date later than the end of first semester and the end date have been input into SIS, otherwise use paper. For RPG only courses, the departments/centres/schools /institutions can choose to use paper or online The online SETL evaluation will normally be conducted for 4 weeks until the start of the assessment period (i.e. for first semester courses, from Nov 9 to Dec 7, 2016) . 2

SETL process Separate SETL forms for courses, teachers, tutors and demonstrators Uses SIS database record of course/teacher/tutor/demonstrator (so these must be input accurately into the SIS) SIS allows joining different courses of the same level in the evaluation, only one overall report will be generated for the parent course to the core department Courses of different level will always be assessed separately, even if the same course code MBBS and Dentistry online SETL evaluation are handled separately 3

SETL process Must use standard forms unless Chairman of TLQC approves Online yields lower response rates, but better open- ended responses, which is good for improvement To ensure good response rates, need to ensure that students are given time at the end of a class to complete the form (although they can complete any time before the deadline) 4

SETL process Cannot currently change teacher names in system, which come from Human Resources All teachers/tutors/demonstrators must have staff ID/University No. (HKU requirement and needed for staff review) For the Department Items, a maximum 5 extra questions for both on-line and paper evaluation, please contact and confirm with SSRC staff, Ms. C.C. Ho at 3917-1623 or setl@ssrc.hku.hk (on or before mid October, 2016 for this semester) 5

SETL process For the online SETL, all courses with enrollment of at least 2 students of the same level must undertake SETL, although the teacher/tutor/demonstrator parts are only done if there are at least 2 students per teacher/tutor/demonstrator (unless VP agrees) For the paper SETL, all courses with enrollment of at least 5 students must undertake the evaluation (unless VP agrees), although the teacher/tutor/demonstrator parts are only done if there are such persons for the class 6

SETL process Up to 25 teachers can be evaluated in each course by a student, but only 1 tutor/demonstrator selected by a student from a list of up to 25 names (or evaluated as a group) of each course Tutor means person running tutorials, demonstrator means person responsible for lab or field sessions (role descriptions, not job titles) 7

SETL Workflow 8 Prepared by Social Sciences Research Centre of The University of Hong Kong 8

SETL Workflow of the Online SETL Departments/Centres/Schools/Institutions must enter/update the course enrollment information in the Student Information System (SIS) especially the staff number linked to the name of teacher/tutor/demonstrator in each course and sub-class information 10 weeks before the assessment date (i.e. before end of September for this semester and finish date for non- semester courses) All information must be updated for the appropriate semester Student enrollment should be updated automatically for courses using SIS for enrollment, but the rest of the information relies on the departments/centres/schools/institutions to update 9

SETL Workflow of the Online SETL SIS system exports the course enrollment information of all undergraduate, TPG and RPG courses of which complete SIS data is available including: Course code; Course title; Term (Semester 1/2/full Year) End dates if not semester based (starting next semester) Student list in each course (student name and university no.) by course level; Teacher name list in each course (teacher name and staff number); Tutor and Demonstrator name list in each course (teacher name and staff number); and Jointly evaluated courses (course list) 10

SETL Workflow of the Online SETL In this semester The course/teacher/tutor/demonstrator lists and indication of whether online or paper process will be sent out to departments/centres/schools/institutions on or before 9 October 2016, so departments/centres/schools/institutions can check for errors and missing information. Departments/Centres/Schools/Institutions should update any data amendments in the SIS system between 9 and 23 October 2016 for this semester. 11

SETL Workflow of the Online SETL In this semester All courses with any undergraduate or TPG students with teacher data in SIS will be evaluated on-line, unless non- semester courses or VP agrees. Please let SSRC know of any non-semester undergraduate courses with the end date and any undergraduate courses which have obtained approval from VP for paper. All RPG courses with teacher names will be assumed to be evaluated on-line, but please let SSRC know if any of them will choose to use paper. 12

SETL Workflow of the Online SETL The final course/teacher/tutor/demonstrator lists and indication of whether online or paper process will be sent out to departments/centres/schools/institutions (i.e. on or before 2 November 2016 for this semester). If any errors occurs, please contact SSRC through email setl@ssrc.hku.hk immediately (i.e. on or before 5 November 2016 for this semester). For information, SSRC will send out the invitation emails to all HKU students on 9 November and 2 reminder emails to all HKU students on 30 November and 4 December, 2016 for this semester 13

SETL Workflow of the Online SETL Next semester Non-semester courses must input dates through SIS system so they can be evaluated on-line and then the course/teacher/tutor/demonstrator lists will be sent out to departments/centres/schools/institutions 2 weeks before the evaluation date (ONLY ONE circulation will be sent). If any errors are found, departments/centres/schools/institutions must respond within one week after receiving the lists from SSRC and please contact SSRC through email setl@ssrc.hku.hk immediately. 14

Instructions for Online SETL Before administration The teacher concerned should inform the students in advance the scheduled time set aside for the online evaluation (i.e. soon after 9 November 2016 for this semester), and that the students are encouraged to bring a mobile internet device (such as laptop, tablet or smartphone) to class at the scheduled time. 15

Instructions for Online SETL On the day of administration Similar to the paper exercise, the SETL questionnaire should NOT be administered by the course teacher, but faculty/department staff. The teacher should briefly explain the objectives of the student evaluation and then leave the room leaving it to faculty/department staff to oversee the process. 16

Instructions for Online SETL On the day of administration Faculty/Centre/Institution/Department staff should read out: Please go to the weblink http://setl.hku.hk where you should see a list of all forms that you need to complete. Your evaluation will be saved anonymously, without any identification. There are separate forms for the course, teacher (and if appropriate) tutor and demonstrator. There are instructions, an FAQ link and a link for you to report any missing courses at the weblink. 17

Instructions for Online SETL During the evaluation period Students should Access the SETL system via the weblink http://setl.hku.hk/ where there are instructions, FAQ and a list of forms for completion (separate links for the course, teacher, tutor, demonstrator forms) Complete the forms for the class during four weeks until the start of the assessment period (e.g. this semester from Nov 9 to Dec 7, 2016) If they cannot find the course that they have enrolled in, send an email to SSRC using the link on the page so SSRC staff can rectify this promptly 18

Workflow for Paper SETL Departments/Centres/Schools/Institutions should be responsible for the accuracy of SIS data before SSRC generates the SETL reports and inform SSRC about the course enrollment information for printing purpose. Departments/Centres/Schools/Institutions should inform SSRC for printing at least 2 weeks before the evaluation date. Departments collects the blank forms one week before the evaluation date and carries out the evaluation. For the paper evaluation forms, departments should deliver to SSRC all completed forms, completed cover sheet on the top of each envelope. 19

SETL Workflow for Reporting The SETL results will be generated upon notice from SSRC after the examination result release date (e.g. 20 January 2016 for this semester) and the following report will be provided to each department/centre/school/faculty/institution and the reports will be saved in MS Excel format (worksheets) : an individual report of every teacher for each course; an overall teacher report for every course; a report for the tutor/demonstrator for every course (if applicable); a course evaluation report for every course; a soft copy of all open-ended questions’ answer for each course using online SETL (in MS excel format by question number); 20

SETL Workflow for Reporting The overall SETL report will be provided to each department/centre/school/faculty/institution and the reports will be saved in MS Excel format (worksheets) : an overall report on the undergraduate courses; an overall report on the TPG courses; and an overall report on the RPG courses 21

SETL Workflow for Reporting Additional requests may be entertained at cost SSRC will upload the SETL results to the APA on or after 21 January 2017 for this semester (after the examination result release date 20 January 2017) SSRC will upload the SETL results to the SSRC online SETL report system on or after 21 January 2017 for this semester (after the examination result release date 20 January 2017) 22

Instructions for Online SETL My colleague, Mr. Samson Lee, will now demonstrate Online SETL, so you can see how the process works in practice: http://setl.hku.hk My colleague, Mr. Samson Lee, will now demonstrate Online SETL report system, so you can see how the process works in practice: http://setl.hku.hk/report 23

Instructions for Online SETL The SSRC Guide (Third Edition, April 2015), Briefing session on Online Student Evaluation of Teaching and Learning (7 October 2016)and SETL Online DEMO are available: http://www.ssrc.hku.hk/setl.html 24

Thank you! I am happy to answer questions. Prepared by Social Sciences Research Centre of The University of Hong Kong 25