Enhancing the assessment experience through closer integration between the SRS and VLE University of York, UK Dr Richard Walker bett 2015 January

York’s journey

York’s service portfolio

York’s technology adoption cycle Adapted from the Gartner Hype Cycle

York’s technology adoption cycle Would you like to see an online submission option made available for all assessed work that you need to turn in to your department for marking? Response PercentResponse Count Yes 76%354 No 11%54 Not applicable 13%59 AgreeNeutralDisagree Not Appropriate Submission of student work 83%10%5%2% Management & allocation of work for markers 51%34%10%6% Marking 66%17%12%5% Creation of feedback 66%22%10%2% Return of work/marks/feedback to students 68%25%5%2% Source: University of York Technology Enhanced Learning Student Survey 2014 Source: University of York Technology Enhanced Learning Staff Survey 2014 It is important for the University to provide online support for all assessed work turned in by your students for marking “It is important for the University to provide *GOOD* online support for the listed activities. I don't believe this is sufficiently the case.”

Joining up SRS and assessment systems  A challenging problem for the HE sector: ‘Despite the relatively limited nature of the core product set, the key integration points between these technologies remain problematic and a source of considerable manual intervention.’ (Jisc, 2014: p12) Jisc (2014). Electronic Management of Assessment (EMA): a landscape review. (pdf)

Original anonymous submission and feedback workflow (summative assessment)  Replacing file name with student exam number from SITS eVision: WhoOriginal Process (2007 – 2013) Students Submit work online through VLE, automated ed receipt VLE instructorsDownload anonymised, work and summary sheet MarkersMark work, create feedback, record marks Admin Archive work, individually match and return feedback and marks (through e-Vision or office) StudentsAccess / collect feedback and marks

Enhanced anonymous submission and feedback workflow (Phase 1: ) WhoProcess Students Submit work online through VLE, automated ed receipt VLE instructors Download anonymised work, summary sheet, individual feedback forms* and mark-sheet Markers Mark work with disability info, complete feedback forms, record marks in mark-sheet Admin Archive work, batch upload feedback & marks to SITS / e- Vision StudentsAccess / collect feedback and marks from e-Vision

E-Assignment architecture

Revised workflow Plan Assessment Assessment created Coded in SITS Anonymous Assignment Set up submission point Associate with SITS assessment Student Submission Students submit work Marking Feedback documents downloaded Marks spreadsheet download Disability stickers and information included Department sends documents to markers Uploading into SITS Documents uploaded into SITS Feedback and marks released to students E-Assessment, marking and feedback work-flow

Workflow benefits  Generation of (configurable) feedback docs (MS Word) inheriting exam ID and disability sticker fields, bundled with assignment scripts in zip file –faster work allocation to markers  Batch upload of marks and feedback docs to SITS –Speeds up return of feedback and marks to students  1 week’s administrative time saved across Politics department (for mark entry for 1600 submissions) –use of feedback template *may* lead to greater transparency in marking process (standardisation & quality)  Summer pilots have led to the following enhancements…..

Enhancements to the anonymous submission & feedback workflow (Phase 2: )  file submissions restricted to a specific file type (pre-selected by the assessment manager)  assessment manager(s) alerted when late submissions have been made  reporting on SITS registered students who have not yet submitted  facility for markers to annotate / insert comments into file submissions as student feedback to be batch uploaded to SITS  Student view of all previously submitted files

Next steps: Phase 3 - enhancing the management and practice of online marking ( )  Processes to support work allocation: –Automated allocation of feedback sheets and scripts to markers  Enhancements to marking experience –Blind marking (student IDs concealed) –Automatic plagiarism checks (Safe Assign & Turnitin)  Assessment management –Reporting & moderation screen  Support for supervisor’s access to marks & feedback Building on SITS – Blackboard data integration (based on LIS 2.0 standards development)??

Next steps: Phase 3 - building a student feedback hub ( ) Creation of individual marks & feedback hub for each student; automatically drawing on formative and summative records stored in VLE Grade Centre, with hooks for data ingest from other sources Supervisor sets key questions / reflection template Automated transfer of assessment feedback / marks to individual hub Student reflects on mark / feedback (feed forward) Supervisor reviews & comments on student's progress

Questions? University of York, UK Richard Walker Reports and summaries on e-assignment workflow management and marking provision are available at: