College of HSS - Course and Programme Monitoring System Ben Armstrong, IS Applications

Product Vision “A flexible, scalable system all schools can use to simplify the reporting and analysis of the quality of taught courses, helping to turn reflections into action and enhance teaching and learning”

Key Benefits Standardised process workflows Reduced re-keying of data Easier access to trusted, relevant data from core university systems Better report tracking Enhanced management information

Roles Course Reporter (e.g. Course Organiser, or Course Manager) Programme Reporter (e.g. Programme Director) Subject Area Reporter (e.g. Head of Subject Area, or Subject Area QA Rep) School QA Director Course/Programme Secretary System Administrator (one for each School) Reviewer (e.g. UG/PG Teaching Director, UG Director) – and at college level

Target Process Course Report published Can begin drafting Prog/Subject Report before all course reports are published System to make school reports available for College staff to view for onward reporting 1. Course Reporter logs into online QA system to view courses, then writes and submits course report 2. System updates completion status Course Report published 4. Writes and submits Programme/Subject reports, system updates status 3. Programme/Subject Reporter logs into system Programme/Subject Area Report published 5. School QA Director logs into system for review 6. Writes and submits school report 7. System updates completion status QA Director Report published System to be configured to send reminders according to school requirement

High Level Requirements HSS QA System Course Reporting Programme and Subject Area Reporting School QA Director Review Administration

Course Reporting Course Reporting View assigned courses View performance data View student feedback View archive reports View deadlines and guidance Customise question sets Save draft reports Submit completed reports Note that the views of data (e.g. student performance and feedback) are currently planned to be obtained from the student data hub/dashboard currently being developed by Student Systems.

Programme and Subject Area Reporting View assigned Programmes/ Subjects View performance data View student feedback View course reports View archive reports View external examiner reports View deadlines and guidance Customise question sets Save draft reports Submit completed reports Note that the views of data (e.g. student performance and feedback) are currently planned to be obtained from the student data hub/dashboard currently being developed by Student Systems.

School QA Director Review/Reporting View Course/ Prog/Subject reports (completed and outstanding) View performance data View student feedback View external examiner reports View archive reports View deadlines and guidance Save draft reports Submit completed reports Note that the views of data (e.g. student performance and feedback) are currently planned to be obtained from the student data hub/dashboard currently being developed by Student Systems.

System Administration Create and maintain user roles Issue system reminders Update question sets It is intended that the system will integrate with EUCLID to maintain administrative data about each school’s courses, programmes, their course organisers and programme directors. EUCLID will also provide the mapping of courses to programmes and vice versa. Courses and programmes will be created and closed in an automated way drawing on EUCLID.

System design principles Each user will be assigned only the views and permissions within the system relevant to their user role Four report templates, based on a question and answer format, will be created which comprise the Course Report, Programme Report, Subject Area Report and School Report Each report template when opened will display pre-populated header information drawn from EUCLID about the relevant course, programme or subject (e.g. date, title, code, responsible owner) The report templates will contain a core minimum set of questions confirmed by the Senate QA Committee, with a facility created for schools to expand this to include additional questions that only they can view The system administrator for each school will be able to select to aggregate courses to the appropriate level; either to Programme, or Subject Area or straight to School level Any local processes (e.g. report sign-offs, local student surveys) will be managed outside the system The system will avoid data duplication by drawing data from ‘golden copy’ data, where feasible

Agile Highlights Bill Lee

Agile Context Agile manifesto 2001 Extreme Programming (XP) Scrum DSDM Not new, not a local invention. Now well established in many parts of the software industry. The agile manifesto presented an umbrella under which each of the software development techniques could exist as frameworks. Feature Driven Development Pragmatic Programming

What’s the difference? More frequent engagement with the project Requirements captured by User Stories The Product Owner role Use of a Product Backlog Developers want to communicate face-to-face

Why Agile? Nice way to work Flexibility in feature delivery Lean principles – ‘just enough’ Fixed time / budget Continuous improvement Deeper understanding Quality built-in Nice way to work – total team buy in. You are in charge of what is delivered when – this gives you the flexibility to adjust as the project unfolds Lean process – ‘just enough’ Fixed time / budget (less admin) Estimation process more realistic Continuous improvement - through feedback Deeper understanding of the requirements through direct contact with the business Quality built in – use of automation

Colin How does Colin want to use a new Online QA System? Course Organiser How does Colin want to use a new Online QA System? View a complete list of all course reports he is responsible for View performance data for his courses View student feedback for his courses from all the different surveys View guidelines for course review (plus templates) within system View submission deadlines for his reports Customise question sets (core questions plus additional as required) Save drafts of partially completed course reports to return to later Complete and submit annual course reports Access archive course reports from prior years Tagline: “I want to have all the right information at my fingertips to deliver timely, accurate and insightful course reports” The above is a sample User Persona for School QA Director. User Personas are fictional characters created as part of the Agile methodology to help capture the requirements for each User Role. An Agile implementation will create a character for each distinct role in the new system.

Peter How does Peter want to use a new Online QA System? Programme Director How does Peter want to use a new Online QA System? View a complete list of all programme reports he is responsible for View performance data for his programmes View course reports for his programmes View student feedback for his programmes from all the different surveys View external examiner reports for his programmes View guidelines for programme review (plus templates) within system View submission deadlines for his reports Customise question sets (core questions plus additional as required) Save drafts of partially completed programme reports to return to later Complete and submit the annual programme report View archive programme reports from prior years Tagline: “I want to easily view all course reports relevant to my Programme and have an efficient means to analyse the key themes in order to deliver timely, accurate and insightful programme reports” The above is a sample User Persona for Programme Director. User Personas are fictional characters created as part of the Agile methodology to help capture the requirements for each User Role. An Agile implementation will create a character for each distinct role in the new system.

Sarah How does Sarah want to use a new Online QA System? School QA Director How does Sarah want to use a new Online QA System? View course, programme and subject area reports for her school View student feedback for her school View external examiner reports for her school Track status of all reports and issue reminders Export analysis for use within separate school report outside QA system View archive course and programme reports from prior years View guidelines for school QA review View submission deadlines for her school Tagline: “I want to easily view all reports relevant to my School to provide an efficient means to analyse the key themes in order to inform my School QA report and reduce the time required in this process” The above is a sample User Persona for School QA Director. User Personas are fictional characters created as part of the Agile methodology to help capture the requirements for each User Role. An Agile implementation will create a character for each distinct role in the new system.