Accreditation for Online Learning Programme (OLP) INQAAHE Biennial Conference 2019 Quality Assurance, Qualifications, and Recognition: Fostering Trust in a Globalised World New Technologies, Innovation and Quality Assurance Accreditation for Online Learning Programme (OLP) Ms Anne LAU Registrar, HKCAAVQ 2019.3.26 HKCAAVQ launched a new service of “accreditation for online learning programme” in April last year in response to a number of requests received in recent years. In case you are not familiar with our organisation, let me introduce ourselves in short. We are the only statutory accreditation authority in HK to accredit programmes for recognition under the Hong Kong Qualifications Framework. We conduct accreditation exercises for different purposes, such as Learning Programme Accreditation for programmes leading to local awards, and accreditation for Non-local Learning Programme with granting bodies outside HK. Before the launch of the OLP accreditation, we conducted a year of desktop study to see how other agencies, such as TEQSA in Australia and CHEA in the US, work on this. At the same time, we conducted a pilot exercise to identify issues as well as solutions regarding OLP accreditation. In this PowerPoint, I will try to summarise the lessons we learnt in the process, and highlight the features of our OLP accreditation.

Approach of OLP Accreditation Definition of OLP Approach of OLP Accreditation Important Considerations Way Forward In the next 20 minutes, I will cover our definition of OLP, our approach to the OLP accreditation with highlights on some important considerations, and finally a conclusion on our way forward. Nowadays, inclusion of some kind of online components in learning is already a norm in higher education, and we called this blended learning mode. The components may be a virtual “storage” area to keep the teaching and references materials for students to download; Or, a video-on-demand server for keeping the video records of lectures for student viewing and reviewing; Or, a discussion forum, or a platform of any kind for students to exchange views and ideas, or even do some kind of collaborative projects; Or, it can be a “venue” for assessments where quizzes and tests be held, and a window for submission of assignments; Or even a chat room where students can have direct dialog with the instructor, no matter human or robot. The only difference is the degree of such components playing in the whole curriculum. If it is only a virtual storage area for download, it will have a minimal impact on the learning process. On the contrary, if it is the only platform for learning, the learning process and the evidence of the learning outcomes may be totally different from the traditional face-to-face delivered programmes. Therefore, to make sure that we are on the same page as our clients, we laid down our definition for OLP.

Definition of OLP structured teaching, learning and assessment “Online delivery is defined as delivery that is enhanced by the use of technology and is delivered through a digital learning platform to provide structured teaching, learning and assessment. Learning programmes with more than 50% of instruction delivered online will be accredited with reference to the additional evidence requirements.” structured teaching, learning and assessment Here is our definition. … The definition is not meant to define the nature of OLP, but to highlight the features we expect for OLP where the online components play a significant role to impact on the programme design and delivery. These features include:  Firstly, the OLP should be a structured learning process. Although “personalisation” is a niche of OLP, the flexibility for adjusting own pace and own path should be carefully designed and directed in the programme, to make sure that the intended learning outcomes can be attained.

Definition of OLP delivered through a digital learning platform “Online delivery is defined as delivery that is enhanced by the use of technology and is delivered through a digital learning platform to provide structured teaching, learning and assessment. Learning programmes with more than 50% of instruction delivered online will be accredited with reference to the additional evidence requirements.” delivered through a digital learning platform Secondly, OLP should be delivered through a digital learning platform. Interactions between students and teachers and among students are important in constructing students’ understanding and meaning of their learning. In traditional face-to-face delivery, the interactions happen in the classroom. For OLP, this is expected to happen on a digital learning platform. The platform should be able to accommodate one-to-one, or even many-to-many interactions.

Definition of OLP enhanced by use of technology “Online delivery is defined as delivery that is enhanced by the use of technology and is delivered through a digital learning platform to provide structured teaching, learning and assessment. Learning programmes with more than 50% of instruction delivered online will be accredited with reference to the additional evidence requirements.” enhanced by use of technology When different online components are included in a programme, it is expected that effectiveness of learning will be enhanced. And the enhancement should be properly measured and evaluated for the sustainable development of the programme.

Definition of OLP more than 50% of instruction delivered online “Online delivery is defined as delivery that is enhanced by the use of technology and is delivered through a digital learning platform to provide structured teaching, learning and assessment. Learning programmes with more than 50% of instruction delivered online will be accredited with reference to the additional evidence requirements.” more than 50% of instruction delivered online For the online components to have obvious impact on the programme design and delivery, we are expecting that the programme is mainly delivered through online, that is more than 50% of the learning activities are delivered online. The percentage can be counted from the notional learning time spent on the activities, or the credits carried by different courses of the programme.

Definition of OLP “Online delivery is defined as delivery that is enhanced by the use of technology and is delivered through a digital learning platform to provide structured teaching, learning and assessment. Learning programmes with more than 50% of instruction delivered online will be accredited with reference to the additional evidence requirements.” These 4 distinctive features of OLP informed our understanding and expectation on the design and operation of an OLP. Based on the understanding and expectation, we laid down our approach of the OLP accreditation.

Approach of OLP Accreditation Eligibility Common Standards Outcome-based Different Learning, Teaching & Assessment Different Evidence There are 5 features in our OLP accreditation; namely Eligibility requirement for OLP accreditation; One common set of accreditation standards for all kinds of programmes; Outcome-based approach in accreditation; Different learning, teaching and assessment of OLP; and Different evidence requirements for OLP. Let’s go through these features one by one.

1. Eligibility Requirement demonstrated competence in operating recognised programme Non-local institution Recognsed status in home country Local institution At least ONE accredited programme recognised under HKQF To be eligible for the OLP accreditation, an institution should possess demonstrated competence in recognised programme. As discussed earlier, OLP is a programme with a mix of delivery methods “enhanced” by technologies. Therefore, the institution offering OLP is expected to be experienced in operating learning programmes in order to identify appropriate technologies to “enhance” the learning.  For an online programme offered by university from, say UK or Australia, the institution has to have a recognised status in its home country.  For institutions in HK, it must be operating at least one accredited and QF-recognised programme. The recognised status is a proof of its competence.

2. One Common Set of Accreditation Standards For each of the accreditation nature, we have a clear purpose. Eg, for a Learning Programme Accreditation, the purpose is two-fold: The first is to ascertain the standard of a “learning programme” as meeting the QF requirements; Then, we also need to ascertain the competence of an “operator”, that is the institution, in managing and monitoring the programme. According to the purpose, we chart out different domains of competence for assessment. Like in a Learning Programme Accreditation, we have 7 domains. And under each domain, we set out the accreditation standard. As you can see from the purpose, the domains and the standards, they are applicable to all programmes of whatever modes of delivery, no matter face-to-face or online or blended. The accreditation standards do not restrict nor confine the design or delivery mode of the programmes. In fact, in our desktop study, we noted that Australia and some European countries also adopt the same standards for accreditation of learning programmes regardless of their delivery modes. Therefore, we have only one common set of accreditation standards for all programmes of different modes of delivery.

3. Outcome-based Approach Organisational Outcomes Mission & Vision KPI Programme Outcomes Graduate Articulations Stakeholder Feedback Learner Outcomes Programme Completion Assessment Results In accreditation, we adopt an outcome-based approach to assess the competence of an institution. That is to assess the different levels of outcomes and their alignment according to the purpose of the accreditation. We will be assessing 3 level of outcomes, namely: The Learner Outcomes which are reflected on the assessment results on completing the programme. Then the Programme Outcomes which are reflected on the further studies and employment pathways of the graduates, and stakeholders’ feedback on the quality of the programme. The final one is the Organisational Outcomes which are reflected on the institution’s achievement of the mission and vision, and attainment of the KPI, the key performance indicators set. The alignment of the design, and effectiveness of operation of a programme, should be reflected on the different levels of outcomes. which are the focus of accreditation, no matter what the delivery modes are.

4. Different Learning, Teaching and Assessment “Online delivery is defined as delivery that is enhanced by the use of technology and is delivered through a digital learning platform to provide structured teaching, learning and assessment. Learning programmes with more than 50% of instruction delivered online will be accredited with reference to the additional evidence requirements.” Support Investment in IT infrastructure Resolving OLP-specific issues Accessibility Time zone differences Technical support Timely feedback PILO – Programme Intended Learning Outcomes As discussed earlier, in our definition for OLP, there are 4 key features. And all of them are related to the learning, teaching and assessment arrangement of a programme. However, our concerns in accreditation are the same, that is: whether the learning and teaching design and arrangement, now put online, align with the claimed objectives, and are effective in facilitating students to attain the learning outcomes; whether support, no matter academic and personal, is adequate to students during their learning, now on the digital platform; whether staff is trained or properly equipped to manage the teaching and learning and assessment, now also on the digital platform; and whether the quality assurance mechanism is able to properly and timely monitor the student learning and the performance of the programme conducted mainly through the online means. So, the concerns in accreditation remain the same, but the issues to be explored in OLP will be different. Like: whether there is sufficient investment in the IT infrastructure to ensure connectivity; and various issues arising from the online mode of delivery, such as accessibility to the online platform, time zone differences when the programme is to be accessible outside the homeland, technical support and timely feedback to students, etc.

5. Different Evidence Requirements Since the issues to be explored will be different, the evidence to be provided by the institution to substantiate their claim of meeting the accreditation standards will also be different.  For this reason, in preparing the Evidence Guide for institutions seeking accreditation, we have included a list of possible source of evidence purposefully for OLP providers. Eg. The QA mechanism must be applicable to OLP. And, learning analytics are being collected to monitor student performance, and review the effectiveness of the online delivery. In the pilot accreditation exercise we conducted, we noted that it is important for a learning platform to provide learning analytics to inform progress of student learning for routine monitoring and support. Therefore, OLP-specific issues have to be supported by OLP-specific evidence.

Important Considerations for OLP Clearly defined and appropriate education objectives. OLP development as one of the strategic goals supported by top-down policies and procedures. Clear targets and milestones. … Organisational Level Learning, teaching and assessment activities effective in delivering content and assessing attainment of intended learning outcomes. Qualified teaching staff well equipped with pedagogies for online teaching. Effective Learning Management System. Programme Level The OLP-specific issues can be broadly categorised as at “Organisational Level” or at “Programme Level”. At organisational level, we are expecting that the development of OLP is a strategic move with clear goals, and supported by top-down policies and procedures. There should also be clear targets and milestones for evaluation of the programme performance. Migrating learning and teaching online should not be a decision on a single programme, nor an ad hoc trial, but an institutional move that reflected in the strategic plans, relevant guidelines, associated committee structures and compositions, related key performance indicators (KPIs), etc. At programme level, the online delivery and assessment should be effective in delivering the content, and evaluating the attainment of the learning outcomes. For assessment, authentication of student identity is one of the key issues to ensure reliability of the assessments in reflecting the learning outcomes of individual students. For staff, expertise or training in virtual learning or related pedagogies is also expected. Whereas the digital platform, or we called Learning Management System, LMS, should also be effective in producing learning analytics to inform progress of student learning and performance. All these are only examples of some key issues related to OLP. As technologies advances quickly, we will not restrict institutions to stick to a specific technology or platform or model for OLP, but the evidence provided must be adequately addressing the requirement of the standards. Through the desktop study and the pilot OLP exercise, we know better the features of OLP, and identified some key issues.  So, we launched the service in April last year.

Way Forward OLP Service Launched Market Demand Survey Briefing and Workshop Collective Exercises April 2018 July - August 2018 October 2018 January 2019 November 2019 March 2020 After the launch, we conducted a market demand survey in the local community, and delivered briefing and workshop to inform interested parties about our approach of OLP accreditation. Two collective accreditation exercises are scheduled later this year and early next year.

Market Demand Survey From the market demand survey, we can see that OLP will be growing quickly in the local community, as one-third of the responding institutions will be offering OLP in the coming 2 year. Among the existing OLP, most of them are in the Business and Management area, The other 2 focused areas are Humanities and Social Sciences. These data provided us a basis for planning and scheduling the coming OLP exercises.

Briefing and Workshop on OLP Briefing in November 2018 66 participants 40 institutions To inform the community about the new service, we held a briefing in November last year. Representatives coming from 40 institutions attended. These institutions included local public-funded universities, self-funded institutions, and overseas institutions offering OLPs. We also organised a workshop in January this year to go into details the accreditation standards and evidence requirements with institutions indicated interest to join the OLP accreditation. Workshop in January 2019 19 participants 8 institutions

Collective Accreditation Exercises Collective Exercise Similar Nature Focused & Efficient Timely Advice Clear Timelines One Panel Generate & Share Knowledge of OLP Accreditation Explore Innovative Arrangements Video conference  Site visit … According to the data collected in the Market Demand Survey and institutions’ indication of interest to join the OLP accreditation collected in the briefing, We scheduled 2 collective accreditation exercises with one site visit in later this year, and another in early next year. By means of “collective exercise”, programmes of similar natures by different institutions will be accredited in one batch by a common panel for better focus and efficiency. The one in this year is in the area of Business and Management, and the one next year is in Humanities and Social Sciences. We see that the collective exercises have a lot of benefits to the institutions as well as to ourselves. For institutions, the collective approach provides clear timelines to them in the preparation of the accreditation exercise; And hence we can offer timely advice and guidance according to the progress of preparation of the participating institutions.  The collective approach is also an efficient means of generating and sharing knowledge and experience for both our internal staff and the accreditation panel members, particularly on key issues regarding OLP, such as authentication of students, and arrangement of valid and reliable online assessments. In OLP exercises, we will explore various innovative and efficient arrangements. For example, online meetings may be arranged for the local and non-local panel members to meet with the representatives of the local and/or non-local institutions of the OLP, instead of confining the meetings to a physical site for the “site visit”.

OLP Accreditation A Learning Process Our OLP accreditation is a result of our learning from other agencies and our predecessors, And, I can see that, the coming OLP exercise will be another learning journey to fine-tune our process and the evidence requirements, and explore for more possibilities.

 annelau@hkcaavq.edu.hk Thank You Anne LAU  annelau@hkcaavq.edu.hk Copyright 2019 © HKCAAVQ. All Rights Reserved. The copyright of this presentation (printed and electronic) belongs to HKCAAVQ and that any use of the printed materials in full or in part must be under prior written consent by HKCAAVQ and with clear reference made to HKCAAVQ as the material source.