AUN-QA Assessments, Challenges and Lesson Learnt Dr. TAN Kay Chuan AUN-QA Chief Quality Officer and Lead Assessor Director, Office of Quality Management National University of Singapore

Outline Background Initiation Implementation Improvement Trial implementation & challenges Tools development Implementation Actual quality assessments Challenges Special projects Improvement Lesson learnt and Challenges

AUN-QA Operating Guidelines Background AUN-QA Revised Manual AUN-QA Operating Guidelines Bangkok Accord AUN-QA Guidelines AUN-QA Assessments AUN-QA Manual 2004 2006 2007 2000 Present Initiation Implementation Improvement

Bangkok Accord Appoint Chief Quality Officer (CQOs) Formulate common AUN-QA policies, criteria and benchmarking procedures Identify and encourage the implementation of QA good practices Enhance mutual collaboration and information exchange Invite and facilitate auditing, assessment and review by other member universities as well as by external bodies Be responsible for the implementation of this Accord Seek further and deeper engagement regarding QA in higher education with ASEAN Dialogue Partners

AUN-QA Guideline Guidelines are benchmark standards towards quality and quality assurance Strengthen the position of higher education in ASEAN Promote the harmonisation of quality assurance in higher education within and outside ASEAN Endorsed by AUN Board of Trustees as well as Presidents, Rectors and Vice-Chancellors of AUN member universities

AUN-QA Guideline AUN-QA Criteria Criteria 1: QA System Criteria 2: Teaching and Learning Course Curriculum Academic Staff Student Assessment Learning Process Environmental Health and Safety Standards Learning Resources Criteria 3: Research Funding and Facilities Research Output Criteria 4: Services Criteria 5: Ethics Criteria 6: Human Resource Development (HRD)

AUN-QA Manual (2006) Facilitate implementation of AUN- QA Guidelines Improve clarity and interpretation of quality criteria and standards Manual for Self-assessment and Auditing Endorsed by AUN Board of Trustees as well as Presidents, Rectors and Vice-Chancellors of AUN member universities

AUN-QA Manual (2006) Tactical (QA at Programme Level) Strategic (QA at Institutional Level) Systemic (Internal QA System) Tactical (QA at Programme Level)

AUN-QA Revised Manual (2011) First revision of the AUN-QA Manual Criteria and assessment process of AUN Actual Quality Assessment at Programme Level Associated resources (templates and samples) Effective August 2011

AUN-QA Standards – Institutional QA Stakeholder Satisfaction Quality Assurance and (Inter)national Benchmarking A C H I E V M N T S Educational Activities Research Community Service Policy Plan Management Human Resources Funding Mission Goals Aims Source: AUN QA Network

AUN-QA Manual – IQA System Internal Quality Assurance Follow up Student Progress Pass Rate Drop-out Rate Feedback from the Labour Market and Alumni Research Performance Evaluation Course and Curriculum Service Assurance Assessments Quality Staff Facilities Student Support SWOT Analysis Inter-Collegial Audits Information System Handbook Monitoring Instruments Special QA Processes Specific QA

AUN-QA Standards – Programme QA Stakeholders Satisfaction Quality Assurance and (Inter)national benchmarking Programme Specification Programme Content Programme Organisation Student Assessment Academic Staff Quality Support Staff Quality Student Quality Facilities & Infrastructure Quality Assurance Teaching/ Learning Student Evaluation Curriculum Design Stakeholders Feedback Graduate Profile Pass Rates Drop Out Rates Employability Expected Outcomes A c h i e v m n t s Didactic Concept Student Advice & Support Staff Development Activities Graduation Time

Initiation

Trial Run Burapha University, May 2007 Challenges To put into practice the “Manual for the Implementing of the Guidelines” of AUN-QA models for IQA system, institutional level and programme level Challenges Clarification of assessment criteria and self-assessment report (SAR) Key documents and records in local language Lack of a structured assessment process Lack of assessment and reporting tools

Tool Development Formalisation of assessment process (PDCA)

Tools Development Desktop Assessment Template Standardisation of assessment and reporting tools

Example Desktop assessment

Tool Development Assessment Report Template

Example Assessment Report

Tool Development Assessment Feedback

Tool Development SAR Diagnosing

Requirement of Self-Assessment Report Part 1: Introduction Content page Executive summary Organisation of the self-assessment Brief description of the university, faculty and department Part 2: AUN-QA Criteria Requirements Write-up on how the university, faculty or department addresses the requirements of the AUN-QA criteria (use checklist as a reference) Part 3: Strengths and Weaknesses Analysis Summary of strengths Summary of Weaknesses Completed checklist Improvement plan Part 4: Appendices Glossary Supporting documents and evidences

Implementation

Actual Assessment (2007 Onwards) 4 Years 7 Universities 13 Assessments 32 Programmes

Actual Assessment No. Period University Programme 1st Dec 2007 University of Malaya Biomedical Engineering Computer-Aided Design/Manufacture Engineering 2nd Aug 2008 De La Salle University Chemical Engineering Applied Economics 3rd Dec 2008 Universitas Indonesia Civil Engineering Mechanical Engineering 4th Institut Teknologi Bandung Physics Pharmacy 5th Oct 2009 Universitas Gadjah Mada Pharmaceutical Sciences Chemistry Medical Education 6th Dec 2009 Vietnam National University - Hanoi Information Technology

Actual Assessment No. Period University Programme 7th Dec 2009 Vietnam National University – Ho Chi Minh City Information Technology Computer Science & Engineering Electronics & Telecommunications 8th Oct 2010 Universitas Indonesia Architecture Electrical Engineering Chemistry Metallurgy & Material Engineering 9th Nov 2010 De La Salle University Psychology Literature 10th Dec 2010 Vietnam National University - Hanoi Economics 11th Jun 2011 Universitas Gadjah Mada Biology Geology Engineering Civil & Environmental Engineering 12th Nov 2011 Physics Applied corporate Management Software Technology 13th Dec 2011 Biotechnology Manufacturing Engineering Vietnamese Studies

Results Overview

Challenges in Actual Assessment Shortage of qualified assessors Different stages of QA development and maturity within ASEAN Improving assessment efficiency and effectiveness Recognition of AUN QA Assessment Harmonisation of AUN QA framework within and outside ASEAN Sharing the knowledge gained and best practices

Special Projects Initiative for ASEAN Integration (IAI-CLMV) Project ADB-CLM Project (Supported by Asia Development Bank) ASEAN-QA Project (Collaboration with DAAD, Germany) Training for Non-member Universities

ASEAN-QA: Joint QA Capacity Development in the ASEAN region IQA: University QA Officers Intro IQA Theory SAR Writing Finalizing SAR Sensitization: Delegation of Vice-Presidents Site Visits Intro EQA Theory Peer Training Preparing Site Visits EQA: Staff of QA Agencies

Training for Non-member Universities Sep 2011, 1st Workshop Apr 2012, 2nd Workshop Dec 2012, 3rd Workshop More than 50 participants from the ASEAN Region

CLM Project Enhance and strengthen the knowledge on QA system implementation and management particularly based on AUN-QA system, Build up qualified University QA personnel team in CLM countries, Establish QA system to leverage the quality of education management in CLM countries, and Provide countries with options for harmonizing university QA system to ensure applicability across Southeast Asia.

Improve

Training of New Assessors 1st AUN New Assessors Training Workshop (2008) 7 new assessors 2nd AUN New Assessors Training Workshop (2009) 12 new assessors Growing of qualified assessors from 6 to 25 Development path: junior to senior assessors

Revision of AUN-QA Manual for Programme Level Original AUN-QA Model for Programme Level Revised AUN-QA Model for Programme Level 18 criteria, 72 statements 15 criteria, 67 statements

Guidelines for AUN Quality Assessment and Assessors Certification

Guidelines for AUN Quality Assessment and Assessors The establishment of an AUN-QA Council which wil govern and formulate policies for the operation of the AUN-QA Network Define roles, responsibilities and requirements of the applying university, assessors, observers and staff of AUN Secretariat Provide a common frame of reference and accountability to the AUN quality assessment process and stakeholders for the purpose of certification

Sharing the knowledge gained and best practices via proposed IT platform SAR Submissions Assessment Results Progress Reports Best Practices & Benchmarking List of AUN Certified Universities Links to other QA agencies AUN Secretariat Applicant QA Agency Assessor AUN Member Public User 38

Lessons Learnt

Lessons Learnt Trial implementation – validation of QA manual Formulate tools and assessment process before actual implementation Start small to gain experience Training and development of assessors Review for continuous improvement Assist CLMV partners within ASEAN to catch-up Gain credibility before considering QA certification Reach out to QA organisations within and outside ASEAN Use of IT to hasten harmonisation and sharing best practices

Challenges Globalisation of education Rising demand for better quality university education ASEAN Charter - promote greater mobility of students, faculty members, talent and labour Harmonisation of AUN-QA and national/local quality assurance framework for higher education

Quality Assurance in ASEAN 11 Countries: Different culture, language, political system, economy and HEI system Common commitment to QA Different approaches, but many similar criteria Most have QA agencies All have IQA, but EQA varies

Quality Assurance in ASEAN Indonesia Malaysia Philippines Thailand Either No Quality Assurance Agency (QAA) or just being established QAA without National Qualification Framework (NQF) QAA + NQF As of end 2011 Brunei-Darussalam Singapore Cambodia Myanmar Lao PDR Vietnam Timor Leste SEAMEO RIHED: Quality Assurance Landscape in Southeast Asia

Comparison of Assessment/Accreditation Criteria AUN-QA Timor Leste Thailand Philippine Malaysia Indonesia Vietnam 1. Expected Learning Outcomes √ 2. Programme Specification   √ 3. Programme Structure and Content 4. Teaching and Learning Strategy   5. Student Assessment 6. Academic Staff Quality 7. Support Staff Quality 8. Student Quality 9. Student Advice and Support 10. Facilities and Infrastructure 11. Quality Assurance of Teaching and Learning Process 12. Staff Development Activities 13. Stakeholders Feedback 14. Output 15. Stakeholders Satisfaction

Harmonisation of AUN QA framework within and outside ASEAN

Thank You nus.edu.sg