SEMESTER /2015 QUALITY AUDIT – AUDIT KUALITI

Audit kualiti dalam pendidikan tinggi (Quality Audit in Higher Education) APRIL 2013 Intended Learning Outcomes – student be able to:   Explain the meaning of quality audit   Describe the components / approaches of quality audit   Propose suitable approaches to conduct quality audit in HEI Quality Audit

Audit kualiti dalam pendidikan tinggi (Quality Audit in Higher Education) A Quality Audit has been defined as: A systematic and independent determination of whether: the planned arrangements are suitable to achieve the objectives; the actual quality activities conform to the planned arrangements; and the arrangements are being implemented effectively. (Australian Universities Quality Agency (AUQA), 2009) Relevant web site: Quality Audit

British Standards  Quality audit is a systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives.

Quality Audit The procedural controls that ensure all quality providers are adequately following the required procedures as prescribed. Quality Audit

Dimensions of evaluation AUQA methods of Evaluation: Evaluates institution quality assurance arrangements:   Approach   Deployment   Results   Improvement   Does not impose an externally prescribed set of standards upon auditees, but uses it as primary starting point for audit of organization’s own objectives. Quality Audit

AUQA AUQA Objectives AUQA has several objectives:   Arrange and manage a system of periodic audits of the quality of academic activities, the quality assurance arrangements, institutional compliance to criteria set out in the National Protocols for Higher Education Approval Processes.   Monitor, review, analyze and provide public reports on the quality of outcomes in Australian universities and higher education institutions.   Arrange and manage a system of periodic audits of the quality assurance processes, procedures, and outcomes.   Publicly report periodically on matters relating to quality assurance. Quality Audit

Approach The trail from an organization’s mission, vision and values (the overall objectives). Audit questions include:   What is this organization about?   What outcome is it trying to achieve?   What, if any, reference points are used in establishing the organization’s objectives?   How does the organization plan to achieve its objectives?   Does it understand its context and capabilities?   Are the organization’s objectives set against appropriate benchmark?   What risk management processes does it have in place?   Is the approach aligned and communicated throughout the organization and more widely? Quality Audit

Deployment The deployment dimension considers whether and how effectively, the approach is being put into effect. Audit questions include:   Is the approach being deployed in the best possible manner? According to whom?   What standards and benchmarks is the organization using to assess this?   If the approach is not being deployed, why not, and how is this managed?   Are staff appropriately trained and resources appropriately deployed, to fulfill the approach? Quality Audit

Results The result dimension looks at results as a means of determining how ell the deployment is achieving the planned approach. Audit questions include:   Is the organization achieving its intended objectives and outcomes?   Does the organization understand why and how it achieved those particular results?   How are the results reported and used within the organization? Quality Audit

Improvement The improvement dimension focuses on whether the organization is actively and continuously engaged with understanding its performance in each of the A-D-R dimensions, and is using this understanding to bring about improvements. Audit questions include:   Does the organization know how it can improve?   How does it know this (e.g. through the use of external benchmarks)?   How is it acting upon this knowledge?   Does the organization have a sustained history of improvement? Quality Audit

Operating Quality Audit  Pre-defined standards are determined  Quality requirements for specific work and implementation will be identified in advance  Specific procedures have to be followed at all stages of implementation  Quality Methods must be defined and followed  Completed work and process implementation must be reviewed for compliance. Quality Audit

The Practice in Malaysia MQA – Malaysian Qualifications Agency  MQA established under the Malaysian Qualifications Agency Act It was the merging of LAN (Lembaga Akreditasi Negara) and QAD (Quality Assurance Division of the Ministry of Higher Education).  MQF (Malaysian Qualifications Framework) was developed to unify and harmonize all Malaysian qualifications.  Code of Practice for Program Accreditation (COPPA) and Code of Practice for Institutional Audit (COPIA) are guidelines, standards and code of practice to ensure quality in the academic performance and institutional effectiveness.  The codes contain an overview of the Malaysian quality assurance system for higher education. Quality Audit

MQA – Malaysian Qualifications Agency definitions  – an evaluation of an institution to determine whether it is achieving its mission and goals, to identify strengths and areas of concern, and to enhance quality.  Institutional Audit – an evaluation of an institution to determine whether it is achieving its mission and goals, to identify strengths and areas of concern, and to enhance quality.  – an evaluation conducted internally by a higher education provider (e.g. the university) to determine whether it is achieving its mission and goals; to identify strengths and areas of concern, and to enhance quality. It generates a Self-Review Portfolio.  Internal Quality Audit – an evaluation conducted internally by a higher education provider (e.g. the university) to determine whether it is achieving its mission and goals; to identify strengths and areas of concern, and to enhance quality. It generates a Self-Review Portfolio.  – an exercise to accredit the teaching, learning and all other related activities to a program provided by a higher education provider.  Program Accreditation – an exercise to accredit the teaching, learning and all other related activities to a program provided by a higher education provider. Quality Audit

MQA – Malaysian Qualifications Agency definitions  Quality Assurance – comprises all those planned and systematic actions (policies, strategies, attitudes, procedures and activities) necessary to provide adequate demonstration that quality is being maintained and enhanced, and the products and services meet the specified quality standards. In higher education, quality assurance is the totality of systems, resources and information devoted to maintaining and improving the quality and standards of teaching, scholarship and research as well as students’ learning experience. Quality Audit

MQA – 7 responsibilities  To implement the MQF as a reference point for the Malaysian qualifications.  To develop standards and criteria and all other relevant instruments as a national references for the conferment of awards with the co-operations of stakeholders.  To quality assure higher education providers and programs.  To accredit programs that fulfill a set of criteria and standards.  To facilitate the recognition and articulation of qualifications  To establish and maintain the Malaysian Qualifications Register (MQR).  To advise the Minister on matters relating to quality assurance in higher education. Quality Audit

MQA Council MQA is headed by a Council comprising a Chairman and 16 members. The functions of the Council are to:  Approve plans and policies for the management of the Agency.  Approve any amendment or update of the MQF.  Approve policies and guidelines relating to audit processes and accreditation of programs, qualifications and higher education providers.  Receive and monitor reports and other information relating to accreditation, institutional audit and evaluation.  Continuously guide the Agency in its function as a quality assurance body. Quality Audit

Malaysian Qualifications Framework - MQF  The MQF is an instrument that classifies qualifications based on a set of criteria that are approved nationally and benchmarked against international best practices.  The framework clarifies the academic levels, learning outcomes and credit systems based on student academic load.  The MQF integrates with and links all national qualifications.  3 types of quality assurance:  3 types of quality assurance: Provisional Accreditation; Full Accreditation; and Institutional Audit. Quality Audit

COPIA, COPPA, HEP, MQA, MQF  COPIA – Code of Practice for Institutional Audit  COPPA – Code of Practice for Programme Accreditation  HEP – Higher Education Provider / Pemberi Pendidikan Tinggi  MQA – Malaysian Qualification Agency / Agensi Kelayakan Malaysia  MQF – Malaysian Qualifications Framework / Kerangka Kelayakan Malaysia (KKM) Quality Audit

Code of Practice for Institutional Audit (COPIA)   Contains guidelines on criteria & Standards for Higher Education Provider (HEP).   Define requirements for higher education in broad terms.   Divided into 9 areas of evaluation.   Standards are categorized into 2 attainment levels:   Benchmark – expressed as a “must”.   Enhanced – expressed as a “should”   Purpose:   multipurpose audit code, comprehensive.   Guided information template.   Explains preparations for audits, roles of audit panels, MQA   Explains processes, responsibilities, audit criteria, report writing formats. Quality Audit

  Pendidikan tinggi merupakan pemangkin utama penjanaan sumber tenaga manusia. Pelan Strategik Pendidikan Tinggi Negara menggariskan 7 tuntutan cabaran pendidikan tinggi:   Peranan universiti dan ahli akademik   Perkembangan kurikulum mengikut keperluan pasaran   Penyelidikan, pembangunan dan pengkomersialan dalam sistem inovasi kebangsaan   Kaedah pengajaran dan pembelajaran   Globalisasi dan piawaian melalui pemeringkatan dan penarafan   Peluang guna tenaga Quality Audit Senario pengajian tinggi negara

Findings and Judgments of COPIA   For Academic Performance Audit (APA):   highlight affirmation, commendation and areas of concern.   Indicate performance of institution on 9 areas of evaluation.   No specific decision of pass or fail.   For continuation or cessation of Program Accreditation:   Maintain accreditation   Cease accreditation   For Self-accreditation Status:   Grant Self Accreditation Status   Not granted Self Accreditation status   MQA Institutional Audit Committee will make decisions base on proposal by the panel of assessors. Quality Audit

Malaysian Qualifications Framework - MQF  Provisional Accreditation – seeking approval from the Ministry of Higher Education (MOHE) to conduct new programs.  Full Accreditation – a conferment to denote that a program has met all the criteria and standards set for that purpose.  Institutional Audit – periodic academic performance audit on all universities or to establish the continuation or maintenance of program accreditation status. It could take the form of an exercise for purposes of verifying data, public policy input or for rating and ranking of institutions and programs. The highest form of IA is the self- accreditation audit which can lead to a conferment of a self-accreditation status for the institution whereby the institution can accredit its own programs. Quality Audit

9 Areas of evaluation COPIA  Vision, mission, educational goals and learning outcomes  Curriculum design and delivery  Assessment of students  Student selection and support services  Academic staff  Educational resources  Program monitoring and review  Leadership, governance and administration  Total continual quality improvement Quality Audit

2 levels of standards attainment  – are standards that must be met and compliance demonstrated during institutional audit. Benchmark standards are expressed as a “MUST”.  Benchmark standards – are standards that must be met and compliance demonstrated during institutional audit. Benchmark standards are expressed as a “MUST”.  Enhanced standards – are standards that should be met as the institution strives to continuously improve itself. Enhanced standards reflect international and national consensus on good practices in higher education. Enhanced standards are expressed by a “SHOULD”. The standards define the expected level of attainment for each criterion and serve as a performance indicator. Quality Audit

9 Areas of evaluation – Benchmark standards Statements of Vision, Mission, and Educational goals:  Set the direction, guide academic planning and implementation, and bring members towards achievement of world class status and tradition of excellence.  HEP state vision, mission and educational goals and make known to internal and external stakeholders.  HEP must state vision, mission and educational goals and make known to internal and external stakeholders.  Mission statements and goals reflect crucial elements of the processes and outcomes of higher education.  Mission statements and goals must reflect crucial elements of the processes and outcomes of higher education.  formulate goals consistent with vision and mission.  Must formulate goals consistent with vision and mission.  Publications accurately cite the current vision and mission.  Publications must accurately cite the current vision and mission. Quality Audit

9 Areas of evaluation – Enhanced standards Statements of Vision, Mission, and Educational goals:  Mission and goals encompass leadership qualities in the areas of social responsibility, research attainment, community involvement, ethical values, profesionalism and knowledge creation.  Mission and goals should encompass leadership qualities in the areas of social responsibility, research attainment, community involvement, ethical values, profesionalism and knowledge creation.  demonstrate that planning and evaluation processes, educational programs, educational support services, financial and physical resources, administrative processes are adequate and appropriate to fulfill stated purpose of mission and goals.  Should demonstrate that planning and evaluation processes, educational programs, educational support services, financial and physical resources, administrative processes are adequate and appropriate to fulfill stated purpose of mission and goals. Quality Audit

9 Areas of evaluation – Benchmark standards Curriculum Design and Delivery:   Must have a clearly defined process by which the curriculum is established, reviewed and evaluated.   All Educational programs must be considered only after a needs assessment have been identified.   All Educational programs must be considered only after resources to support the programs have been identified.   All aspects of educational programs must be related to HEP’s vision and mission.   Must show that educational content and approach, teaching learning methods are appropriate, consistent with and support the attainment of learning outcomes.   Teaching learning methods must ensure that students take responsibility for their own learning and prepare for life long learning. Quality Audit

9 Areas of evaluation – Enhanced standards Curriculum design and delivery:  The curriculum encourage multi-disciplinary approaches to enhance personal development through electives, study pathways and other means which should be monitored and appraised  The curriculum should encourage multi-disciplinary approaches to enhance personal development through electives, study pathways and other means which should be monitored and appraised  There be co-curricular activities that will enrich students’ experiences, foster personal development and responsibility.  There should be co-curricular activities that will enrich students’ experiences, foster personal development and responsibility. Quality Audit

9 Areas of evaluation – Benchmark standards Assessment of students: Assessment and Learning   Assessment principles, processes, methods and practices must be aligned with learning outcomes and program content.   The process and methods of assessment must reflect the change to the program outcomes from any review exercise.   The assessment must be consistent with the levels defined in the MQF and the MQF eight domains of learning outcomes. Quality Audit

9 Areas of evaluation – Enhanced standards Assessment of students: Assessment and Learning  Assessment methodology be reviewed periodically to ensure currency with development in best practices.  Assessment methodology should be reviewed periodically to ensure currency with development in best practices. Quality Audit

9 Areas of evaluation – Benchmark standards Assessment of students: Assessment methods   Frequency and methods of student assessment including grading criteria and award must be documented and communicated to students.   The assessment methods must comprise of summative and formative purposes compatible with learning outcomes.   A variety of methods and tools must be used appropriately for assessing the given learning outcomes and competencies (communication, problem solving, teamwork and self-directed learning).   There must be mechanisms to ensure the validity, reliability and fairness of the assessment system. Quality Audit

9 Areas of evaluation – Enhanced standards Assessment of students: Assessment methods  Methods of assessing be comparable to international best practices.  Methods of assessing should be comparable to international best practices.  The assessment systems should be reviewed periodically. Quality Audit

9 Areas of evaluation – Benchmark standards Student selection and support services: admission and selection – 13 standards (here are five examples)   The HEP must have an admission policy with clear statements on the criteria and processes of student selection including transferring students.   The criteria and processes of selection must be published and disseminated to students.   If a selection interview is used, the HEP must demonstrate that it is objectively and fairly structured.   Student selection must be free from discrimination and bias.   The HEP must provide evidence that it selects students whose capabilities are consistent with the admission policies. Quality Audit

9 Areas of evaluation – Enhanced standards Student selection and support services: admission and selection   The admission policy should be reviewed periodically.   The outcome of selection and student performance should be monitored to improve the selection process.   The size of student intake should be reviewed in consultation with stakeholders.   Students intake should include a consideration of gender and ethnic balance and social responsibilities such as policies for disadvantaged students. Quality Audit

The Practice: Academic Performance Audit – Audit Prestasi Akademik   Effectiveness of the internal management systems for quality by which institutions manage their core functions with resources and manpower available.   Strength & areas of concern in the academic performance of the universities.   Improvements needed in the areas of concern.   Action plans by institutions.   Institutions demonstrate that they have systems and resources in place for institutional effectiveness. Quality Audit

The Practice: Academic Performance Audit – Audit Prestasi Akademik   The audit is in the form of an external quality assurance in which institutions perform self evaluation against standards and criteria.   The quality assurance standards and criteria are intended to help develop, enhance, and strengthen the institutional effectiveness.   The results do not “pass” or “fail”.   Reports are not intended to penalize or reward institutions.   Information will give trends, differences between institutions, indication of impact of policy implementation, constraints and expectations. Quality Audit

Some Comparisons between COPIA & COPPA   Area 1: Vision, Mission, Educational Goals & Learning Outcomes   1.1 Statements of vision, mission and educational goals:   COPIA – sets the direction of the HEP and guides educational planning & implementation.   COPPA – at the program level, it is more directed to see how a specific program supports the larger institutional vision. Quality Audit

Some Comparisons between COPIA & COPPA   Area 1: Vision, Mission, Educational Goals & Learning Outcomes   1.3 Academic Autonomy:   COPIA – specifies the autonomy of HEP over academic matters.   COPPA – autonomy in designing the curriculum of a specific program. Quality Audit

Some Comparisons between COPIA & COPPA   Area 1: Vision, Mission, Educational Goals & Learning Outcomes   1.4 Learning Outcomes (LO):   COPIA – relates LO and competencies to vision and mission of HEP.   COPPA – emphasis on learning outcomes of a particular program. Quality Audit

Some Comparisons between COPIA & COPPA   Area 2: Curriculum Design and Delivery   2.1 Curriculum Design and Teaching Learning Methods:   COPIA – perspective of curriculum design is largely about institutional policies, structures, processes and practices related to curriculum development across the institution.   COPPA – refers to specific description, content and delivery of a particular program. Quality Audit

  Higher education as catalyst for human capital development National Higher Education Strategic Plan:   Academics and University Roles   Curriculum development in line with current needs   Research, development and commercialization   Teaching and learning methodology   Globalization and standardization   Opportunities for human resources utilization Quality Audit Higher education scenario

  Universiti awam dikategorikan kepada tiga kumpulan iaitu universiti penyelidikan, universiti berfokus (teknikal, pendidikan, pengurusan dan pertahanan), universiti komprehensif. Universiti penyelidikan:   Ciri-ciri – bidang pengajian fokus kepada penyelidikan, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 50:50.   UM   USM   UPM   UKM   UTM Quality Audit Senario pengajian tinggi negara

  Public universities into several categories: APEX university, Research universities, Comprehensive universities, Focus universities Research universities:   Focus on research, intake competitive, quality academics, ratio undergrad and postgrad 50:50.   UM   USM   UPM   UKM   UTM Quality Audit Higher education scenario

Universiti komprehensif:   Ciri-ciri – pelbagai bidang pengajian, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 70:30.   UiTM   UIAM   UMS   Unimas Quality Audit Senario pengajian tinggi negara

Comprehensive universities:   Diversified of program of studies, quality academics, intake competitive, ratio undergrad postgrad 70:30.   UiTM   UIAM   UMS   Unimas Quality Audit Higher education scenario

Universiti berfokus: Ciri-ciri – bidang pengajian berfokus, kemasukan kompetitif, pensyarah berkualiti, nisbah siswazah dan pascasiswazah 70:30.   UUM   UPSI   UTHM   UTeM   UMP   UMT   UDM   UMK   UPNM Quality Audit Senario pengajian tinggi negara

Focus universities Focus program of studies, quality academics, intake competitive, ratio undergrad postgrad 70:30.   UUM   UPSI   UTHM   UTeM   UMP   UMT   Unisza   UMK   UPNM Quality Audit Higher education scenario

 Public universities  Private universities  Branch campus universities – nottingham  Universities wakaf / Fund By Foundation– Albukhari University  kolej university

Some Comparisons between COPIA & COPPA   Area 2: Curriculum Design and Delivery   2.2 Curriculum Content and Structure:   COPIA – relates to HEP’s processes in developing all program content in general.   COPPA – focuses on the processes of developing one particular program. Quality Audit

Some Comparisons between COPIA & COPPA   Area 3: Assessment of Students   3.3 Management of Student Assessment:   COPIA – looks at the HEP’s students assessment system.   COPPA – how is the students assessment system implemented for a specific program. Quality Audit

Some Comparisons between COPIA & COPPA   Area 4: Student Selection and Support Services   4.1 Admission and Selection:   COPIA – how the institution comply to the MOHE admission and selection policies.   COPPA – how are the policies cascaded down (diturunkan) and implemented at the departmental level for a particular program. Quality Audit

Some Comparisons between COPIA & COPPA   Area 4: Student Selection and Support Services   4.5 Alumni:   COPIA – incorporating alumni into curriculum development of HEP.   COPPA – involvement of alumni in developing a specific program Quality Audit

Some Comparisons between COPIA & COPPA   Area 5: Academic Staff   COPIA – looks at HEP policies on academic staff recruitment, appointment and promotion, and academic staff development.   COPPA – looks at policies and requirements for academic staff involved in teaching a specific program. Quality Audit

Conclusion   Quality Audit   In Quality Audit it is expected that all procedural controls are adequately followed according to the prescribed procedures.   The impact of Quality Audit will be across all parts of the quality cycles. Quality Audit