Medicine, Nursing and Health Sciences Quality Assurance and Evaluation in Education Assoc. Prof Leanne Boyd Director of Academic Programs (Middle East) Head, Department of Community Emergency Health and Paramedic Practice

2 Session Outline This presentation will be in three parts: 1. The Monash University approach to quality, which is conceptualised in terms of a continuous cycle of planning, action, evaluation and improvement will be outlined; 2. The differences between quality assurance and evaluation will be explored: and 3. The structure and function of accreditation bodies within higher education will be discussed.

3 Session Objectives At the completion of this workshop, participants should be able to: 1)Differentiate between Quality Assurance and Evaluation. 2)Apply the quality cycle within a Fatima College of Health Sciences context. 3)Discuss the structure and function of accreditation bodies within higher education 4)Identify strategies to assist in planning for accreditation

4 Section 1: Quality Cycle

5 Monash University Quality Cycle

6 Plan  This includes formal planning at all levels such as broad university level planning (IAT and FCHS strategic plan), discipline planning, program or course planning, campus and team planning. At the individual level it reflects the planning that people do either by project, or over time, including yearly or daily planning. ……plan to embed evaluation

7  This includes all the intentional activities that are undertaken to meet objectives, implement plans and produce outcomes. Act

8 This comprises two major aspects: 1.Monitoring; and 2.Review. Evaluate

9 1. Monitoring Monitoring is a short and medium term activity mainly for developmental or formative purposes. It may use formal or informal methods and make use of existing data or generate new data. Action and monitoring usually develop together, informing each other, hand-in-hand.

10 2. Review Review is a longer term and more formal process that has both formative and summative purposes. (eg. 5 year course reviews)

11 Improve  The process by which the results of evaluation - both monitoring and review - are fed back in order to generate improvement.  Often this causes modification to an existing plan or development of a new plan and thus the cycle commences once more.

12 Section 2: Quality Assurance and Evaluation

13 What is evaluation?  Joint Committee on Standards for Educational Evaluation (1994) states that: “...evaluation is a systematic assessment of the worth or merit of an object.” (p.3) (3)

Evaluation is a part of everyday life  You probably evaluate appearance, peers, students on a daily basis  You evaluate the course at the end of semester  Student feedback evaluates you

Evaluative Research  Different to ordinary research  More than just making observations and collecting data  Two processes 1.Observation and measurement 2.Comparison of what you observe with some criterion or standard of what is considered to be an indication of good performance

Types of Evaluations There are many different types of evaluations depending on the object being evaluated and the purpose of the evaluation. Perhaps the most important basic distinction in evaluation types is that between formative and summative evaluation.

Formative Evaluation -Designed to strengthen or improve the object/program being evaluated - Help form the program by examining the delivery, technology, quality of its implementation, and the assessment of the organizational context, personnel, procedures, inputs, and so on.

Examples of Formative Evaluations  Needs analysis  Evaluability assessments  Implementation evaluations  Process evaluations

Summative Evaluation  Examine the effects or outcomes of a program – they summarize it by describing what happens after program implementation;  Assesses cause and effect (whether the object can be said to have caused the outcome)

Examples of Summative Evaluations  Outcome evaluations  Impact evaluations  Cost-effectiveness and cost-benefit analysis  Meta-analysis

21 What is the difference between quality assurance and evaluation? Quality Assurance is a process in which evaluation is an integral part HOWEVER Evaluation can also occur in isolation and can have alternative agendas

Competing perspectives Project Worker  Did the program achieve its objectives?  What needs to change?  Further research? Organisation  Efficiency and cost  To be accountable for accreditation  Future planning and allocation of resources Funding Body  Demonstrate program effect for political purposes  Provide evidence for more financial support Communities  To increase participation  To promote +ve public relations

23 Where does it fit together?  The quality cycle is the model Monash University uses for quality assurance  Evaluation is an integral component of quality assurance, however, it is subjective.  Enter External Quality Agencies (EQA)

24 Section 3: Accreditation

Accreditation  “Accreditation stresses the ‘gatekeeper’ role of an external quality agency (EQA), holding Higher Education Institutions (HEIs) to threshold requirements... Conventionally, accreditation has become associated with accountability of the institution (to someone, for something)” (1)

26 Rapid growth in EQA’s in Higher Education due to:  Increased numbers of students  Fiscal accountability  Increased government attention to national needs for graduates, leading to governments wanting to hold HEIs explicitly accountable for the nature of the graduates they produce. Why do we need External Quality Agencies?

27  Increasing demand for HE, leading to increasing numbers of private providers, some of dubious provenance, leading to a demand for stringent external checks on them  Globalisation, leading to transnational mobility of students and educational export, which in turn leads to a need to have a national QA process that is visible to other countries (for educational export)

28 COMMISSION FOR ACADEMIC ACCREDITATION, UAE “To assure prospective students, their families, and the public that the academic programs offered by institutions licensed in the U.A.E. meet international standards, each program offered is individually accredited.” (4) EXTERNAL QUALITY AGENCY FOR HIGHER EDUCATION IN UAE

 The CAA released a new edition of the Standards for Licensure and Accreditation in 2011;  The eleven standards of excellence and the criteria that all institutions must meet for licensure and program accreditation provide measures of quality and also reflect a consensus within the international higher education community about the essential characteristics of institutions that achieve a level of excellence and continuously improve. (4)

How often does this formal process occur?  Licensed institutions in the United Arab Emirates must receive Initial Accreditation for each academic program they plan to offer before recruiting students or enrolling them.  After Initial Accreditation of a program, an institution applies for Accreditation within two years after the graduation of the first cohort of students.  Accredited programs must apply for the Renewal of Accreditation every five years. (4)

31 In addition to CAA accreditation, each health care discipline has its own accreditation processes to monitor quality within the discipline Professional accrediting bodies vary according to the discipline. Some health science disciplines have international accreditation capacity, many do not. Discipline Accreditation

Breakout groups  What data do you collect in your role at the moment?  How is that data stored? Is it backed up and easily retrieved?  Is it a planned approach based on established criteria?

Some examples of data needed for accreditation  Needs analysis  Policies and procedures  OH&S  Governance, Terms of reference, Minutes of meetings  Student data, enrolments, withdrawals, grade distributions, file notes  Support structures for students and staff  Resources  Industry engagement  Student and staff evaluation data  Staff development

35  Quality cycle needs to be overt, understood and implemented within all elements of the organisation. In the planning phase, identify what data will be needed and include this in your processes  Evaluation needs to be embedded in all that you do  Data must be collected in a systematic way and stored for easy retrieval Summary

36 Plan to evaluate and evaluate your plan Summary

38 1. Woodhouse, D A short history of quality. CAA Quality Series No Woodhouse, D. & Kristoffersen, D. (2006), ‘An Overview of World Issues in Quality Assurance’, AUQA Occasional Publication No 7, August 2006, Australian Universities Quality Agency, Melbourne References

39 3. Stufflebeam D, Shinkfield A. Evaluation Theory, Models and Applications. San Francisco: Jossey-Bass Commission for Academic Accreditation (CAA) United Arab Emirates Available at: ?tabindex=0&tabid=1 ?tabindex=0&tabid=1 References

 Group Image: