Let Quality Guide Evaluation Quality: Recent Trends Implemented in the Middle East An Eclectic Approach to Evaluation for Higher Education Institutions: Lessons Learned in Saudi Arabia Dr. Eqbal Z. Darandari National Commission for Academic Assessment and Accreditation (NCAAA) Mrs. Tahira Hoke Prince Sultan University (PSU) Presented at American Evaluation Association, November 13, 2010

Overview Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia Role of HE Evaluation in Saudi Arabian Higher Education NCAAA’s Assessment, Quality Assurance and Accreditation System: Built in Evaluation Approaches. Common Evaluation Approaches Used by HE Education Quality Assurance Directors Program Evaluation and Paradigm Shift: Benefiting from Other Disciplines and Quality Systems Quality Talk and Walk: How can new trends in quality systems enlighten evaluation quality?

Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia Middle East and Gulf states trends and practices Recent trends and practices implemented in Evaluation and quality systems in higher education. Evaluation- Accreditation Internal-External Quality Management – Quality Assurance/Improvement

Rapid growth in post-secondary institutions. Establishment of Saudi National Commission for Academic Accreditation & Assessment (NCAAA) in In only four years, NCAAA has developed, established, tested, and implemented a new quality assurance system for higher education, benefiting from the international expertise, while keeping its unique characteristics. Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia

Accreditation means…. quality education outcomesIs providing quality education outcomes Meets standardsMeets standards for leadership, facilities, teaching and learning, assessment, ……. (national or international) internal systemHas established an internal system of self evaluation and monitoring. Is committed to external peer evaluation continuous improvementIs committed to continuous improvement Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia

Saudi Institutions Implement the New Quality Assurance System Capacity Building at National and Institutional Level Quality assurance has become central in educational reforms. Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia

Self-evaluation involving negotiations- reflections with clear directions and expectations led to collective group decisions and forced systematic and sustainable improvements. Recent Trends in Evaluation of Higher Education: Middle East - Saudi Arabia

Role of HE Evaluation in Saudi Arabia Higher Education New drives behind Saudi HE Quality Assurance and Accreditation System: Educational Reform, Global Recognition, Public Trust, transformation, and integration of stakeholders and cooperation of various actors.

What is behind HE Assessment + Quality + Accreditation System?

Criteria for Quality Assurance and Accreditation in HE in SA Supporting materials such (such as handbooks detailing quality assurance processes, BQF, Institution + Program Standards, and self- evaluation scales, templates for programme plans and reports, Key Performance Indicators, and student surveys). Fitness for purpose Standards Fitness of Purpose Role of HE Evaluation in Saudi Arabia Higher Education

NCAAA 11 Standards A) Institutional Context: 1) Mission, Goals and Objectives; 2) Governance and Administration; 3) Management of Quality Assurance and Improvement; B) Quality of Learning and Teaching: 4) Learning and Teaching; C) Support for Student Learning: 5) Student Administration and Support Services; 6) Learning Resources; D) Supporting Infrastructure: 7) Facilities and Equipment; 8) Financial Planning and Management; 7) Employment Processes; E) Community Contributions: 10) Research; 11) Institutional Relationships with the Community.

Standards as A system: Reach to the Main Beneficiary (Maffet, 2010) (Maffet, 2010) Systemic Thinking

NCAAA Quality Assurance and Accreditation Process: System Environmental Scan: SWOT Analysis Culture Of Quality Self Study NCAAA External Reviews and Recommendations Building Capacity Meeting Standards + Mission and Objectives Initial Self Evaluation Establishing Baseline Establishing a continuous system of internal evaluation and assessment Addressing Improvement Plan

Shift in learning, teaching, and assessment system System based on Learning Outcomes National Qualifications Framework: specifies generic standards of learning outcomes based on educational level Learning OutcomesCourses Course Code and Number Knowledge Facts Concepts, theories Procedures Cognitive Skills Apply skills when asked Creative thinking and problem solving Interpersonal Skills and Responsibility Responsibility for own learning Group participation and leadership Act responsibly-personal and professional situations Ethical standards of behavior Communication IT and Numerical Skills Oral and written communication Use of IT Basic math and statistics Psychomotor Skills from To NCAAA Quality Assurance and Accreditation Process: Outcomes

Plan Monitor Implement Review Saudi NCAAA Quality Cycle

NCAAA’s Assessment, Quality Assurance and Accreditation System: Built in Evaluation Approaches New quality system borrowed from the effective evaluation and other disiplines tools and approaches Successfully integrated them into multilevel staged transformative processes through designed qualitative change.

Common Borrowed Approaches Input-Processes-Outcome Empowerment Participatory Responsive Critical reflections Self-Evaluation Dialectical Evaluation Developmental Evaluation System thinking Strategy Evaluation

Depending on the context and institution quality culture, quality assurance directors employed evaluation methods most appropriate for the size, type of institution. Common Evaluation Approaches Used by HE Quality Assurance Directors

Note: Multiple Response Item Common Evaluation Approaches Used by HE Quality Assurance Directors : Survey Results

The Percentage Increase in Evaluation Activities in a Sample of Saudi Universities and Colleges Between March 2007 and August (Darandari, et al., 2009; Darandari, & Hoke, 2007)

Quality Assurance as An Empowerment Tool for Women: A Case from Saudi Arabia INQAAHE Conference, 2009 Eqbal Z. Darandari Common Evaluation Approaches Used by HE Quality Assurance Directors : A Case Study

Vision for the Future: How can new trends in quality enlighten evaluation quality. Program Evaluation and Paradigm Shift: Benefiting from Other Disciplines and Quality Systems

Benchmarking Peer Review Context - Dynamics Systems theories and thinking Complex Systems Change theory Multifaceted Need analysis Compliance vs. Reliance Evaluating Strategy Mission Vision Fitting purpose Truth Engagement Context Beyond methods Global thinking Advocate Evaluation Developmental Evaluation Utility Innovation Visualizing Data for monitoring and decision making Performance Measurement Loops Self Evaluation and Reflection Learning Process Self Reporting Emerging Quality Themes in Program Evaluation Evidence based Documented

Evaluation Paradigm Shift: Are we there yet?

Continuous Improvement (Plans-Indicators- benchmarks) Ownership Commitment Quality Culture Leadership Periodic- External and internal Review Quality Asurance Standards + Criteria (Evaluator- Program – Client) Quality Talk and Walk: How can new trends in quality systems enlighten evaluation quality? Framework Alignment Connection ????? Balance and Direction

Standards Quality is not just meeting Standards and following a process protocol. system. It is al about comprehensive quality system. It is about culture and built in continous improvement Quality Talk and Walk:

Improvement Process Components 28 Plan and Implementation What future are we pursuing ? Vision Profile What is our current reality? What actions will we take to improve? What have we accomplished? Results Monitoring Drawing quality map clarified to all stakeholders. Participants need to know where they are, where they want to go, how they are similar or different from others nationally and internationally, what should be changed and to what level, what the alternatives are.

29 Evaluation Act Plan implement Review Continuous improvement Quality Evaluation Needs to work within a system and guide to incremental continuous improvement

30

Dr. Eqbal Z. Darandari NCAAA Consultant King Saud University Psychology Department