National Quality Assurance and Accreditation Committee & Quality Assurance and Accreditation Project Assessment Design and its relationship to NARS and ILOs May, 2007 Mostafa Mohsen Radwan Vice Dean, Faculty of Engineering, Fayoum University Director, Fayoum University Quality Assurance and Strategic Planning Center Member of the Quality Assurance and Accreditation Committee

Assessment Design Intended Learning Outcomes (ILOs) National Academic Reference Standards (NARS) / Benchmarks

Intended Learning Outcomes (ILOs) Learning outcomes may be defined as “Statements of what a learner is expected to know, understand and/or be able to demonstrate after completion of a process of learning”. Intended learning outcomes (ILO) are simply the learning outcomes that were intended by the writer(s) of the module descriptor (Program/Course specification).

ILOs ILOs Teaching, learning and assessment Curriculum design design What students are expected to be able to do Resources Grading criteria criteria In education, ILOs should inform Adapted from Dr. Catherine Tang (2005,2006)

Backwash Effect of Assessment Teacher’s and student’s perspectives on assessment Teacher’s perspective: ILOs Teaching activities Assessment Teacher’s perspective: ILOs Teaching activities Assessment Student’s perspective:Assessment Learning activities Outcomes (or just activities) (or just activities) Assessment becomes the curriculum to the students. * Assessment should be aligned with the ILOs so that students will learn what we intend them to learn.

Testing for Learning, Not Learning for Testing

Assessment is the contextual factor which most strongly affects how students learn. It is the students’ perceptions of the assessment requirements that constitutes the hidden curriculum, and drives how students learn. Assessment system should be an integral part of the learning process, and it is a key element in the success of any education enhancement or reform process.

National Academic Reference Standards (NARS) / Benchmarks

WHY SHOULD STUDENTS PARTCIPATE IN QUALITY? Are students customers in higher education? ◦ If yes, what rights do they have in the institution? Are students stakeholders in higher education? ◦ Do they participate in decision-making ◦ Do they get support in order to participate effectively

……………………………….. …………………………………

National Academic Reference Standards (NARS) / Benchmarks

Common Part

*Practical *General& transferable *Intellectual *Knowledge & understanding

NARS/Benchmark Program Specification Course Specification

Students Assessment Design Often we are more concerned with proving growth and giving marks than we are with providing opportunities for students to learn.

Important Issues of Assessment.Clear purpose..Clear purpose..An integral part of the curriculum design and planning..An integral part of the curriculum design and planning..Relevant criteria based on ILOs..Relevant criteria based on ILOs..Appropriate methods to align with ILOs to be assessed..Appropriate methods to align with ILOs to be assessed.. Clear communication and transparent.. Clear communication and transparent..Consistent procedures..Consistent procedures..Valid and practicable tasks..Valid and practicable tasks.. Realistic workload.. Realistic workload..Provision of constructive feedback to facilitate.Provision of constructive feedback to facilitate and improve learning and teaching.. Regular review and improvement of the system.. Regular review and improvement of the system. Adapted from Dr. Catherine Tang (2006)

Processes Involved in Assessment 1. Set assessment criteria. 1. Set assessment criteria. 2. Select assessment methods to align with the ILOs. 2. Select assessment methods to align with the ILOs. 4. Communicate the results of the judgement and 4. Communicate the results of the judgement and provide constructive feedback. 5. Make use of the results of the assessment to improve teaching and learning. 5. Make use of the results of the assessment to improve teaching and learning. 3. Judge and grade the achievement of the ILOs based on the evidence provided. 3. Judge and grade the achievement of the ILOs based on the evidence provided.

1. Assessment Criteria: What do we want to assess our students?. Should be criterion-referenced.. Based on explicitly set ILOs.. Consider both content and level of understanding or standards of performance to be assessed.. Assessment criteria in-use align with espoused ILOs.. Clear to students and teacher(s) involved.

2. Methods of Assessment Assessment should not be labelled as “good” or Assessment should not be labelled as “good” or “bad”, but rather “appropriate” or “inappropriate”. “bad”, but rather “appropriate” or “inappropriate”. The important issue is how the assessment is designed The important issue is how the assessment is designed and implemented to appropriately address the and implemented to appropriately address the intended learning outcomes. intended learning outcomes.

Different methods of assessment assess different learning outcomes. A range of different methods of assessment may be required to address the range of ILOs of a course. Questions to be asked in selecting assessment methods: 1. What ILOs are to be assessed? 2. Are the assessment methods concerned able to address those ILOs? 3. Will the assessment methods encourage students to relate to real-life contexts? Are they authentic? 4. Are the assessment methods practicable with respect to time allowance and available resources? 5. Taking the course as a whole, will the different assessment methods offer a balanced assessment of all the course ILOs? 6. Is the assessment workload realistic for teachers and students? Selection of Methods of Assessment

Some Common Methods of Assessment Papers Papers Quizzes Quizzes Graded discussions Interviews Interviews Storytelling (digital and narrative) Storytelling (digital and narrative) Group or individual projects Group or individual projects Peer review and peer teaching Peer review and peer teaching WebQuests WebQuests Case studies Case studies Web page production Web page production Media production (pictures, diagrams, photos, videos) Media production (pictures, diagrams, photos, videos) Journals Journals Portfolios Portfolios Demonstrations Demonstrations Critiques Critiques

Constructive Alignment of ILOs and Assessment Methods Consider: If all ILOs are being addressed? If there is a balanced coverage of the ILOs? Course ILO 5 Course ILO 4 Course ILO 3 Course ILO 2 Course ILO 1 Method 3 Method 2 Method 1 Assessment Methods Course ILOs

3. Judging and Grading of ILOs Judgement and grading should be based on evidence presented in the students’ work of how well the ILOs have been achieved. Judgement and grading should be based on evidence presented in the students’ work of how well the ILOs have been achieved. Grades should be awarded to reflect how well the ILOs have been achieved in terms of the level of understanding / performance in a particular content area. Grades should be awarded to reflect how well the ILOs have been achieved in terms of the level of understanding / performance in a particular content area. Grading criteria (rubrics) should be set to clearly define the standards and quality of the students’ performance expected of each grade for the ILOs. Grading criteria (rubrics) should be set to clearly define the standards and quality of the students’ performance expected of each grade for the ILOs.

4,5. Communicating, Feedback and Making Use of the Results of Assessment.Feedback to improve teaching and learning..Feedback to improve teaching and learning.. Reflect on assessment results to identify areas for improvement of assessment and teaching.. Reflect on assessment results to identify areas for improvement of assessment and teaching. Consider procedures for:. providing feedback to students on their achievement of the ILOs.. providing feedback to students on their achievement of the ILOs.. staff to discuss results of assessment of the ILOs.. staff to discuss results of assessment of the ILOs.. review and revision of assessment practices.. review and revision of assessment practices.

Closing the Loop Develop, modify, or review a curriculum, course/program. Develop Student Learning Outcomes Design & Measure Student Learning as a result of the Curriculum, Course, or Program Collect, discuss, and analyze data. Determine refinements based on outcomes data. Closing the Assessment Loop Adapted from: Sue Granger-Dickson & Terrence Willett (2005)

…………………………………………………..……………………… …

University Assessment Strategy

Minimising plagiarism

Would a well designed assessment system solve the problem?

THANK YOU 42