1. King Saud University, College of Science Workshop: Programme accreditation and quality assurance Riyadh, June 13-14, 2009 II.4 Exercise: Assessment of learning outcomes www.asiin.de

2. Assessment of LOs central for quality assurance & improvement Educational Objectives learning outcomes / competence profile knowledge, skills, competencies job / career profiles occupational area(s) work environment(s) Educational Objectives learning outcomes / competence profile knowledge, skills, competencies job / career profiles occupational area(s) work environment(s) Input of HEI support processes e.g. student services, staff resources, infrastructure, programme structure, curriculum, didactic concept, quality assurance Input of HEI support processes e.g. student services, staff resources, infrastructure, programme structure, curriculum, didactic concept, quality assurance Outcome correspondence of educational objectives and learning outcomes results of outcomes assessment and internal/external evaluation Outcome correspondence of educational objectives and learning outcomes results of outcomes assessment and internal/external evaluation Assessment of the process: coherence of goals, input + outcomes 2

3. Assessment on the module / course level… 3

4. …and assessment on the programme level Direct measures: Exit and other interviews Standardised exams Locally developed exams Portfolios Simulations Performance appraisal External examiner Oral exams Behavioural observation Direct measures: Exit and other interviews Standardised exams Locally developed exams Portfolios Simulations Performance appraisal External examiner Oral exams Behavioural observation Indirect measures: Focus groups Archival records Written surveys and questionnaires Exit and other interviews Indirect measures: Focus groups Archival records Written surveys and questionnaires Exit and other interviews Assessment methods 4

5. Direct and indirect measures o Direct measures provide for the direct examination or observation of student knowledge or skills against measurable learning outcomes. o Indirect measures are those that ascertain the opinion or subjective report of the extent or value of learning experiences. o Direct measures provide for the direct examination or observation of student knowledge or skills against measurable learning outcomes. o Indirect measures are those that ascertain the opinion or subjective report of the extent or value of learning experiences. 5

6. Validity and utility o Relevance: the assessment method assesses the learning outcome as directly as possible. o Accuracy: the method assesses the learning outcome as precisely as possible. o Utility: the method provides formative and summative results with clear implications for programme improvement. => Find a combination of module and programme assessment methods that generates valid and useful results (for student and programme assessment) with reasonable use of resources. o Relevance: the assessment method assesses the learning outcome as directly as possible. o Accuracy: the method assesses the learning outcome as precisely as possible. o Utility: the method provides formative and summative results with clear implications for programme improvement. => Find a combination of module and programme assessment methods that generates valid and useful results (for student and programme assessment) with reasonable use of resources. 6

7. 7 Exercise: Assessing learning outcomes a)Individual: Read the selection of assessment methods and the respective descriptions. (5 minutes) b)Group: Discuss for each single method what kind of outcomes you would try to assess with. (10 minutes) c)Group: List your results. (2 minutes) d)Group: Chose one of the listed programme learning outcomes. (3 minutes). e)Group: Decide if one of the assessment methods of the selection on your table is appropriate for assessing the selected outcome. (10 minutes) f)Group: Find an alternative assessment method besides the selection on your table that would fit as well / better for assessing the selected learning outcome. (5 minutes) Please be prepared to briefly present your results to the plenary. 35 minutes group workTime available:

8. Exercise: Assessing learning outcomes Written surveys and questionnaires Exit and other interviews Standardised exams Locally developed exams Archival records Written surveys and questionnaires Exit and other interviews Standardised exams Locally developed exams Archival records Focus groups Portfolios Simulations Performance appraisal External examiner Oral exams Behavioural observation Focus groups Portfolios Simulations Performance appraisal External examiner Oral exams Behavioural observation Please find the examples of assessment methods to work with on your table. They are taken from the list below. 8

9. 9 Exercise: Assessing learning outcomes a)Individual: Read the selection of assessment methods and the respective descriptions. (5 minutes) b)Group: Discuss for each single method what kind of outcomes you would try to assess with. (10 minutes) c)Group: List your results. (2 minutes) d)Group: Chose one of the listed programme learning outcomes. (3 minutes). e)Group: Decide if one of the assessment methods of the selection on your table is appropriate for assessing the selected outcome. (10 minutes) f)Group: Find an alternative assessment method besides the selection on your table that would fit as well / better for assessing the selected learning outcome. (5 minutes) Please be prepared to briefly present your results to the plenary. 35 minutes group workTime available:

10. 10 Exercise: Assessing learning outcomes Graduates are able... 1. to communicate effectively 2. to work effectively as member of a team 3. to solve a field-specific (e.g. technical, methodological, analytical,...) problem independently Possible programme learning outcomes:

11. 11 Exercise: Assessing learning outcomes You have got an impression of the variety of assessment methods that can be used for bringing the achievement of learning goals to evidence You have gained an idea of the difficulties and challenge for your creativity when looking for the “right” method to assess the achievements of learners You have experienced a typical situation of a programme coordinator who practices outcome-oriented curriculum development. Objectives: Did you find the exercise challenging? In how far?