Writing Learning Outcomes Mohamed Nadzri Mohd Yusoff (Hj.)

Presentation Outcomes At the end of the presentation, the participants will be able to: Explain the components in an outcome statement Write effective learning outcomes for different taxonomy level of the cognitive, psychomotor and affective domains based on 3 components

Programme Educational Objectives (PEO) Programme Learning Outcomes (PLO) Programme Learning Outcomes (PLO) Course Learning Outcomes (CLO) Weekly/Topic Outcomes Upon graduation - Nine (9) MOE Learning Outcomes Upon course completion Use Bloom’s Taxonomy of Learning Domains (C/A/P) Few years after Graduation – 4 to 5 years..will be able to demonstrate critical thinking skills to solve.. will be able to explain the physical principles of …… Competent engineers who are leaders in ….... will be able to explain Archimedes principle of …… Employer Survey, Alumni Survey My3S, Exit Survey, Prog.Survey Sum/Form/Cont. Assessments Upon topic completion Use Bloom’s Taxonomy of Learning Domains (C/A/P) Sum/Form/Cont. Assessments Different Levels of Learning Outcomes Shahrin Mohamed, 2007

Curriculum Development Process Purpose of Qualification Purpose of Qualification Programme Educational Objectives (PEO) What the graduates will become in 4-5 years time Principle Learning Outcomes Programme Learning Outcomes (PLO) What skills the students will have immediately upon graduation INPUTS FROM STAKEHOLDERS IDENTIFY ENABLING LEARNING OUTCOMES Kemahiran Insaniah + Knowledge + Psychomotor Skills What level of achievements the students should achieve for each LO Ref: Garispanduan Penulisan Program Akademik JPT, KPM IDENTIFY TEACHING AND LEARNING STRATEGIES SPECIFY ASSESSMENT METHODS AND TOOLS Ref: Modul Pembangunan Kemahiran Insaniah utk IPT ASSESSMENT CRITERIA AND CREDIT GUIDELINE ADDITIONAL INPUTS FROM STAKEHOLDERS MQF(1) MQF(2) MQF(3) OBE 1

Curriculum Development Process Identify Courses In Programme Programme Standards / Academics & Industry Captains Develop CLOs for courses Course Learning Outcomes (CLO) What students should know, understand and can do upon the completion of a period of study. IDENTIFY BLOOM’S LEVEL AND PLOs ADDRESSED Kemahiran Insaniah + Knowledge + Psychomotor Skills What level of achievements the students should achieve for each CLO Ref: Garispanduan Penulisan Program Akademik JPT, KPM IDENTIFY TEACHING AND LEARNING STRATEGIES SPECIFY ASSESSMENT METHODS AND TOOLS Ref: Modul Pembangunan Kemahiran Insaniah utk IPT CREDIT GUIDELINE ASSESSMENT CRITERIA CONSTRUCTIVE ALIGNMENT Ref: Garispanduan Penulisan Program Akademik JPT, KPM OBE 2

WRITING PROGRAMME EDUCATIONAL OBJECTIVES (PEO)

PEOs 4-5 statements on what the graduates are in 4-5 years time Must include all 9 MOE learning outcomes in the statements (for degree programmes and above) Usually in-line with the industry/profession’s needs Can include non-measurable verbs such as understand, know, acquire, etc.

1.Knowledge 2.Practical Skills 3.Thinking and scientific skills 4.Communication skills 5.Social skills, teamwork and responsibility 6.Values, ethics, moral and professionalism 7.Information management and lifelong learning skills 8.Managerial and entrepreneurial skills 9.Leadership skills

WRITING PROGRAMME LEARNING OUTCOMES(PLO)

PLOs Statements on what skills the students will have upon graduation (in line with programme standards or professional bodies) Must include all 9 MOE learning outcomes in the statements (for degree programmes and above) i.e at least 9 PLOs Usually a one-to-one mapping to the 9 MOE learning outcome (i.e PLO1 map to MOE LO1, PLO2 map to MOE LO2) Can include non-measurable verbs such as understand, know, acquire, etc.

The QUALIFICATION PLO’sWhat skills they have as (THE JOB(S)) Bachelor of Civil Engineering PLO1Acquire and apply knowledge of sciences and engineering fundamentals to civil engineering field. (LO1) PLO2Demonstrate comprehensive technical expertise in civil engineering. (LO2) PLO3Identify, formulate and provide creative, innovative and effective solution to civil engineering problems. (LO3) PLO4Communicate effectively both in written and spoken form with engineers, other professionals and community.(LO4) PLO5Function individually or in teams, with a capability to engage effectively with other people or team members.(LO5) PLO6Understand and commit professionally, ethically and with humane responsibility, in line with the engineer’s code of conduct.(LO6) PLO7Recognise the need for and to engage in, life-long learning and professional development.(LO7) PLO8Self motivate and enhance entrepreneurship skills for career development.(LO8) PLO9Realise and demonstrate effective leadership responsibility.(LO9)

1.Knowledge 2.Practical Skills 3.Thinking and scientific skills 4.Communication skills 5.Social skills, teamwork and responsibility 6.Values, ethics, moral and professionalism 7.Information management and lifelong learning skills 8.Managerial and entrepreneurial skills 9.Leadership skills

WRITING COURSE LEARNING OUTCOMES (CLO)

Characteristics of a Good Course Outcome  should be mapped to the learning domain in Blooms or other Taxonomy.  Must state the major skills, knowledge, attitude or ability that students will acquire.  expressed in terms of measurable and/or observable behaviors (hint: ask yourself how you would test the outcome).  more general than behavioural objectives.

Characteristics of a Good Course Outcome Outcomes The student will be able to: orally present a new patient's case. Behavioural Objectives The student will be able to: orally present a new patient's case in a logical manner, chronologically developing the present illness, summarising the pertinent positive and negative findings as well as the differential diagnosis and plans for further testing and treatment. Instructional objective focuses on what will be taught and how it will be taught Behavioural objective focuses on any visible activity displayed by a learner or what the learner can demonstrate what has been learned.

Characteristics of a Good Course Outcome (Cont.)  Each course will usually have between 3 and 5 major outcomes. (Garispanduan Penulisan Akademik, JPT, KPM)  Should be written in clear language and in the future tense  These outcomes should be the same across courses with the same code.  Begin with an action verb (e.g., write, install, solve, and apply).

3 components of a course outcome 1) Action verb ( describes what the learner will be doing, or the behaviour ) By the end of this course/semester, students should be able to: describe the principles used in designing X. evaluate the strengths and weakness of … Well-written verbs must be (SMART) - Specific - Measurable - Achievable - Realistic - Time frame - Observable Avoid these words - understand - appreciate - know - learn - aware - familiar

2) Condition ( context under which the behaviour is to occur ) describe the principles used in designing X.(V) orally describe the principles used in designing X. (V&C) design a beam. (V) design a beam using Microsoft Excel design template. (V&C) 3 components of a course outcome

3) Standard ( criteria of acceptable level of performance ) describe the principles used in designing X.(V) orally describe the principles used in designing X. (V&C) orally describe the five principles used in designing X. (V&C&S) design a concrete beam. (V) design a concrete beam using Microsoft Excel worksheet. (V&C) design a concrete beam using Microsoft Excel worksheet based on MS EN : 2010 (NATIONAL ANNEX) (V&C&S) 3 components of a course outcome

Course Outcomes: An example Identify the a) verb b) condition c) standard. From the first principles, calculate the beam deflection at the centre to within one decimal point. Identify the a) verb b) condition c) standard. write an effective course outcomes that include lower and higher order cognitive skills for a one-semester course.

Common weaknesses in writing CO Non-observable/Non-measurable CO Vague CO or CO that are too broad or general At the end of the course, the students should be able to: 1.understand the theory of X. 2.know how to write an effective learning outcomes 3.appreciate the importance of keeping the environment clean. By the end of the course, students should be able to: 1. use the computer. 2. make presentations. 3. comment on designs. 4. design research

Note for writing CLO It is NOT compulsory for every CLO to have all 3 components (i.e. V+C+S) But must have at least V+C or V+S Can have more than 1 action verb, BUT must measure them. If a statement has 2 “verbs” in the same domain (e.g. state and explain the basic principles.....), use the highest level of taxonomy only (i.e. explain the basic principles....) A statement can have 2 “verbs” in different domains (e.g. explain and perform the standard procedure ), BUT must measure both (i.e. there must be assessments for both outcomes) CLO can have sub outcomes (e.g. Topic Outcomes, Lesson Outcomes) which might be more detail and can have ALL 3 components to ensure the attainment of the CLO.

Improve on the following learning outcomes by adding a condition and standard Poor Students should be able to design research. Better Students should be able to independently design and carry out experimental and correlational research. Best Students should be able to independently design and carry out experimental and correlational research that yields valid results. Source: Bergen, R A Program Guideline for Outcomes Assessment at Geneva College

By the end of the course, students should be able to: 1)independently seek and present information on the collapse of the WTC (Lifelong Learning & Communication Skills) 2)participate actively in class discussion on issues related to... (Communication Skills) 3)work collaboratively in groups to complete an assigned project on… (Teamworking) 4)demonstrate positive teamworking attributes by contributing actively in group projects. (Teamworking) Writing Course Outcomes for Psychomotor and Affective Domains

Cont… 5)state and critically evaluate the main principles of.... (Problem Solving) 6)analyse data gathered from a target situation analysis and design instructional materials for a specific group of learners. (Problem Solving) 7)orally present information and answer questions with confidence on an assigned project. (Comm. Skills and Self- Esteem) 8)apply principles of management in organising an assigned project within stipulated schedules and with available resources. (Teamworking & Adaptability) Writing Course Outcomes for Psychomotor and Affective Domains

EXERCISE 1 In Handout “Exercise 1”, complete the table.

Outcome exampleAction verb ConditionStandardLevel By the end of the chemistry course students will be able to list all of the elements in the Periodic Table. By the end of the special education course students will be able to diagnose learning disabilities in elementary school level settings By the end of the French course, students will be able to translate a paragraph of text from English to French. By the end of the BIS course, students will be able to apply basic Web development skills to an actual Web page on the internet By the end of the music education course, students will be able to judge student performances based on musical interpretation. By the end of the art course, students will be able to create at least 12 original works in oil painting Listall of the elements in the Periodic Table.Cognitive 1 (C1) Knowledge Diagnoselearning disabilities in elementary school level settings C4 Analysis Translatea paragraph of text from English to French. C2 Comprehension Applybasic Web development skills actual Web page on the internet C3 Application Judgestudent performances based on musical interpretation. C6 Evaluation Createat least 12 original works in oil paintingC5 Synthesis P7 Origination

Outcome exampleAction verb ConditionStandardLevel By the end of the music theatre course, students will be able to relate types of music to particular dance steps. Relatetypes of music particular dance stepsPsychomotor 1 (P1) Perception C5,A4,P1 By the end of the industrial education course, students will be able to demonstrate proper use of woodworking tools to vocational school students. Demon- strate proper use of woodworking tools vocational school students. P5 Complex Overt Response C3,A3,P5 By the end of the physical education course, students will be able to demonstrate/show the proper stance for batting a ball. Demon- strate/ Show proper stancefor batting a ball.P2 Set C3 By the end of the dance course, students will be able to create a dance step. Createdance stepP7 Origination C5,P7 By the end of the physical education course, students will be able to display/perform a golf swing as demonstrated by the instructor. Display/ Perform golf swingas demonstrated by the instructor. P3 Guided Response P2,P3,P4,P5, A5 By the end of the industrial education course, students will be able to adapt their lessons on woodworking skills for disabled people. Adaptlessons on woodworking skills for disabled people.P6 Adaptation By the end of the biology course, students will be able to assemble laboratory equipment, appropriate for experiments. Assemblelaboratory equipment appropriate for experiments. P4 Mechanism C5,P3,P4,P5

Quiz ! 1.Which outcome statement is better? A.Students will be able to understand the connections between morality and law. B.Students will be able to explain the connections between morality and law.

Quiz ! 2.Which outcome statement is better? A. Students will be able to demonstrate information handling. B.Students will be able to demonstrate effective information handling and research skills.

Quiz ! 3.Which outcome statement is better? A.Students will be able to evaluate the various criminological theories in relation to drug issues. B.Students will be able to appreciate the various criminological theories in relation to drug issues.

Quiz ! 4.Which outcome statement is better? A.Students will be able to demonstrate proficiency in Japanese. B.Students will be able to demonstrate proficiency in conversational Japanese by the end of the semester.

Quiz ! 5.Which outcome statement is better? A.Students will be able to get an introduction to statistics and its practical application. B.Students will be able to explain the basic foundations of statistics and its practical application.

Quiz ! 6.Which outcome statement is better? A.Students will be able to be a part of a team. B.Students will be able to work effectively as part of a team.

Quiz ! 7.Which outcome statement is better? A.Students will be able to provide an overview of the issues, principles, and practices associated with higher education. B.Students will be able to discuss the issues, principles, and practices associated with higher education.

Quiz ! 8.Which outcome statement is better? A.Students will be able to know the behavioral, cognitive, and social interaction models of learning. B.Students will be able to compare and contrast the behavioral, cognitive, and social interaction models of learning.

Quiz ! 9.Which outcome statement is better? A.Students will be able to design and deliver a lesson which addresses a variety of student learning style. B.Students will be familiar with a lesson which addresses a variety of student learning style.

Quiz ! 10.Which outcome statement is better? A.Students will be able to analyse various learning style of mature students. B.Students will be able to consider various learning style of mature students.

What are the courses/ subjects the students should learn in the proposed programme? Must comply with Programme Standards – What subjects (core/elective/major/minor etc) – What percentages (core vs electives, etc) – How long is the duration of the programme

What level of learning should the subjects offer? Bloom’s Taxonomy of Learning – Cognitive, Affective, Psychomotor Diploma (C3,P4,A3) Undergraduate (C4, P4, A3) Masters (C5, P5, A4) PhD (C6, P6, A5) General Rule of Thumb –at least 5% of total no.of courses for the programme must achieve proposed Bloom’s level E.g. For Undergraduate programme with total no.of courses = 40 courses/subject Hence, at least 2 courses must achieve C4, at least 2 courses must achieve P4, and at least 2 courses must achieve A3.

For each course/ subject what are the outcomes? CO – Course Outcomes Must be aligned to PO Must outline level of learning taxonomy Must be written clearly – Action verb > condition/performance > standard/ criteria Must have at least V+C or V+S Can have more than 1 action verb, BUT must measure them

Note for writing CLO It is NOT compulsory for every CLO to have all 3 components (i.e. V+C+S) But must have at least V+C or V+S Can have more than 1 action verb, BUT must measure them. If a statement has 2 “verbs” in the same domain (e.g. state and explain the basic principles.....), use the highest level of taxonomy only (i.e. explain the basic principles....) A statement can have 2 “verbs” in different domains (e.g. explain and perform the standard procedure ), BUT must measure both (i.e. there must be assessments for both outcomes) CLO can have sub outcomes (e.g. Topic Outcomes, Lesson Outcomes) which might be more detail and can have ALL 3 components to ensure the attainment of the CLO.

Examples of CLOs for different PLOs Course Learning Outcomes (CLO) At the end of the course the student should be able to: Learning Outcomes Domain (MOE LO)  explain the principles of  analyse a given text based on the concept of knowledge (LO1)  display the procedure to  demonstrate how to perform technical skill (LO2)  propose a solution to management problems using principles of  describe a solution to thermodynamic problems using critical thinking problem solving (LO3)  explain and present the principles of LO1 and communication (LO4)  organise in teams to  explain in groups teamwork (LO5)

Examples of CLOs for different PLOs Course Learning Outcomes (CLO) At the end of the course the student should be able to: Learning Outcomes Domain (MOE LO)  propose a solution to ethical problems related to  describe a solution to teenage problems related to morality ethics (LO6)  explain the current issues related to  evaluate given case study with respect to evidence- based practice..... lifelong learning (LO7)  identify business opportunity related to the  plan a business venture with regards to entrepreneurship (LO8)  lead a project to solve a  lead a group to plan a CSR project leadership (LO9)