CONSTRUCTIVE ALIGNMENT & EDUCATIONAL ASSESSMENT ADIBAH BINTI ABDUL LATIF QRiM UTM

1 Staff 1 Question https://padlet.com/p_adibah/gy7mljmg7zd8

CONSTRUCTIVE ALIGNMENT

CONSTRUCTIVE ALIGNMENT Constructive = Constructivist theory that learners using their own activity to construct their knowledge as interpreted through their own existing schemata. Alignment = Principle in curriculum theory that assessment tasks should be aligned to what it is intended to be learned , as in criterion referenced assessment. What the student does is actually more important in determining what is learned Than what the teacher does

The good teaching is getting most students to Adlina The good teaching is getting most students to use the level of cognitive processes needed to achieve the intended learning outcomes that the more academic students use spontaneously Zaquan Adapted, Biggs [2011]

OUTCOMES Level Of Outcomes What students able to do PROGRAMME EDUCATIONAL OBJECTIVES PEO Level Of Outcomes 3 to 5 years after graduation PROGRAMME LEARNING OUTCOMES PLO Upon graduation COURSE LEARNING OUTCOME CLO After completing each course UNIT LEARNING OUTCOME ULO After completing each unit

STEPS ON CONSTRUCTIVE ALIGNMENT Graduate Attribute Program Learning Outcome Course Learning Outcome Topic / Unit Learning Outcome

UTM GRADUATE ATTRIBUTE COMMUNICATING SKILLS (Kemahiran Berkomunikasi) CS Thinking Skills (Kemahiran Berfikir) TH Scholarship (Kesarjanaan) SC Team working skills (Kemahiran kerja berpasukan) TW Adaptability (Kebolehsuaian) AD Global Citizen (Warga Global) GC Enterprising Skills (Kemahiran Berdaya Usaha) ES

PLO Kluster Kejuruteraan

PLO kluster sains sosial dan Sains Teknologi(UG)

PLO bagi Pasca Siswazah Bil PLO Kod 1 Advanced Knowledge AKW 2 Research Skills RS 3 Critical Thinking and Problem Solving CTPS 4 Ethics, values, professionalism EM 5 Communication CS 6 Lifelong Learning LL 7 Social Skills SS 8 Team working TS 9 Leadership LS 10 Information management IM 11 Managerial and Entrepreneurial ME core Optional

OPERATIONAL DEFINITION PLO (UG)

CS (COMMUNICATING SKILLS)

CS (COMMUNICATING SKILLS)

TH (THINKING SKILLS)

TH (THINKING SKILLS)

SC (SCHOLARSHIP)

SC (SCHOLARSHIP)

TW (TEAM WORKING)

TW (TEAM WORKING)

AD (ADAPTABILITY)

AD (ADAPTABILITY)

GC (GLOBAL CITIZEN)

GC (GLOBAL CITIZEN)

ES (ENTERPRISING SKILLS)

ES (ENTERPRISING SKILLS)

OPERATIONAL DEFINITION PLO (PG)

ADVANCED KNOWLEDGE

RESEARCH SKILLS

CRITICAL THINKING

ETHICS, VALUES, PROFESSIONALISME

COMMUNICATION

LIFE LONG LEARNING

SOCIAL SKILLS

TEAMWORKING

LEADERSHIP

INFORMATION MANAGEMENT

MANAGERIAL AND ENTREPRENEURIAL

MQF 2.0 MQF 2.0 Membuat 2 penambahan PLO baharu iaitu Numeracy skills and Digital Skills

Constructive Assessment  Constructive Teaching and Learning Method  Constructive Verb in CLO’s

Define the meaning of validity Verb C1 in Bloom Taxonomy

Define the meaning of validity by comparing the literature given Verb Condition C4 in Bloom Taxonomy

Compare 10 referred articles on validity in a group Verb standard Condition C4 in Bloom Taxonomy

Explain the meaning of validity using students own word. Verb C2 in Bloom Taxonomy

Explain the meaning of validity by evaluating the theories of CTT and IRT Verb Condition C5 in Bloom Taxonomy

Evaluate at least three advantages and limitations of CTT and IRT theory Verb standard C5 in Bloom Taxonomy

DEVELOP COURSE LEARNING OUTCOMES

DEVELOP COURSE LEARNING OUTCOMES

DEVELOP COURSE LEARNING OUTCOMES

DEVELOP COURSE LEARNING OUTCOMES

A B C D 3 COMPONENTS OF CLO DEVELOP COURSE LEARNING OUTCOMES At the end of the course, students should be able to evaluate the quality of two examination papers for Multiple Choice Question and Essay question based on Classical Test Theory analysis. A B C D CONDITION AUDIENCE BEHAVIOR / VERB DEGREE/ STANDARD

DEVELOP COURSE LEARNING OUTCOMES VERB C CONDITION S STANDARD

DEVELOP COURSE LEARNING OUTCOMES

DEVELOP COURSE LEARNING OUTCOMES

Boleh dimasukkan kemahiran generik yang dikaji

1.Buag iCGPA dan ubah kepada PLO Code sahaja 2. CLO 1 weight dan assessment Method tidak sama nilainya 3. Masukkan juga PLO yang Keberapa 4.To understand tidak boleh Diukur 5. Mengapa CLO 1 ada P? 6. Bagaimana TS diukur melalui Exam?

Perlu ada di kotak constructive alignment sebelum ini

O bukanlah observation, tetapi others Bagaimana nilainya telah cukup 120 sedangkan exam belum lagi dimasukkan? CLO1 CLO3 dan CLO 8? Jumlah pemberat markah dan SLT tidak selari

Brerlebihan bila campur yang di atas

Tidak semua memerlukan ACP Verb dan aras kelihatan tidak selari

Face to face melebihi 42 jam?

Ada 2 verb dalam satu CLO. Adakah mengukur domain yang berbeza? Ambil C/P/A yang tertinggi sahaja untuk satu CLO

Tunjukkan di mana PBL itu berlaku dalam Jadual Constructive alignment Aktiviti yang ada dalam Jadual CA perlu juga diletakkan dalam Jadual di bawah

1. Estimatekan masa yang diperuntukkan khusus untuk buat tugasan tersebut

Wajib ada

1. Final exam bukan continuous assessment 2. Estimatekan masa yang diperuntukkan khusus untuk buat tugasan tersebut

1. Bukan bahan sedia ada

Perlu ada rujukan yang terkini

1. Terlalu banyak verb dalam satu CLO 2. Pilih verb tertinggi sahaja 3. Tidak perlu ada semua C A P

Pecahkan kepada Assignment 1,2,3 dan anggarkan masa untuk melaksanakan

COURSE INFORMATION UTM 2018 Weekly schedule adalah flexible berdasarkan kepada perancangan aktiviti dan sesuai dengan SLT yang ditetapkan

COURSE INFORMATION UTM 2018 Masukkan kemahiran generik yang ditaksir Kemahiran generik lain yang diterapkan walau tidak ditaksir

Student Learning Time 1 kredit bersamaan dengan 40 jam kredit Diagihkan mengikut kesesuaian perlaksanaan proses pengajaran dan pembelajaran Penetapan jam kredit adalah selari dengan weight pentaksiran Contoh:- CLO 2 menggunakan pentaksiran individual presentation dan diberi markah sebanyak 10 markah Maka (10/100)*80 (jam kredit) = 8jam Maka agihkan 8 jam untuk F2F (L/T/P/O), NF2F secara guidance atau non guidance

COURSE INFORMATION UTM 2018 PLO 1 PLO 5 PLO 2 PLO 6

COURSE INFORMATION UTM 2018

KEMAHIRAN GENERIK Bukan bersifat ordinal, maka satu CLO boleh lebih daripada 1 operational definition yang ada dalam kemahiran generic tersebut Perlu diingatkan bahawa kemahiran generik bukan bersifat knowledge dan technical skills. Contoh: Jika thinking skills bukan penyelesaian masalah dalam masalah matematik di dalam ujian. Tetapi thinking skills semasa melaksanakan kajian / projek / masalah di luar silibus pembelajaran.

ISU PENULISAN CLO Mempunyai terlalu banyak CLO dalam satu kursus. Ada sehingga 14 CLO Pensyarah kurang kemahiran mensintesiskan silibus ke dalam bentuk outcome yang boleh diintegrasikan Pensyarah tidak memulakan verb yang diukur sekurang-kurangnya dengan application level dalam taksonomi Bloom Pensyarah yang menggunakan MCQ dalam final exam merasa sukar untuk mappingkan item dalam soalan exam dengan CLO.

ISU PENULISAN CLO Pensyarah tidak revised CLO, hanya menggunakan CLO sedia ada dan tidak menggunakan konsep OBE CLO tidak align dengan Aktiviti dan Pentaksiran CLO dibina tanpa sebarang data yang meyakinkan (contoh perbincangan degan panel luar/stakeholder dan tanpa data penyelidikan daripada pencapaian pelajar)

ISU PENULISAN CLO Pensyarah menggunakan dua verb bagi satu domain pembelajaran yang sama Pensyarah gagal membezakan setiap aras dalam domain pembelajaran samada kognitif, afektif dan psikomotor Pensyarah terlalu terikat dengan “keyword” dalam Taksonomi pendidikan

ASSURING QUALITY OF ASSESSMENT

“Assessment is at the heart of student experience” Brown and Knight (1994) “If you want to change student learning then change the method of assessment” Brown, Bull & Pendlebury (1997)

Give full measure and weight with justice

Give just measure and weight

PRINCIPLE OF ASSESSMENT Well-aligned with educational learning outcomes. Assessment should be valid and reliable Formative assessments needs to scaffold students in the summative assessment Student should receive feedback on their work in timely manner. Assessment should be inclusive and equitable for all students Assessment is not used to threaten and intimidate students. Assessment should help student mastery learning.

TYPES of ASSESSMENT Assessment Of Learning (AoL) Assessment For Learning (AfL) Assessment As Learning (AaL)

TESTING A tool to determine student’s ability to complete specific tasks or demonstrate mastery of a skill or knowledge of content The most critical basis in ensuring the validity of students interpretation score Examples: Q&A session in class, assignment, performance task, test, quiz and final exam

MEASUREMENT A systematic process of assigning numerals (quantitative) to the test administered. It can be in raw scores, percentile, standard score, etc. Examples: Assignment marks, total sore in a final exam, mean of PLO, KPI score, mean of elppt, ranking score.

SPP2032::Educational Measurement and Evaluation Ability Score 19 November 2019 SPP2032::Educational Measurement and Evaluation ~ZNDMD~

EVALUATION The process of describing, obtaining, and providing useful information for judging decision alternatives. This process allows one to make a judgment about the desirability or value of something. Examples: ABCD, Pass and Fail, HL,TM,MM, Description from the value (Baik, Cemerlang, Sederhana), Description of P1 to P5 in e-LPPT.

ASSESSMENT The process of gathering information to monitor and reflect the progress in learning and teaching and to make educational decisions if necessary. Examples: Dr A found out that her students are excellent in formative assessment, but they cannot perform well in their final exam. What to do? Dr B realises that there are two groups of ability/level of skills in his class. How can he do to make sure the learning environment can help students’ learning?

Assessment Evaluation Measurement Testing SCOPE AND FUNCTION

Which come first? Assessment or Evaluation?

Takes up where the formative leaves off TYPES OF TEST At the end of the course On going during the process of learning. Befor the class Takes up where the formative leaves off © Copyright PresentationGO.com – The free PowerPoint template library PLACEMENT FORMATIVE DIAGNOSTIC SUMMATIVE

Norm Referenced Test and Criterion Referenced Test

ASSESSMENT METHOD ASSIGNMENT SURVEY TOOLS RUBRIC RATING SCALE

ASSESSMENT ALTERNATIVE CONVENTIONAL PERFORMANCE BASED AUTHENTIC

Examples of authentic assessment Research Project Debate Writing Speech / summary Studio Work Portfolio Article Review Writing Journal / proposal Case Study

AUTHENTIC ASSESSMENT New Academia Learning Innovation Not only performance based, but happen in the real setting. Emphasizing more on process rather than product Soft skills development Holistic assessment Rubric

CAPSTONE PROJECT

TEACHING PRACTICES

SERVICE LEARNING

2U2I PROGRAMME

WORK BASED LEARNING

JOB CREATION

SCORING AUTHENTIC ASSESSMENT METHOD CHECKLIST RATING SCALE RUBRIC HOLISTIC ANALYTIC

OBJECTIVITY VALIDITY INTERPRETABILITY RELIABILITY ADMINISTRABILITY USABILITY

CONTENT VALIDITY

TABLE OF SPECIFICATION SPPP 2032

Table of Spesification

FORMULA Formula A Relative weight for the importance of content = ( The number of the TLO @ class period for one topic ÷ total TLO @ class period ) ×100% (3/10)*100 = 30

Relative weight of the subject TLO / Hours spent Content %30 3 Topic 1 %10 1 Topic 2 Topic 3 %20 2 Topic 4 Topic 5 Topic 6 100% 10 Total TLO / class periods for teaching the unit

FORMULA Formula B Relative weight for the item = (% of weight in each Bloom level x total item of the test) (0.3*20)= 6

Application and Analysis 50% Evaluation and Synthesize Objectives Totals 100% Topics Knowledge and Comprehension 30 % Application and Analysis 50% Evaluation and Synthesize 20% Topic 1 (30 %) Topic 2 (10 %) Topic 3 (10 %) Topic 4 (20 %) Topic 5 (10 %) Topic 6 (20 %) Weight for item 6 10 4 20

FORMULA Formula C Identify the number of questions in each topic for each level of objectives = (The total number of test x relative weight of the topics x relative weight of Bloom level) (20*0.3*0.3)= 1.8

30 % 50% 20% Totals 100% Number of questions 10 20 Objectives (Totals 100%) Topics Knowledge and Comprehension 30 % Application and Analysis 50% Evaluation and Synthesize 20% Totals 100% Topic 1 (30 %) 1.8 (2) 3 (3) 1.2 (1) 6 Topic 2 (10 %) 0.6 (1) 1 (1) 0.4 (0) 2 Topic 3 (10 %) Topic 4 (20 %) 2(2) 0.8 (1) 4 Topic 5 (10 %) 0.6 (0) 0.4 (1) Topic 6 (20 %) Number of questions 10 20

CONSTRUCT VALIDITY

CONSTRUCT VALIDITY Rubrik ini dapat menguji 51.6% kemahiran komunikasi pelajar

CONSTRUCT VALIDITY Rubrik ini dapat menguji 24.6% kemahiran komunikasi pelajar

Validity and Reliability

Validity and Reliability

Validity and Reliability

PROFIL SECARA INDIVIDU

Distribution of Marks

Distribution of Marks

Kajian Kes Tidak sejajar di antara CLO dan aras taksonomi

CADANGAN Kuiz, test, dan peperiksaan akhir dirangkum dalam satu CLO sahaja. CLO lain memfokuskan pentaksiran secara praktikal dan kemahiran generik

Kajian kes Bagaimanakah sebenarnya bilangan item ditetapkan? Berasarkan jam kuliah? Pemberat CLO? Ini kerana jumlah Item yang sama sahaja bagi jumlah jam kuliah yang berbeza

CADANGAN Aplikasikan formula yang dicadangkan Masukkan hanya topik yang paling signifikan terutama bagi soalan berbentuk subjektif

Kajian Kes Tidak salah sebenarnya bagi peringkat sijil / diploma untuk mempunyai CLO / aras item yang melebihi aras Aplikasi

CADANGAN Gunakan satu sahaja CLO bagi mengukur PLO1 (Pengetahuan) Aplikasikan konsep penjajaran konstruktif.

Kajian Kes Soalan MCQ bukanlah dibina untuk aras mudah sahaja. Malah bagi ujian berbentuk sumatif seharusnya memenuhi konsep normal distribution

CADANGAN Kemahiran dalam pembinaan item peperiksaan Memenuhi prinsip pembinaan item

Kajian kes Antara contoh CLO yang tidak sesuai diukur dengan ujian bertulis

CADANGAN Tidak semua CLO perlu diukur dengan peperiksaan Bina satu CLO khas untuk pengetahuan (PLO1)

Kajian Kes Item ini bukanlah item berbentuk struktur

CADANGAN Pensyarah perlu dapat membezakan apa yang dimaksudkan dengan item jawapan pendek, respon terhad, respon terbuka dan soalan berstruktur.

Kajian Kes Bagaimana penetapan dan pemilihan topik bagi ujian berbentuk esei? Adakah perlu melibatkan semua topik? Bagaimana dari segi aras kesukaran?

Kajian Kes Instead of letakkan bilangan item, letakkan nombor soalan, Contohnya soalan 1 (a), 1(b), 2(a),

Kajian Kes Tiada jajaran dengan CLO Tiada pecahan aras kesukaran item

Practical skills should not be assessed by examination

As conclusion…

Need to learn about Learning Taxonomies

Need to learn about Generic Skills

Need to learn about Teaching and Learning Methods

Need to learn about Assessment and Evaluation

From Grade Oriented to Outcomes Oriented From Content Based to Ability Based From Lecturer Centred to Student Centred

Special Thanks to Prof Shahrin bin Mohammad PM Dr Shafry Bin Abd Rahim Tn Hj Yahya bin Samian For all the materials shared in this power point

Look at the measure not the score… Look at the measure not the score….. Emphasize on the outcome not the output….