Assessment and Problem Based Learning PBL 2004 Glen O’Grady Director, Center for Educational Development

Overview of the Workshop More about you Assessment Assessment in PBL Grading

How do you Assess in your PBL Programmes? Draw your response – How it is done? – Why? – The effect (intended or unintended)

Why do we assess? To measure learning To foster learning

What are we assessing? Learning….but what is learning?

Conceptions of Learning A. Increasing one’s knowledge B. Learning as memorizing and reproducing C. Learning as applying D. Learning as understanding E. Learning as an interpretive process aimed at understanding reality F. Learning as changing as a person Willis 1993

Subject X Subject Z Syllabus Instruction- based Teacher Students Instruct, discipline, assess Dissemination of Knowledge Assessment is testing how well students have retained what the teacher has disseminated.

Prior Knowledge Problem- based Student Team members Facilitator The Problem (Curriculum) Construction of Knowledge Assessment is about coming to know how students construct their understanding of knowledge by giving tasks that also helps to foster this construction of knowledge

How can assessment be organized in PBL so that it both fosters & measures learning for understanding? Assessment needs to be… considered as an integral element in facilitation based on multiple methods. focused on both product and process. holistic

Case Study on Assessment in PBL

PBL at The Republic Polytechnic (RP-PBL) Each semester, 5 modules (subjects) each day students attend and work exclusively on one module Students follow a regular schedule each day

RP-PBL: 1 st meeting Class of 25, 5 teams of 5 students Presented a problem Students under the guidance of the facilitator work on defining the problem and identify issues they will do research on. Approximately 1 hour

RP-PBL: 1 st Breakout Student work individually and in their teams to: – Find and review resources – Begin to develop tentative solutions for the problem – Refine their definition of the problem

RP-PBL: 2 nd Meeting Meet with the facilitator who checks on their progress Focus on any difficulties students may be having Helps students to develop learning strategies

RP-PBL: 2 nd Breakout Student continue to work in their teams Review resources Develop a solution/ explanation based upon their shared understanding Produce a presentation 2-3 hours

RP-PBL: 3 rd Meeting Meet with the facilitator Students present their solutions/explanations Students observe how others have solved the problem Facilitators probes and critique these solutions giving additional information where necessary Students further check their understanding by doing a quiz focussed on the key issues

RP-PBL: Assessment Performance & Presentations Self & Peer Evaluation Reflection journal Students get feedback everyday – Verbal feedback in class (interactions) – Daily grade derived holistically – Written feedback Students every month sit an understanding test Module grade is determined by a combination of daily grades and understanding tests performance.

Exercise Method of Assessment How does it foster learning? How can it assist facilitators in their summative judgment (measure learning) Key challenges

I. Team Oral/Written Presentations How can it foster learning? – Develops communication skills (Hay 1994) – Verbal articulation (Rowntree 1987) How can it measure learning? – It manifests the application of knowledge in the context of a problem Key challenges – Criteria – Collusion – Tip of the iceberg (Lakomski 2002)Lakomski 2002

II. Peer and self assessment How can it foster learning? – Develops the capacity and ability to judge (Warren Piper et al. 1996, Boud 2001) – Engenders normative values in the way students relate to one another (Shoop 2000) How can it measure learning? – Manifests the way students think about themselves and others Key challenges – Reliability of these judgments

III. Learning Journal How can it foster learning? – Writing as a tool for organizing thoughts – Reflection the integration of theory and experience (Schon 1983, Butler 1994) – Focuses on the process of attributing meaning How can it measure learning? – It allows the facilitators to have an insight into the experience of the learner (process of reasoning, their interaction with others, their emotive connection) Key challenges – Students not knowing how to reflect

IV. Tests and Quizzes How can it foster learning? – Emphasizes specific skill sets and the need to be familiar with particular information. How can it measure learning? – Demonstrates understanding that can be compared – Gives insight into reasoning processes (Gibbs 1992b) Key challenges – Artificial and ritualized could encourage memorization & recall if questions are not carefully craftedcarefully crafted – Difficult to infer understanding (and its gradations) – How to give feedback

V. Observations of students’ discussions How can it foster learning? – Allows the facilitator to interact with students in meaningful ways (Barrows 1980,1988, Huba 2000) How can it measure learning? – Observe first hand elements of the construction of meaning Key challenges – Facilitators perspective (Pratt 1998)

Other methods? How can it foster learning? How can it measure learning? Key challenges

Marking & Giving Feedback Managing the task of marking – Criteria – Mark broadly, give specific feedback Making a difference with feedback – Personal – Regular – Cumulative – Holistic

How do we derive a grade?

Deriving the daily grade in a manner the fosters learning for understanding. – Collect as much information that is possible and practical. – Consider information collected in assessment as evidence that should be weighted in relation to the context. – Have explicit criteria but be comfortable with the “subjectivism” in applying that criteria. – Make it explicit (in feedback) the rationale for the judgment link it to the evidence. – Be prepared to explain this judgement to peers (and others)

Holistic Judgements Marks for student work should not be seen simply as individual bricks in a wall that represents a student’s final grade. Rather students’ work in its various forms are pieces of information or (evidence) that should be used collectively to form a picture of the quality of a students’ learning.

Holistic Judgements The final grade of a student should represent our best professional judgment about the learning of a student both in terms of process and product. The learning and the judgement should enduring.