Assessment of Learning Prepared By: Rhys Andrews Main Credit: Fenwick, T and Parsons, J. The Art of Evaluation

Outline Part 1 – Introduction to Assessment  A Learning-centered approach  Linking learning and assessment  Assessment planning  Evaluation methods Part 2 – Reviewing your Assessment  Reflect on your practices  Linking outcomes with assessments  Identify changes and / or new assessment methods to use

Is Anybody Learning Out There? Ask Yourself Who is responsible? What are they responsible for? Who are they responsible to?

The Learning-Centered Approach Teacher versus Learning – Centered Approaches Facilitation - Content and delivery Learning strategies and behaviors Assessment

Knowledge Comprehension Application Analysis Synthesis Evaluate Bloom’s Taxonomy Remember Create

Experiential Learning Cycle Experiencing Applying Reflecting Concluding

Defining Experiential Cycle Experiencing – “doing”, learning activities. Reflecting – “thinking”, examining the experience. Concluding – “conclusions”, what went well and what didn’t. Transfer learning to the real-world. Applying – “now what”, reflections and conclusions turned into action plans.

Assumptions about Learning A natural and dialectic process Is cyclical Learners have preferred strategies Learning is non-normative Learning is a function of the central nervous system Learning takes place within a context Strong emotions affect learning Learning involves learning how to learn What we value and believe to be true about learning influences our learning and how we facilitate learning

Learning Has An interactive dimension – we receive meanings from others, we develop meaning from our own experience, we integrate these two sources into a whole A constructive dimension – we construct meaning, we assign reality A transformative dimension – meaning has the ability to lead to change

Learning Principles Learners should: Understand the outcomes and assessments established by others Be able to accept these Be willing to commit themselves Learning is improved if learners can: Assess their own learning needs Select some of their own learning goals and directions for change.

What are Learning Outcomes? They are a statement of what the student should know and be able to do when they complete a program or course Knowledge Skills Attitudes Learning outcomes are related to the needs analysis of the student (for employment) rather than the needs of the instructor

Defining a Learning Outcome 3 main questions to answer: What is required of a student in a employment situation? What teaching methods/strategies would be effective? How will the learner be evaluated? Note. Evaluation methods similar to workforce requirements

Linking Learning and Assessment

Nine Purposes of Assessment 1.Compare learner performance to learning outcome. 2.Help learners make decisions about their next action. 3.Monitor students’ ongoing progress 4.Assess teaching methods 5.Revise the program 6.Provide information to stakeholders. 7.Assess a learners background knowledge. 8.Determine learner satisfaction. 9.Develop self-assessment in learners.

Training Model for Evaluation 1.Determine a baseline of performance. 2.Establish learning outcomes. 3.Conduct the learning, modify as you go. 4. Give an end test. 5. Conduct on the job measurement. 6. Measure and evaluate.

Authentic Assessment People are honest, learning is relevant, and evaluative feedback is helpful 1.On-going 2.Valid and reliable 3. Comprehensive 4.Communicated

Assessment Validity and Reliability Validity Measures what it is supposed to measure Strong link between learning outcome and assessment process Reliability Assessment is consistent and can be used over and over

Traps in Assessment 1.Measure what is easiest to measure 2.Underestimate the learning in the assessment 3.Power of teacher over student 4.Reduce learning to what is evaluated

Examine your Beliefs 1.What are the key things learners should know and do at the end of the program / course? 2.Should learners be helped to create their own knowledge that you deliver? 3.Which is more important: collaborative or individualized learning? 4.Should the instructor control the content and process, or should learners have more control? 5.Does learning happen in systematic steps or in holistic, unpredictable ways? 6.Does learning happen in a “Ah-ha” way or the through gradual change? 7.Should learners be able to demonstrate learning at the end of the course or at a later time.

Review your Practices 1.Do your methods for evaluating show if learners have mastered the essentials? 2.Do students reveal personally constructed knowledge, or knowledge you have constructed and they have memorized? 3.Does the evaluation match the learning activities? 4.Does the language in evaluation match the learning activities? 5.Do your assessments allow students to show what they know? 6.How much do learners participate in the assessment methods? 7.Do your evaluation methods embody your deepest beliefs about teaching and learning? 8.Could you justify each assignment or method of evaluation you use as valid?

Reflective Practice Remember as Learner 1.A product you were proud of – was the evaluation helpful? 2. Grades or comments that were negative - was it helpful? Remember as a Teacher 1.An evaluation you tried once and decided not to use again – what went wrong? 2. An assignment you gave that learners enjoyed – what made it successful? 3. An evaluation where performance was low – why?

Assessment Planning What do you want learners to know? Why? What? How? When? Summative or formative Qualitative, quantitative

Criteria for Assessment Criteria – compares learning to a absolute external standard Normative – compares one learners performance to others in the group Self – compares what the learner understands today with their knowledge in the past

Choosing a Strategy Methods of Evaluation Evaluate at Many Levels – understanding, apply, analyze, evaluate concepts, and create new ideas Multiple Methods – use more than one method Be Aware – of your ability and the needs of your learners Timing – formative (during the term) and summative (at the end of the term) Data – Best data to determine learning

Methods Test – multiple choice, fill-in blank, true/false Test – short and long answer Demonstrate skill Informal writing Student created product Informal observation Self-assessment Peer assessment Integrated projects Labs and assignments Portfolio

Evaluating Performance What is the minimum level of performance? Pre-test and post-test

Measuring Knowledge Evaluate at levels of Bloom’s Taxonomy where necessary

Assessing Technical Skills Student should be able to. Know how to do it – the parts and process and how to operate What to do and when Know consequences of actions How can we help? Demonstrate and describe Practice and formative evaluation Learner performs and summative evaluation Self-evaluation

My Course Course Level 10 – 13 lab assignments linked to specific learning outcomes Assignments designed as “real life” activities Formative and summative exam – lecture and practical In-class exercises Individual and group-based Program Level Integrated project – with presentation Applied research project – with presentation

Relevance to a job: an individual learning activity Faculty: Define the problem Students: Explore Students: Plan the Solution Students modify And carry out Students and Faculty Evaluate Problem-Based Learning Students: Additional Information needs? Learning that simulates what will be encountered in the workforce …. Faculty provide Theory/Practice Faculty evaluate the plan Learning Processes….

Course 1 Course 2 Course 3 Course 4 Integrated Activity Example: SeptemberDecember Integrating Learning

School Requirements Faculty are required to: Develop learning outcomes Assess based on outcomes Use many means of assessment Use criteria where necessary Assess based on employer needs Provide feedback in a timely manner Be unbiased Be assessed by students on performance

Resources Fenwick, T., Parsons, J. (2000). The art of evaluation: A handbook for educators and trainers. Edmonton, AB: Thompson Educational Publishing. Hubball, H.. (2000). A learner-centered approach to curriculum re-design: Implications for learning communities, planning, evaluation and programming in higher education. Vancouver, BC: UBC Press. MacKeracher, D. (2004). Making sense of adult learning (2 nd. ed.). Toronto, ON: University of Toronto Press.