Characteristics of good assessments Module 3 Khalid Khan

‘if we wish to discover the truth about an educational system, we must first look to its assessment procedures’ (Rowntree, 1987, p.1)

Assessment needs be Fair Unbiased Reliable Valid

Features Effective Assessments are Congruent with Instruction Effective Assessments are Based on Authentic Tasks Effective Assessments Use a Wide Range of Tools and Methods Effective Assessments are Based on Criteria That Students Know and Understand Effective Assessments are a Collaborative Processes Involving Students Effective Assessments Focus on What Students Have Learned and Can Do Effective Assessments are Ongoing and Continuous http://www.edu.gov.mb.ca/k12/cur/ela/docs/sr2classassess2.html

Assessments Types Assessments Restricted Response Extended Response Essays Multiple choices True-False Problems Solving Matching Research Projects Completion

Restricted Response Objective Tests Highly reliable and have Less chance of subjectivity However, they also have Low level of Comprehension and Measure only factual knowledge

Restricted Response High reliability High Validity Objective tests are reliable but most of the time they do not assess what they claim, hence are Not Valid Dumb down the curriculum Teach only what can be tested

Characteristics: 1- Reliability Reliability refers to the consistency. It is the degree to which student results are the same when they take the same test on different occasions, when different assessors score the same task, and when different but equivalent tests are taken at the same time

Improving Reliability -- Collect evidence from different locations and activities and at different times -- Collect evidence by using a number of different assessment methods -- State clearly what needs to be assessed well in time

Consider Portfolios They are : However, they also are : Useful Increasingly common More reliable and valid However, they also are : Fuzzy Raise questions about the process

Inter-rater reliability Means the level of agreement or degree of agreement among assessors When objective tests are not used it give rise to the questions about the reliability of the examiner. Is he/she consistent? Would another assessor give the same rating/mark?

How to Improve ? Clear criteria before the assessment Define and illustrate what is to be expected at each level of grading

Characteristics of Assessment: Validity Measures what it claims to measure

Characteristics of Assessment: Validity: Face Validity Content Validity Construct Validity the extent to a test is viewed as covering the concept it claims to measure the extent to which it represents all facets or all dimensions the degree to which a test actually measures what it claims to be measuring refers to the transparency or relevance of a test as they appear to candidates do the assessment items appear to be appropriate? Covers entire range of possible items it should cover does the assessment content cover what you want to assess? Essential to the overall validity of the test are you measuring what you think you're measuring? it "looks like" it is going to measure what it is supposed to measure Brown, J. D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall Regents

How to Improve ? Ensure that assessment : --Focuses on appropriate tasks and skills --Covers a sufficient range of skills --Resembles tasks students have already encountered --Uses a variety of approaches to assess --have goals and objectives clearly defined --have clear expectations of students

Characteristics of Assessment: Fairness Refers to not putting any person at any disadvantage

How to Improve ? Help to clarify, in advance, what is expected and what is the format of the test There should be no discrimination on the bases of gender, socio-economic background , race of ethnicity Performance criterion is clear to all students who are being assessed Provide opportunities for review or appeal

Remember Trick questions have no place in good assessments Assessment do no just rely on written forms Concern only what the student know and can do.

Assessment & Dimensions of Learning The Dimensions of Learning is a model of education developed by Dr Robert Marzano in 1992 to generalise learning process in order to provide instructional tools for students and teachers .

Marzano proposed five types of thinking that are essential to successful learning. He named them as five dimensions of learning.

Dimension 1: Attitudes and Perceptions Key aspect for learning is student’s attitude and perception. In the context of Assessment it means: Students needs to develop positive attitudes and perceptions about tasks They need to perceive tasks as valuable and interesting They need to believe they have the ability and resources to complete tasks

Dimension 2: Acquire and Integrate Knowledge Meaning students are able to relate new knowledge with what they already know. In the context of Assessment it means: Student are able to gather information given in the tasks Students are able to create and display physical and graphic representation of information Students are able to report on their accuracy Students are able to discover and construct new information

Dimension 3: Extend and Refine Knowledge Meaning the ability to process and extend knowledge In the context of Assessment it means: Tasks include some aspects for Comparing – are these things alike/different? Classifying – divide into groups, explain different characteristics Induction – based on the given facts what do you conclude/find ? Deduction – what are your conclusions from the given scenario? Error Analysis – What are errors in the present reasoning, how is this misleading? Analysing – why would this be good/bad ? Abstraction – where or in which situations can we apply this ? Constructing support – What data support given/your claim ? Underline assumptions

Dimension 4: Using Knowledge Meaningfully Being able to use knowledge to achieve a goal/meaningful purpose In the context of Assessment it means: Being able to -- make decisions --solve problems --do experimental inquiry --investigate --Analyse parts of a system --Assessment contribute in achieving success

Dimension 5: Habits of Mind Meaning developing powerful mental habits to think critically and regulate behaviour In the context of Assessment it means: Being accurate Creative and clear Able to plan and evaluate Sensitive to feedback being aware of resources available

Ask the following Question1: What does the student know? Question2:What does he/she needs to know ? Question3: How does he/she learns best? Question4: What resources are at your disposal? Question5: What will constitute learning journey?, what is the context? Question6: Who will do what? Question7: How will I check? Question8: How will I inform about the progress ?

Ways to promote quality Consider aesthetics Go beyond the data Deep and coherent knowledge Tackling problems (individually and collaboratively) Be pragmatic Encourage crossing the content boundaries Complex reasoning process based on student’s own thinking

Reporting Ensure that feedback is to inform and improve learning and contributes toward success and achievement.

The end Reference 1. ROWNTREE, D. (1987) Assessing Students — how shall we know them? London: Kogan Page 2. Marzano, R and Pickering, D et al (1997) Dimensions of Learning: Trainer’s Manual (2nd Edition) ASCD: McRel (can be accessed at : http://www.ascd.org/ASCD/pdf/siteASCD/publications/books/Dimensions-of-Learning-Trainers-Manual-2nd-edition.pdf)