Assessment Matters for Diversity Ginny Saich DAICE

Overview The H.E. Context for Assessment The Stirling Context for Assessment What is assessment? - What are its characteristics? Why assess? - What is its purpose? What is the effect of assessment? How do we assess? What are common problems with assessment? What trends are apparent in assessment? How do we grade? What are the characteristics of good feedback? Good practice in assessment

H.E. Context of Assessment Recent legislation including:  the disability discrimination act (DDA Part IV)  the race relations amendment act places an onus on university’s to embrace inclusive practices (including for learning and teaching) Assessment is one of the Quality Assurance Agency’s (QAA) enhancement themes for 2003/4. Implications? Concerns about plagiarism with increasing prevalence of online resources

Stirling Context of Assessment The University of Stirling’s Learning, Teaching and Quality Enhancement Strategy: has the following action items relating to assessment for implementation during 2003/4:  Instigate a critical review of assessment practices with reference to the relevant section of the QAA Code of Practice and in light of the Quality Enhancement engagement of  Investigate and report on the implications for Learning and Teaching of Disability, Special Educational Needs, and Racial Equality legislation, with recommendations for action  Monitor progress on embedding the accessible curriculum  Report on use of diagnostic testing to identify the needs of students  Report on effectiveness of the Special Academic Arrangements Panel after its first year in operation  Report on use of JISC Plagiarism Detection Service pilot

Preventing and Detecting Plagiarism Prevention is more effective than detection! Good Practice Guide from Oxford Brookes contains recommendations on preventing plagiarism (see - web site of the JISC Plagiarism Advisory Service) JISC Plagiarism Detection Service - detects plagiarism of online resources (and collusion) - to be piloted at Stirling in Spring 2004

Why Assess? The Student to pass or fail to grade or mark to allow to proceed to licence to predict success to select for further training to detect strengths & weaknesses to motivate to give feedback for public relations Licence to proceed Ranking order Improving learning Licence to proceed Ranking order Improving learning to meet expectations to indicate performance to help learning

Purpose of Assessment

If we wish to discover the truth about an educational system, we must look into its assessment procedure. What student qualities and achievements are actively valued and rewarded by the system? Rowntree (1987) Assessing Students London: Kogan Page, 2nd edition, p.1 What is Valued?

Assessment influences student learning eg. Surface learning is induced by:- Heavy workloads, assessment by unseen exam alone, multiple choice questions that test only recall. Deep learning is induced by:- Reasonable workloads, some choice, a variety of assessment tasks, project work, multiple choice questions that test understanding (Marton and Säljö, 1976: surface and deep learning) The Effects of Assessment on Learning The kinds of assessments set are important!

If you want to change the mode of learning then change the assessment procedure. Brown, G., Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education London: Routledge

Assessment is ….. a “systematic basis for making inferences about the learning and development of students … the process of defining, selecting, designing, collecting, analysing, interpreting and using information to increase student’s learning and development” (Erwin, 1991: 15)

What is Assessment? Sample Draw inference Estimate worth Formative assessment Summative assessment Feedback ?

Student Experiences of Assessment “Continuous assessment meant that I knew how I was getting on, rather then waiting until the end of the session, then failing because I didn’t know what was required.” “I failed the first one but I don’t really know why. All I got were a lot of crosses and a few ticks and some exclamation marks.”

Functions of Assessment

In practice So-called formative assessment is often multiple summative assessment - and little feedback is given.

Characteristics of Assessment Efficiency - saves you time Validity - effectively tests what you intended ie. fitting your learning outcomes etc. Reliability - provides consistency Enablement- develops understanding and expertise The difficulty lies in achieving all these characteristics together.

What is effective assessment? It assesses what is taught and learnt It emphasises deep learning It measures skills and transfer It is efficient for lecturers It is a reasonable workload for students Do you have an assessment strategy? Does your department?

Reliability of Assessment Reliable assessment strategy  same results if repeated or if marked by a different marker Even apparently objective assessment may have a highly subjective component.

Objectivity 1 A candidate was required, for 10 marks, to multiply 4567 by 23. Below is the candidate’s answer compared with the correct answer. Candidate4567Correct: 4567 x 23 x What mark would you award?

Objectivity 2 Four markers comments: 1.This candidate cannot do simple long multiplication. Mark = 0 2.Layout and general procedure correct. Accuracy missing. Mark = 4 3.Errors in multiplication, but most correct. Addition correct. Mark = out of 14 arithmetical operations carried out correctly. Mark = 8

Validity of Assessment A valid assessment is trustworthy  tests what it is supposed to test. A test may be technically highly reliable but not valid eg. a test for critical thinking that asks questions requiring only memorisation

Inclusive Assessment Planning for inclusiveness in assessment means:  Ensuring that one group of students are not unduly advantaged or disadvantaged by the selected mode of assessment.  Ensuring that all students have the means to acquire the assumed knowledge and skill to successfully complete the course.  Being clear about the essential learning outcomes to be attained and willing to consider alternative means for student to demonstrate the attainment of them Stirling has a developing set of policies and guidelines for supporting students with disabilities. (Further information is available from departmental disability officers or the University’s disability officer, Catriona Mowatt in SISS.)

Levels of Objectives (Bloom’s Taxonomy) 1.Knowledge 2.Comprehension 3.Application 4.Analysis 5.Synthesis 6.Evaluation

Assessment Terms Methods Essays Problem sheets Unseen prose Lab reports Presentations Projects Group projects Posters Work-based learning Written exams Vivas MCQ's MRI'sInstruments Implicit criteria Global Explicit criteria Criteria reference grading Specific criteria Marking scheme Dimensions Rating schedules ChecklistsSources Lecturer Other tutors Postgraduate tutors Demonstrators Student self Student peers Employers Mentors External examiner

Criterion referenced Assessment

Norm referenced Assessment

Assessment methods: what are they good for? Essays- for analysis, synthesis and creative perspectives MCQs- to sample knowledge and understanding of principles Problems- to sample problem solving skills! Practicals - to sample practical skills! Lab sheets- to sample report writing, understanding of procedures, data analysis and interpretation data analysis and interpretation Projects - as for lab sheets plus deeper understanding, literature review (?), reflective learning (?), literature review (?), reflective learning (?), time management time management Learning - reflective and experiential learning Logs

Common Weaknesses Overload of staff and students Overload of staff and students Too many assignments set with the same deadline Too many assignments set with the same deadline Lack of alignment between assessment and teaching Lack of alignment between assessment and teaching methods and/or learning outcomes and course aims methods and/or learning outcomes and course aims Poor assessment design  plagiarism/surface learning Poor assessment design  plagiarism/surface learning Wide variations in unit demands in unit demands in marking across units in marking across units in marking criteria/guidance in marking criteria/guidance Inadequate feedback (quality/quantity/timeliness) to students

Aligning Assessment Feedback Criteria Marking Aims Aims Intended Learning Outcomes Methods of Learning Methods of Learning Assessment methods and tasks

An emerging model Feedbac k Quality Management: Standards Benchmarking Code of Practice Prog. specification. University: Policy Strategy Professional Bodies Aims Aims Outcomes Outcomes Learning Methods Marking Assessment Criteria External Influences on Assessment

Trends in assessment Tutor led Tests, Examinations Student led Peer marking, Peer assessment Presentations Group work Projects, Group projects Choice of assignments, dissertations StudentCentred

Marking Who does it? What points of reference are used? Types of assessment criteria Marking schemes Marking strategies Factors influencing marks awarded

Feedback Problems with feedback Characteristics of good feedback The feedback ‘sandwich’ Providing feedback efficiently Care required…….

Good Practice in Assessment Consider: 1. What kinds of things do you want them to learn? 2. What learning opportunities will be provided? 3. What assessment tasks will be set? 4.What criteria will be used?