Is our assessment up to standard? David Boud University of Technology, Sydney and Centre for Research in Assessment and Digital Learning, Deakin University University of Sydney Teaching Colloquium, September 2014

Overview Part 1. What are academic standards now and why should we bother with them? Part 2. The changing ground of assessment Part 3. What are the implications of the standards agenda for assessment? Part 4. What else is there to consider?

Part 1 What are academic standards now and why should we bother with them?

What do we now mean by academic standards? From positional rhetoric: ‘we are committed to maintaining high academic standards’ to something we are judged by: ‘are your assessment practices up to standard?’

From where do academic standards arise? Intrinsic to the structure of disciplinary knowledge consensus views of experts analysis of actual professional practices

From whom do they come? Ourselves/our discipline Professional bodies Accreditation agencies (eg. AACSB) International agreements (eg. AHELO)

The new landscape of academic standards AQF TEQSA The OECD move Professional registration bodies Accrediting agencies, eg. AACSB Threshold discipline learning outcomes Most importantly, ourselves, but beyond the implicit standards of disciplines Underpinning this is empirical justification of standards and comparability NB: most TEQSA standards are not about academic matters at all!

Part 2 The changing ground of assessment

What does assessment need to do? Contribute to certifying student performance – Summative assessment Provide students with useful information to aid their learning now – Formative assessment Build students’ capacity to make judgements about their own learning – Sustainable assessment

Managing the complexity of demands One activity cannot meet all the requirements of assessment – This is not a simple matter of having a diversity of methods Some purposes are incompatible because of timing or type of information needed – Eg. grades versus rich information In any given instance one purpose needs to be dominant – This needs to change over the progress of a course or program

Assessment decision-making framework assessmentdecisions.org/ decision framework assessmentdecisions.org/

Which of the following will you take into account in your design for assessment? – Learner characteristics – Institutional assessment policies /conventions – Professional, vocational requirements – Departmental, disciplinary and personal ‘signatures’ – The overall program – Learning environment

Part 3 What are the implications of the standards agenda for assessment?

The context of assessment today Assessment carries the burden of many sets of expectations Academics mediate these to ensure courses are worthwhile and are not captured by any one interest Assessment decision-making, is about balancing different sets of considerations, with a core of ensuring students have high quality experiences

What is assessment (in the context of learning outcomes)? Assessment is a judgement about whether students can demonstrate attainment of learning outcomes to a given standard. Transparent standards must be established for assessment tasks – Setting a pass mark is not setting a standard – Setting a general set of standards for a course is not enough – Use of terms such as good, superior, excellent does not indicate a standard or communicate a level All assessment must be standards-based (criterion- referenced) – Norm-referencing is excluded – Judging students against each other undermines standards

What does this imply? All assessment involves the identifying appropriate standards for the tasks students undertake A range of assessment approaches must be used appropriate to the range of learning outcomes to be assessed The balance of assessment approaches must reflect the range of learning outcomes Ensuring all the necessary outcomes are met by all students

What does it not imply? Standards are unilaterally applied Students are not involved in assessment All learning can be predetermined All learning can be easily measured/judged or is worthwhile Existing use of marks and grades is compatible with a standards framework

Part 4 What else is important?

Standards implies fixity, but how can we be responsive?

The role of feedback: what can it do? Bridges the gap between teaching and learning, ensuring the curriculum is adjusted to the needs and learning of each student Cannot be enacted without the engagement of participants—students and teachers. Needs to respond to what students actually do.

What is feedback? “a process whereby learners obtain information about their work in order to appreciate the similarities and differences between appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work” Boud and Molloy 2013

Properly understanding feedback Feedback is about effects not inputs Feedback is not synonymous with the comments we provide to students’ about their work Feedback cannot be said to have occurred unless a student’s work is positively influenced Implies: course units need to be designed to allow for multiple feedback loops to be incorporated within a semester.

Feedback as Iterative Task Design Nesting of tasks enables feedback loop to be completed through knowledge of the effects of earlier information provision in subsequent tasks. From Boud and Molloy (2013) Activity 3 Activity 2 Activity 1 Degree of task challenge Time through semester Overlap of learning outcomes

What do/can students do? Be expected to act on feedback –at least to respond to it Ask for particular feedback –to position them as active players Engage in a following task in areas in which they have not reached an appropriate standard Be expected to monitor their own achievement

The feedback-enabled curriculum Has early strategies to shift learner identity to becoming self- regulated Positions feedback as part of learning, not as an adjunct of assessment Equips students to be skilled and comfortable with negotiating learning outcomes, feedback processes and information needs Fosters ongoing ‘dialogue’ between students and teachers about feedback processes, the nature of standards and the practicing of judgement. Introduces activities to enable students to calibrate judgement (of their own work and that of others)

Finally, being responsive is not just about effective feedback

Student engagement in assessment Why involve them? They need to know how to judge the work of themselves and others They need to appreciate standards and criteria for good work to be effective learners They can’t act effectively unless they know what they know/don’t know

Every act of assessment should build students’ capacity to judge work The most basic outcome of any course is that students can judge good work This capacity needs to be developed over time in conjunction with the assessment of tasks Implies: Design the activities involved in each task to enable students to develop skills of judging their own work and that of others

How can students be more actively involved in assessment? Applicable to all students Through not being the passive recipients of assessment acts Through choosing assessment tasks appropriate to learning outcomes Through identifying standards and developing suitable criteria by which to judge their work Through acting as sources of feedback information for others Through being expected to act on feedback in subsequent work

Realising the potential of assessment Assessment continues to be the single most powerful influence on learning Attention to assessment tasks and what students (and we) do in association with them is more likely to shape students positively than anything else we can do Assessment standards are so important, we can’t maintain a monopoly on ensuring they are applied: responsibility for ensuring standards are met is a continuous responsibility for all practitioners, including learners

Some criteria for good assessment 1.Together, assessment activities address the different purposes of assessment 2.Linked to full range of unit and (agreed) program outcomes 3.Enables multiple feedback loops to be utilised 4.Aligned with learning activities, not just presentations 5.Tasks are worthwhile/significant in their own right (not unnaturally contrived for assessment purposes) 6.Enables judgements to be made about whether all students have met given standards 7.Equips students to judge their own work 8.Builds students’ capacity to continue to learn

Resources Assessment decision-making for course units OLT project: assessmentdecisions.org assessmentdecisions.org Feedback University of Edinburgh tla.ed.ac.uk/feedback/index.html tla.ed.ac.uk/feedback/index.html Re-Engineering Assessment Practices in Scottish Higher Education reap.ac.uk reap.ac.uk Assessment for future learning OLT Fellowship: assessmentfutures.com assessmentfutures.com

Remove spurious levels of accuracy Use of percentages implies detection of variations impossible to make Never grade on a scale more fine-grained than the distinctions that can reasonably be made Summative assessment is about judging if learning outcomes have been met, not making measurements of detailed performance on a task Implies: are more than the four common passing grades ever needed?