Margaret Macleod Keithia Wilson Griffith University Quality Improvement of First Year Assessment: A process and template for the review & enhancement of assessment design & management Margaret Macleod Keithia Wilson Griffith University

Acknowledgment to Country In the Spirit of Reconciliation Following on from Sorry Day I would like to acknowledge & honour the Traditional Custodians of the land we are meeting on today, Turrbal and the Jagera People, and pay respect to their Elders past & present And acknowledge the contribution of our First Nation Peoples to Higher Education & learning Keithia Wilson GU - May 2012

Introductions Margaret Macleod: Currently the Griffith Health Group Assessment Consultant 10 years previous experience as Learning Adviser at Griffith University, Logan campus Prof Kethia Wilson: Program Director of the FYE, Griffith Health ALTC National Fellow for the FYE (2010-2012)

Session Overview 1. Mini-input Reflection & group discussion Context for the intervention Literature - Importance of assessment Assessment review process Assessment review template Evaluation & Outcomes Reflection & group discussion 3. Final summary of themes

Griffith University context Large metropolitan university in Brisbane (1 of 7 in S-E Qld, & 1 of 4 in Brisbane) Multi-campus - 5 campuses x 60 k corridor Student enrolment of 40,000 70% of students are first-in-family at uni - FIF correlates with low SES & lower entry scores 6th highest low SES student intake in Australia 3rd highest Indigenous student intake in Australia 25% International student enrolment 10% students studying in distance mode Keithia Wilson GU - May 2012

Why assessment matters! There is very strong evidence in the literature to suggest – Assessment is a central feature of curriculum Assessment practices have a profound effect on student behavior (Entwistle, 1991) Assessment is believed to have a more powerful influence on student behavior than teaching (Boud, 2012) Assessment has also been identified by leading educators as one of the least sophisticated aspects of learning and teaching in higher education (James, 2010)

In Summary “While students can with difficulty escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment”. (David Boud, 1995, p 35)

Effects of Poor Assessment The effects of poor assessment have been well documented in the literature (Rust, 2002) – Student disengagement from learning, and The development of negative attitudes towards learning

Impact of poor assessment on Non-Traditional students Anecdotal & observational data from Professional staff (e.g., Learning Advisors) indicates that poor assessment contributes to students, particularly non- traditional students, experience a sense of personal inadequacy, disempowerment, perceived inability to negotiate the complexities of university culture, and lower retention rates.  

First Year Students’ Appraisals of Assessment Wilson & Lizzio (2012) ALTC Grant Commencing students evaluate their assessment tasks in terms of two factors: The motivational value of the task, and Its manageability, or the extent to which the task is scaffolded, with resources provided to enhance engagement and understanding.

Key Findings for FY Assessment Students’ perceptions of the motivational value of assessment tasks are key in predicting student engagement, Students’ perceptions of the effectiveness of the design of the assessment task are key in predicting their levels of motivation for the task, Student evaluations of task manageability are key in predicting student’s sense of self-efficacy or confidence in undertaking the assessment task.

Dual Assessment Factors guiding the Review Process Thus, the dual factors of Effective /appropriate task design Combined with processes for managing the assessment experience and scaffolding student understanding of the assessment task are core to a positive, empowering and potentially successful student experience with assessment Reinforced by feedback from Professional staff, in particular, Learning Advisors

Common problems students experience with assessment Difficult, ambiguous or complex tasks Tasks that lack relevance to learning outcomes or ‘sense of purpose’ Misleading or generic marking criteria Inconsistencies in task information Insufficient resources to help scaffold tasks Inappropriately timed tasks

Review Process Overview Academic-professional staff partnership Worked together to design and implement a process to review and enhance first year assessment, and work effectively with academic staff Macro (FY Program) and micro (Individual tasks) level review and enhancement of first year assessment Undertaken in two Schools in Griffith Health in 2010 Strategy: Focus on student voice and understanding of student experience of assessment Academic staff engagement Design of template to evaluate written assessment tasks Report, including recommendations and resource development Change process combining FYE Team & Individual Convenor discussions for decision-making at FY Program & Course/Unit levels

The Review Process: Design of Templates Macro level Template Programmatic focus: comparison of all assessment tasks re timing, type, variety, word length, weighting Micro Level Template Individual assessment task focus: Assessment design Scaffolding of tasks and targeted Resources Final report: Overall summary and detailed evaluation of written assessment in all first semester, first year courses in each School Emphasise the importance of this being an enabling as well as evaluative process – focus on enhancement, with clear suggestion for how tasks could be strengthened – esp. through resource enhancement

2. Clarity: Is assignment task clear and unambiguous? Assessment Design 1. Relationship to Course Learning Outcomes: Does the assignment task clearly relate to/develop the learning outcomes for the course? 2. Clarity: Is assignment task clear and unambiguous? 3. Criteria: Do marking criteria assist in understanding the assignment task? 4. Level of difficulty: Is assignment task suitable for students’ level of skill development? 5. Weighting: Is weighting appropriate for assignment task? 6. Size of task: Is word length appropriate for assignment task? Assessment Resources Are there sufficient resources provided to assist students with this task? Suggestions for Scaffolding & Enhancement of student learning Assessment redesign Resource enhancement Margaret Macleod, GU, 2011

The Review Process: Academic staff engagement Ensuring academic support: Creation of FY leadership team, comprising Dean L&T (Health), HOS/DHOS, Degree Program Convenor/s, First Year Advisors, & FY Course/ Unit Convenors Creation of FY Enhancement/Management team, comprising FY Leadership Team + FY Course/Unit Convenors Facilitating staff buy-in & engagement: Consultation process with FY staff prior to review to seek commitment & agreement (or otherwise) to the audit process 3. Conducting the Assessment Review: Accessing Course/unit outlines for analysis Preparation of Final Report Facilitating the feedback process: Macro level: programmatic discussions with FY leadership teams to discuss report, share information & make FY Program decisions Micro level: Learning Adviser met with individual course convenors to assist in redesign of assessment & creation of resources to scaffold student engagement & learning

Assessment Review Outcomes: Macro level audits Program Level changes Relocation of some more difficult courses & tasks to 2nd year (e.g., research methods) Reduction of high stakes, highly weighted (e.g., 40 to 50%) first assessment tasks in favour of early, formative low stakes assessment tasks in most courses Agreement in FYE Teams on total word lengths for semester 1 tasks (from 5-6,000 to 4,000 words); Reduced standardised weightings for word length (e.g., 1,500 word range from 15 to 45%); Reduction in number of group work tasks for assessment purposes & later submission dates (not before week 6) Reduction in number of referencing styles (from 2/3 to 1) Coordination of submission dates Blackboard Audit resulting in consistent information storage

Assessment Review Outcomes: Micro level audits Individual assessment task changes: Total redesign of some assessment tasks Partial redesign of many tasks re task clarity, level of difficulty, variety, relation to course aims, marking criteria Development of task specific resources to scaffold student learning

Assessment Review Evaluation Staff and student outcomes: evaluated using data from commencing students (Starting@Griffith Survey – week 7/semester 1) and feedback from teaching staff (end sem 1) Student feedback: Increases in 2011 from 2010 in student engagement between 5 & 7% student satisfaction between 6 & 8% student capability up by 10% perceptions of good teaching between 11 & 14% These were the highest in the university Academic staff feedback: Substantial reductions in student enquiries, student anxiety, & failure rates in threshold courses/units Positive perceptions of process – requests that review process be continued into 2nd and 3rd years. Starting@Griffith survey: weeks 4-7, all commencing students , 40% responbse rate evaluates experinces of orientation and early learning environment, compares this to previous years. e Cond