Assessment and Feedback in PACE Units

What is PACE? and What is Participation? PACE Project Team: Theresa Winchester-Seeto & Marina Harvey Debra Coulson & Jacqueline Mackaway (Learning and Teaching Centre, Macquarie University)

What is PACE? Participation And Community Engagement This includes curricular and co-curricular activities.

WHAT IS PARTICIPATION? Learning through contribution to the community. Learning that occurs within an academic framework and produces academic outcomes. Learning that is potentially transformative.

Criteria for Participation Units Include community engagement; and development of graduate capabilities Can now be viewed on http://www.mq.edu.au/ltc/projects/curriculum_renewal/pace_criteria.htm

EXAMPLES OF PARTICIPATION Work Integrated Learning Internships Practicum Service Learning Community projects e.g. shop front style Academic research that include community engagement

FLEXIBILITY Location Locally (Sydney Regionally (NSW and Australia) Overseas Curricular or Co-Curricular Frequency Block Weekly Accumulated

IMPLEMENTATION 2009-2011 – pilot and preparation stage From 2012 – participation a requirement for all undergraduate students at Macquarie – integrated within undergraduate degree structure

Challenges - Organisational Scale Communication Resourcing Logistics Partnership development and maintenance Equity and Access

Challenges for Learning and Teaching Providing quality learning experiences to all students Providing a robust academic experience for students Ensuring a rigorous scholarly framework for curriculum design and delivery Support for staff in curriculum design and delivery Working out the role of “reflection” Addressing issues of assessment

Learning Outcomes You will be able to: recognise features of effective assessment and feedback identify issues related to designing and implementing assessment and feedback within a Participation context discuss implications and recommendations for assessment and feedback in Participation units

Assessment works best when… Assessment is a central feature of all learning & teaching: Assessment is aligned with all aspects of the curriculum (Biggs, 2003) Assessment is used to actively engage students in learning: Appropriate and authentic tasks Peer and self assessment (Falchikov, 2005) Useful and timely feedback (Nicol & MacFarlane-Dick, 2006)

Assessment works best when…cont’ Assessment is transparent: Linked to learning outcomes & activities (Ramsden, 2003) Clear & specific criteria/standards Assessment is inclusive and equitable: Variety of methods/tasks used Combination of formative and summative (ALTC, Assessment 2020) Assessment serves “triple duty”: Supports: learning; certification; life long learning (Boud, 2000; Boud & Falchikov, 2006)

Students Value Clear expectations and transparency Clear links between lectures, tutorials, content and assessment tasks Authentic tasks Autonomy – possibilities for negotiation… Timely and useful feedback (CSHE, UniMelb 2002)

Students are demanding more feedback Hot Issue @ Macquarie Students are demanding more feedback

The feedback loop Hounsell, McCune, Hounsell and Litjens (2006)

Assessment & Feedback in Participation Units: an activity Read scenarios Discuss in group: What is the purpose of this Participation unit? Issues relating to assessment and feedback Recommendations Present findings to workshop group

Literature Review Approach: Findings Steps taken: Grounded theory approach Identifying key issues/themes from literature Iterative process Analysis /aggregation Findings Steps taken: Development of resources Conferences/Workshops

“In search of a scholarly underpinning…” Education Theorists Bloom Biggs Boud Brookfield Dewey Kolb Lave & Wenger Mezirow Moon Schon Vygotsky Relevant Research/ Discourse Brodie & Irving (2007) Correira & Bleicher (2008) Eyler (2000;2002) Dressler & Keeling (2004) Hatcher, Bringle & Muthiah (2004) Kreber & Castleden (2009) Leung & Kember (2003) Falchikov (2007; 2005) Holland (1996,2001) Hounsell (2007) Nichol & Macfarlane-Dick (2006) Coll & Eames (2004) Coll, Fleming ,Hodges (2009) Poikela, (2004) Meta analysis Evidence-based Practice Ash, Clayton & Atkinson (2005) Astin, Vogelgesang, Ikeda & Yee (2000) Van Gyn, Branston, Cutt, Loken, Ricks (1997) Choy (2009) Clarke & Burgess (2009) Hayward & Blackmer (2009) Keily (2005; 2004) McAllister et al (2006) Stupans & Owens (2009) McNamara & Field (2007) McNamara (2009)

General findings from the Literature State of literature: Dispersed Fragmented Dominated by certain types of Participation / Disciplines Under-representation of “assessment” within the literature Lack of longitudinal/empirical studies Reliance on student self-reporting

Assessment Issues and Themes Principles of good teaching / assessment still apply Purpose of Participation impacts assessment Participation is different from classroom learning “Old tools” may not work when measuring “new/different” outcomes Multiple variables impact assessment Reliance on reflection “Old tools” (i.e, exams & essays) may not work when measuring “new/different” outcomes: - Not going to discuss in a BIG way here as we are aiming to design a workshop that specifically examines assessment design and tools in a more practical / hands on way. * May need a more “holistic” approach and range of assessment tools. Will say here: - Participation units are often attempting to assess different outcomes or a combination of outcomes not usually assessed: eg: integration of knowledge/skills ; competencies for certification purposes ; “soft skills” or generic work ready type skills ; OR some type of “transformation” in terms of a student’s underlying beliefs, attitudes, etc (often the case in a Service Learning type context).

Principles of good teaching and assessment still apply Transparent alignment Flexible approach Formative & summative assessment Authentic tasks / assessment Effective feedback Already touched on at the beginning of the workshop. Couple of important points in relation to the Participation context: Theme emerging for the literature is that of having a flexible approach to assessment and feedback. Could mean: a “re-think” about the way you have approached assessment in the past – may mean you use different tools. - could mean that you find ways to accommodate the various different workplaces and projects, etc each student will experience within your unit. The other important point raised in the literature is that of authentic experiences and assessment. This is particularly relevant if the unit is designed to support integration of knowledge/practice and prepare students for transition to workplace. 22

Purpose of Participation impacts assessment Different types of Participation Range of learning outcomes Discipline influences Personal influences Support life long learning Issues 23

Participation is different to classroom learning Issues Student centered Self directed Stakeholder involvement Off- campus Different learning outcomes / expectations Student centred & self-directed nature of Participation maybe a different experience for some students who use to a more teacher directed/supported classroom learning environment. Off-campus nature of Participation is an obvious difference. Clear challenge can be when the student is remotely located and the nature of frequency of contact with peers and teachers will be different – impacting feedback in particular. Need to re-think how to support. Stakeholder involvement – refers mainly to the involvement of workplace supervisor. Issues around how & extent of involvement / guidelines / support etc to help manage issues related to standards/consistency, etc What role should they play when it comes to assessment? Le Maistre warns about the “dual role” – mentor/assessor and how this can be particularly difficult when a student is underperforming. Reluctance to be involved (Thomas & Goa) Accounting for desire for authentic experience & assessment 24

Variables that Impact Assessment See handout

Reliance on Reflection Equity Validity Reliability Subjectivity and consistency Task weighting Work loads Issues The use of reflection for both learning and assessment purposes in Participation units is pervasive. Too many examples to cite. Equity – not all students naturally reflect & it relies on high language proficiency and written skills Validity – What to assess – process or content? Reliability – Untruthfulness of students – “writing to please” (Pieridies). Activities need to be well designed, structured and guided if going to be reliable/valid for assessment purposes (McNamara – QUT) Workloads – takes time and needs to be scaffolded (Moon); Students find it time consuming (Pierides) as do staff when it comes to reading/marking. 26

Implications for Practitioners “Paradigm shift” Design related Tool related Time / resource related

Things to consider…. Graduate Capabilities MQ Assessment Policy Unit evaluation

Want to Know More? Further workshops: 6 May: Assessing Reflection in PACE Units In planning: 1)Scaffolding reflection 2)Assessment tools Learning and Teaching Centre website: http://www.mq.edu.au/ltc/projects/curriculum_renewal/pace_resources.htm 29 29 29

Resources Macquarie University: http://www.mq.edu.au/ltc/projects/curriculum_renewal/pace_resources.htm http://www.mq.edu.au/ltc/resources/index.htm Assessment Futures: http://www.iml.uts.edu.au/assessment-futures/index.html Australian Universities Teaching Committee http://www.cshe.unimelb.edu.au/assessinglearning/index.html

References Biggs, J. (2003), Teaching for Quality Learning at University, 2nd Edition, Open University Press Boud, D., (2000) Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22 (2), pp.151-167. Boud, D. & Falchikov, N., (2006) Aligning assessment with long-term learning, Assessment and Evaluation in Higher Education, 31 (4), p.399-413. Falchikov, N., (2005) Improving assessment through student involvement: Practical solutions for aiding learning in higher and further education. New York: RoutledgeFalmer. Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2006) The Guidance and Feedback Loop. University of Edinburgh. Accessed: http://www.cshe.unimelb.edu.au/colloq06/Hounsell3.pdf

References cont’ Moon, J.A. (2004). A handbook of reflective and experiential learning: Theory and practice. London: RoutledgeFalmer. Nicol, D. J. & Macfarlane-Dick, D. (2006), Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education Vol 31, 2, April 2006, pp.199-218. Ramsden, P (2003). Learning to Teach in Higher Education. 2nd edition. London & New York : RoutledgeFalmer Note: Please refer to Annotated Bibliography on Assessment and Feedback in PACE units found at : http://www.mq.edu.au/ltc/projects/curriculum_renewal/pace_resources.htm