Assessment and Learning in Practice Settings (ALPS) © ALPS CETL What is ALPS ( )? A centre for excellence in teaching and learning focussing on assessment and learning in practice settings. 5 Universities (Leeds, Leeds Metropolitan, Bradford, Huddersfield and York St John) crossing 16 health and social care professions What was the aim of this programme? To achieve excellence in assessment and learning based on interprofessional common competency maps, delivered by mobile technology* To ensure that students graduating from courses in health and social care are fully equipped to perform confidently and competently at the start of their professional careers

Assessment and Learning in Practice Settings (ALPS) © Inspire our students! We aim to; Introduce the outcomes of the ALPS programme; the ALPS Common Competency maps, the ALPS generic assessment tools, the ALPS “Competence in Practice Assessment” (CiPA) tool To share an example of how one of the ALPS professions used these tools Enable participants to investigate the opportunities to apply the ALPS outcomes to their programmes Identify the potential benefits of utilising the ALPS outcomes for our students

Assessment and Learning in Practice Settings (ALPS) © Common Competency Mapping The purpose; To agree the predominant common competences for the 16 different health and social professions. To map these agreed common competences The process Developed multi-professional working groups Agreed communication systems; meetings and “sharepoint” website Workshops for professional statutory and regulatory bodies. Service user involvement Extensive consultation

Assessment and Learning in Practice Settings (ALPS) © Common Competency Maps Result – 3 Maps Communication Skills Map Teamworking Skills Map Ethical Practice Skills Map Holt et al (2010)

Assessment and Learning in Practice Settings (ALPS) © Development of globalised maps The purpose; The 3 health and social care maps (Communication, teamworking and ethical practice) contained very “health and social care-type” language. However, it was realised that these skills and attributes are a common requirement across all disciplines and professions. They also link with the “Leeds for Life” The process Wording of the 3 maps “globalised” Extensive consultation with departments outside health and social care and other Universities.

Assessment and Learning in Practice Settings (ALPS) © Activity; 1 – the maps We ask participants to; Read the resultant globalised communication map. Are these maps compatible for the requirements of your undergraduate students? Discuss how they could enable your students to fulfil the requirements for the Leeds for Life and employability.

Assessment and Learning in Practice Settings (ALPS) © ALPS generic assessment tools The ALPS assessment tools are ; Reflective in nature, students can take feedback from a variety of sources, and hence enabling life-long learning skills. (Boud 2000) Primarily assessments for learning; a skill which is important for employability. (QAA 2009) Interprofessional, as they are based on the 3 agreed common competences – therefore students can take feedback from other professions and service users – they are student-led. Designed to be used by students whilst on their practice placement and represent commonly occurring scenarios Valid in nature as extensive consultation was used for their development. Evidence was collected and analysed to demonstrate their reliability and utility.

Assessment and Learning in Practice Settings (ALPS) © The ALPS assessment tools

Assessment and Learning in Practice Settings (ALPS) © ALPS Assessment Tools Assessment Types: -Gaining Consent -Demonstrating Respect for Service Users and Carers -Knowing when to consult or refer -Working interprofessionally -Providing information to colleagues Assessment tool options; Self, peer, service user, practice assessor & action plan Assessment delivery; Mobile, PC (online) paper

Assessment and Learning in Practice Settings (ALPS) © Reliability and Validity of the Tools Challenge of ALPS Assessments: -Flexibility/options working well for 16 ALPS professions BUT means getting enough of any one type/selection of assessments is difficult -Have collected 200+ Assessment Tools, but diversity of Tool usage means we do not yet have a ‘critical mass’ for analysis Triangulation – minimum 3 sections completed – the more responses, the more ‘reliable’ the assessment.

Assessment and Learning in Practice Settings (ALPS) © Assessments Completed by Profession AssessmentAudiologyMedicineOccupational Therapy Social WorkTOTAL Gaining Consent Demo Respect Providing Info Working Interprof Knowing when to Consult & Refer TOTAL Roberts 2010

Assessment and Learning in Practice Settings (ALPS) © Assessments Completed by Type of Assessor Roberts 2010 AssessmentAudiologyMedicine Occupational Therapy Social WorkTOTAL Student Student + Peer Student + Service User03104 Student + Practice Assessor Student, Peer +Service User Student, Peer + Practice Assessor Student, Service User + Practice Assessor Student, Service User, Peer + Practice Assessor TOTAL NB 1 missing result (SW)

Assessment and Learning in Practice Settings (ALPS) © Inspiring students through multisourced feedback AssessmentStudent Profession AudiologyMedicine Occupational Therapy Social Work TOTAL Gaining consent Demonstrating respect TOTAL Roberts 2010

Assessment and Learning in Practice Settings (ALPS) © Reliability Gaining Consent Cronbach’s alpha = 0.55 Demonstrating respect for Service User Cronbach’s alpha = 0.66 Roberts 2010

Assessment and Learning in Practice Settings (ALPS) © Inspiring students through multisourced feedback “Well the tool gave us a structure, the learning structure to fulfil criteria; if we didn’t have that then I don’t know how focused you would be” (Radiography Student, P2 Evaluation) “You don’t get to know other professions the same way you do the Doctors”. (3 rd Year Medical Student, P2 evaluation) “Yes, it I think it really depends on the service user group that you’re working with, I was working with teenagers” (3rd Year Social Work Student, P2 evaluation)

Assessment and Learning in Practice Settings (ALPS) © Inspiring students through multisourced feedback “ We are at the same level, you’d just be talking out of your...” No but we did give each other tips”. (3 rd Year Radiography Student, P2 evaluation) “Yes I’ve done it, say they’d done a Gracchi view, you would say like “Wow”. (3 rd Year Radiography Student, P2 evaluation) “I was moving between wards a lot and nurses are always very busy”. (3 rd Year Medical Student, P2 evaluation “It gets you into the habit of reflecting. Would be better done earlier, perhaps year 1? With clinical placements becoming earlier in the curriculum this may be possible.” (Year 3 Medical Student) “The Idea of multi source feedback is good and links well with what we are expected to do in other parts of the course”. (Year 3 medical Student)

Assessment and Learning in Practice Settings (ALPS) © Inspiring students through multisourced feedback On peer assessment: I think a benefit is probably feedback is less threatening in a sense like when if your educator sort of says you kind of think is that going to effect my grade is that where as the peer assessment they can always pull you on the side and go I did noticed you know what I mean so I think there is a lot of benefits to it but its just like I say not everybody will have that opportunity depending whether there is other people in on the same placement. (BFD OTS) It is the least threatening type of feedback. (LDS 3 rd Yr Medics) On service user feedback: I think like, I would value patient feedback because they’re the ones ultimately who can give you the most honest for of, you know, feedback on how you perform because they’re the patient but it is difficult to do, if there weren’t all the problems then yea I would value patient feedback but it’s just difficult so …(LDS AUD PC) … a discussion was better rather than just handing the device (3 rd Yr Medics) You would have to do it in some departments than others. (3 rd Yr Radiography)

Assessment and Learning in Practice Settings (ALPS) © Inspiring students through multisourced feedback Multi- and interprofessional: Learning Assessment Feedback Linking tools with occupational standards. A structure that enables learners “hit their roles”. Ability to generate evidence for own professional requirements. Professional interactions. Seeking and giving permission to share information and to receive feedback. Reflection

Assessment and Learning in Practice Settings (ALPS) © Inspiring students through multisourced feedback Issues for Consideration from Focus Groups Culture – the way it has always been. Perception – do others understand what we do/say? – nurses were/are too busy Environmental constraints: confined space; nature of task/intervention; moving about. Professional – citadels of age old beliefs and ideals. Linked with culture.

Assessment and Learning in Practice Settings (ALPS) © September Staff Interviews Social Work Practice Educator “Yes I do, I think it looks at different perspectives, different approaches, I do definitely think …” “Yes, they, it took work, it was more work I have to say, but because I know the benefits at the end it was worth the input, even from my point of view it was more work for me and so you need to really be organised...” Radiography Tutor ‘Providing Information to other Healthcare Professionals’, for us those are essential pieces, I mean conveying the information, one of the things students have to do now is produce commenting forms...” “... that Radiographers now, when they are qualified, have got to be able to communicate what is on an x-ray to other members of staff...” “... and I think using that tool is going to be something that we can use throughout to help enhance that”.

Assessment and Learning in Practice Settings (ALPS) © The ALPS assessment tools

Assessment and Learning in Practice Settings (ALPS) © Activity; 2 – the tools We ask participants to; Discuss commonly occurring scenarios in their academic area – related to employed activity and based on the common competences Consider how to develop assessment tools to enhance your students learning in practice skills/employability Can you name some examples to share with other participants.

Assessment and Learning in Practice Settings (ALPS) © Social Work – an example Paper versions of the 5 assessment tools were mapped to the National Occupational Standards for social work with completed examples Workshops with students and with practice educators were held Students on all three years of the BA Social Work programme used the tools and were asked to give feedback in focus groups

Assessment and Learning in Practice Settings (ALPS) © Date and place of assessment Social Services Information Point Questions for Student Student statementExamples of claims in relation to GSCC Code of Practice / Key Roles Describe your interaction with a service user. I took a telephone call on duty. The service user wanted to know about car badges I checked whether or not the service user was already known to us and whether or not they had any other needs that we might be able to help with. I was aware that sometimes straightforward requests might hide other issues and needs that could be discussed and may be more formally assessed under our legal obligations in relation to for instance the NHS&CC Act 1990 or the Carers, (Recognition and Services) Act 1995 and Carers (Equal Opportunities) Act 2004 I filled in the relevant forms and passed them on to the relevant colleague I discussed the work I did with an experienced colleague to check I had done things correctly Key role 1 Prepare for and work with people Unit 1. Evaluate all information to identify the best form of initial involvement Key role 5 Manage social work practice Unit 14.3 Monitor and evaluate the effectiveness of your programme of work in meeting the organisational requirements and the needs of individuals, families, carers, groups and communities. Key role 6 Demonstrate professional competence Unit 19.3 Work within the principles and values underpinning social work practice. Unit 19.5 Use supervision and support to take action to meet continuing professional development needs. GSCC Code of Practice 6. Be accountable for the quality of their work and take responsibility for maintaining and improving their knowledge and skills.

Assessment and Learning in Practice Settings (ALPS) © Social Work Focus Group Outcomes Students liked; - They understood how the observations in practice related to their NOS and produced evidence for their portfolio –Consistency of gathering evidence across a range of activity –More confidence in asking for feedback; particularly from service users –Encouragement of analysis and reflection –Student control over the assessment feedback process –Higher quality of evidence for portfolio There was consistency of feedback across the three focus groups over two placements and three academic levels

Assessment and Learning in Practice Settings (ALPS) © Social Work Focus Group Feedback ‘I found (the questions) really helpful because…..some of the service users …..didn’t know what to say so it gave good guidance (as to) what kind of things we wanted them to look for’ ‘My practice placement educator, she likes the guidance’ ‘There’s an increased emphasis on our reflective practice so it’s very good for that’ ’Looking at the (NOS) examples that were given it …made me think ‘ah right that does meet that role’ when I would have thought my work actually wasn’t meeting as many roles as it was. It helped me to reflect and (realise) I’m doing a lot more than I think I am’ (several students agreed with this) ‘It’s more focused evidence’ ‘I probably feel less anxious….maybe because it is student led…I know what I am being observed on’

Assessment and Learning in Practice Settings (ALPS) © Activity; 3 – the Competency in Practice Assessment questionnaire We ask participants to ; Consider using this questionnaire for their students, to enhance and enable their understating of their own competence to practice. Any comments or feedback?

Assessment and Learning in Practice Settings (ALPS) © Bibliography Boud D. (2000) Sustainable Assessment: rethinking assessment for the learning society. Studies in Continuing Education , Gibbs G, Simpson C, Macdonald R (2003) Improving student learning through changing assessment – a conceptual and practical framework. EARLI Conference, Padova. Holt J, Coates C, Cotterill D, Eastburn S, Laxton J, Mistry H, Young C. (2010) Identifying common competences in health and social care; An example of multi-institutional and inter-professional working. Nurse Education Today Quality Assurance Agency for Higher Education(2009). Transforming assessment and feedback: enhancing integration and empowerment in the first year. Quality Enhancement Themes. The First Year Experience. ISBN Roberts T (2010) Utility of the ALPS Assessment Tools. Presentation at the PSRB Seminar. June 2010.

Julie Laxton Joe Cortis Alan Murphy Shupikai Rinomhota Assessment and Learning in Practice Settings (ALPS) © ALPS Contact Details