Evaluation of service user-led role play feedback for social work students Dr Eleni Skoura-Kirk, Lecturer in Social Work Sarah Brown, Senior Lecturer in Social Work Dr Rasa Mikelyte, Research Assistant - CHSS

Introduction Evaluation of service-user led role play interviews as part of the Readiness for Direct Practice module (BA year 1 students) Examine the development of professional skills (procedural and meta-competencies; Bogo, 2006) for social work students through assessment feedback from service users, practice educators and student self-assessments. Do role-plays with service users lead to better communication, interpersonal and reflective skills?

Skills development through role-plays Skill development through experiential learning (Kolb, 1984); importance of reflective observation (Cheung and Dalavega 2014; Schon, 1987) Service user and carer involvement: involvement in role-plays creates a more ‘authentic’ experience self-reported improvements in developing empathy, active listening skills, self- awareness and challenge to preconceptions (Skilton, 2011; Duffy, Das and Davidson, 2013; Hitchin, 2016)

Role play interviews: design & implementation OSCE model adapted for social work (Bogo et al, 2006): observed standardized role-play interviews, assessed for development of professional skills Procedural skills (ability to undertake set tasks to a required level; in this case relationship- forming and professional communication skills) Meta-competencies (cognitive, critical and self-reflective capacities of the learner) Formative assessment based on: PI feedback, Student self reflection and PE group feedback

Role play interviews: design & implementation Role-play based on scenario written by Partnership Initiative members (PIs), given to students in advance. Reflecting PCF Readiness for Direct Practice levels, assessing building rapport, engaging with the individual and identifying some initial assessment information. Learning and Teaching Enhancement Fund (University of Kent) Roleplay 1 Roleplay 2 15 minute roleplay between 1 Student and 1 PI member Feedback collected from Students and PIs soon after PE-facilitated reflective discussion 1 week later, PE feedback collected 15 minute roleplay between 1 Student and 1 PI member Feedback collected from Students and PIs soon after PE-facilitated reflective discussion 1 week later, PE feedback collected November 2017 March 2018

Methodology Students: Participants: 70% 20-30yos 17% 30-40yos 97% Female

Methodology Qualitative section: Quantitative section: Thematic analysis of: PI qualitative feedback on assessment forms Student self- assessment forms completed soon after the role plays (qualitative feedback) PE reflective discussion feedback form (qualitative) Quantitative section: PE Survey on roleplay performance 3 items; 10-point semantic differential scale Student Self-Rating Survey on roleplay performance 5 items; 10-point semantic differential scale PI Survey on roleplay performance 15 items; 5-point Likert scale Repeat measures

Qualitative Findings

1. Overall improvement Students PIs PEs “Since the first role play I feel that I have gained many new communication skills, I am a lot more confident when speaking to service users” (S18) PIs From: “Student was so nervous that it is very difficult to answer these two questions. She shows signs of future promise but will have to work on her approach to service users and lack of confidence” To: “Student goes from strength to strength. She is confident and capable and knows how to ask the right questions” (P03 for S22) PEs “[The students] were more confident regarding asking open-ended questions compared to when they completed the initial role plays” (E02)

2. Diverging emphases of what improved Students Procedural & Explicit Improvement PEs Cognitive & Theory-to-Practice Improvement PIs Relational & Embodied Improvement

2. Diverging emphases of what improved Students: Procedural & Explicit Improvement PEs: Cognitive & Theory-to-Practice Improvement PIs: Relational & Embodied Improvement “I ensured that I was: summarising, reflecting, using minimal encouragers, showing empathy, used empowering speech and my body language was open” (S24) “More discussion about theories used – strengths-based, Active listening and paraphrasing, Narrative theory, Ecological Theory, Systems Theory and even Solution-focussed” (E01) “Student was very engaged and completely interested in and focused on what I was saying” (P03)

3. Competing ideas on what improvement is desired Problem-Solving Students “I probably attempted to problem solve too early, rather than establishing a relationship/rappo rt with the service user” (S27) PEs “We discussed the temptation to problem- solve and offer solutions before the assessment stage has been completed. It was explored that there could be a risk of not only raising unrealistic expectations, but also not listening to the service user” (E02) PIs “Willingness to find out what can be done.” (P07) “Very quick to think of solutions” (P05)

Quantitative Findings

Results Time 1 Time 2 M SD df t-test PE Rating Cognitive Development   Time 1 Time 2 M SD df t-test PE Rating Cognitive Development 6.63 1.19 7.97 .67 29 -7.62*** Reflective Development 7.80 .41 -1.43 Affective Development 8.10 .86 6.87 1.14 -4.85*** Student Self-Rating Performance 6.70 1.44 7.48 1.55 26 -2.01 Knowledge of theory 6.29 1.78 7.93 1.33 27 -4.65*** Confidence 6.46 1.80 1.54 -4.27*** Helpfulness of Reflection 8.89 1.83 9.18 1.52 -.90 Need to develop skills 6.15 2.07 4.44 2.08 3.23** PI Ratings Introduction 4.07 1.16 4.76 .44 28 -3.10** Explanation for meeting 1.08 4.40 .72 -1.44 Explanation of agenda 3.83 1.12 4.37 .77 -2.44* Sticking to time 4.20 .93 4.60 .68 -2.56* Listening to PI 4.80 -3.03* Allowing for PI views 4.50 .63 4.77 .50 -2.28* Understanding PI views 4.14 .83 4.55 .57 -2.35* Look out for PI comfort 3.89 .85 4.48 .58 -3.65** Treating PI with respect 4.66 .61 4.83 .38 -1.54 Care for PI feelings 4.30 .84 4.73 .52 -2.90** Asking relevant questions 4.17 .87 -1.84 Avoiding jargon 4.43 -2.19* Preparedness for meeting 4.24 .51 -1.88 Ending the meeting .75 -1.86 3.90 .96 .56

Results Time 1 Time 2 M SD df t-test PE Rating Cognitive Development   Time 1 Time 2 M SD df t-test PE Rating Cognitive Development 6.63 1.19 7.97 .67 29 -7.62*** Reflective Development 7.80 .41 -1.43 Affective Development 8.10 .86 6.87 1.14 -4.85*** Student Self-Rating Performance 6.70 1.44 7.48 1.55 26 -2.01 Knowledge of theory 6.29 1.78 7.93 1.33 27 -4.65*** Confidence 6.46 1.80 1.54 -4.27*** Helpfulness of Reflection 8.89 1.83 9.18 1.52 -.90 Need to develop skills 6.15 2.07 4.44 2.08 3.23** PI Ratings Introduction 4.07 1.16 4.76 .44 28 -3.10** Explanation for meeting 1.08 4.40 .72 -1.44 Explanation of agenda 3.83 1.12 4.37 .77 -2.44* Sticking to time 4.20 .93 4.60 .68 -2.56* Listening to PI 4.80 -3.03* Allowing for PI views 4.50 .63 4.77 .50 -2.28* Understanding PI views 4.14 .83 4.55 .57 -2.35* Look out for PI comfort 3.89 .85 4.48 .58 -3.65** Treating PI with respect 4.66 .61 4.83 .38 -1.54 Care for PI feelings 4.30 .84 4.73 .52 -2.90** Asking relevant questions 4.17 .87 -1.84 Avoiding jargon 4.43 -2.19* Preparedness for meeting 4.24 .51 -1.88 Ending the meeting .75 -1.86 3.90 .96 .56

Results Change between Time 1 and Time 2   N M SD Correlations 1 2 3 1. Change in PE Ratings 30 1.00 1.06 2. Change in Student Rating 27 .50 1.22 2.55 3. Change in PI Rating 26 .42 .63 -.12 -.02 Most ratings showed improvement at Roleplay 2, whether they were made by PIs, PEs or the students (based on t-test results) However, PI, PE and Student self-ratings did not go together Student self-perceived improvement or deterioration in roleplay performance was not linked to that seen by the PIs or the PEs

So… Do role-plays with service users lead to better communication, interpersonal and reflective skills?

Key messages Evidence suggests overall improvement in students’ professional skills. The repetition of the task contributed to increased confidence, allowed exercise and refinement of procedural skills and linking theory to practice. Three-fold assessment promotes a holistic approach to the task (procedural/cognitive/relational). Competing ideas: importance of service user and carer involvement in social work education, avoiding exclusive reliance on professional narratives.

References Bogo, M., Regehr, C., Woodford, M., Hughes, J., Power, R. & Regehr, G. (2006) ‘Beyond competencies: field instructors’ descriptions of student performance’, Journal of Social Work Education, vol. 42, no. 3, pp. 579–593. Cheung, M. and Dalavega, E. (2014) Five-way Experiential Learning Model for Social Work Education. Social Work Education, vol 33, no. 8, pp 1070-1087. Duffy, J., Das, C. and Davidson, D. (2013) Service User and Carer Involvement in Role-plays to Assess Readiness for Practice, Social Work Education, 32(1), pp. 39-54. Hitchin, S. (2016) Role-played interviews with service users in preparation for social work practice: exploring students’ and service users’ experience of co- produced workshops, Social Work Education, 35:8, 970-981 Kolb, D. A. (1984) Experiential Learning. Englewood Cliffs: Prentice Hall. Schon, D. (1987) Educating the Reflective Practitioner, Jossey Bass, San Francisco. Skilton, K.J. (2011) Involving Experts by Experience in Assessing Students’ Readiness to Practise: The Value of Experiential Learning in Student Reflection and Preparation for Practice, Social Work Education, 30(3), pp. 299-311

Thanks! Any questions? You can contact us at: E.Skoura-Kirk@kent.ac.uk