1. Michelle Lefevre University of Sussex OSWE Symposium 15.5.09 Evaluating the teaching and learning of communication skills with children and young people

2. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.092 Focus of this presentation This presentation will discuss a 2 year evaluation of the approach at Sussex University to teaching MA qualifying social work students how to communicate effectively with children and young people. Qualitative and quantitative data collected at four points during the programme will be presented Implications for curriculum structure, and further data analysis and data collection will be discussed. The usefulness of vignettes as a research tool will be considered

3. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.093 Why measure the effectiveness of teaching and assessing communication skills with children? SCIE Knowledge Review (Luckock, Lefevre & Orr et al, 2006) identified what counts as effective communication in practice (‘Knowing-Being- Doing taxonomy) – so what to teach in a whole programme approach But - concerning findings … -Communication with children is obscured and marginalized in the taught curriculum -Dearth of evidence on what counts as effective methods of teaching and assessment -Pedagogical methods are posited anecdotally or theoretically, lacking hard evidence on their effectiveness -( See Luckock, Lefevre & Tanner, 2007 Lefevre et al, 2008 ).

4. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.094 What kinds of pedagogical evidence might be needed? -The norm is post-hoc studies of satisfaction with teaching – but they are subjective and do not measure the skills/knowledge which have been developed -More robust/objective approaches needed/possible? Collection of data regarding effectiveness of the teaching approaches across the whole curriculum which enhance communication skills with children Measures of student knowledge and skills pre- and post- teaching to evaluate whether the input (i.e. teaching) produces the desired outcomes (i.e. enhanced knowledge and skills in practice). -Evaluation tools which can be disseminated so building the evidence base for different approaches

5. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.095 Methodological position – c ritical realist approach (Pawson and Tilley, 1997) Important to look at what is learned, and after what interventions, not just what students think they have learned or ‘liked’ ‘Effectiveness’ is subjective and constructed, being personally and contextually-dependent. But shared understandings can be developed. The taxonomy developed through the Knowledge Review viewed not as a straightforward ‘checklist’ of requirements but as a source of possibilities which would be drawn upon depending on the given needs of the situation. Intention to find a more nuanced way of including how students can apply contexualised knowledge in situ

6. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.096 Evaluating this aspect of the Sussex MA curriculum Evaluation Question: To what extent, and in what ways, does the MA in Social Work Programme at Sussex contribute to social work students’ development of confidence and competence in communicating with children and young people? Aims to uncover which aspects of the programme have had what effects Doesn’t assume no skill at the beginning but looks at individual and collective journeys to skilled and confident practice

7. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.097 Mixed methods approach Initial Questionnaire: Collection of students’ personal characteristics which would allow for analysis on the basis of their profile, i.e. have particular kinds of student learned most/least? Ratings: Confidence in communication with children rated by students at each stage and mapped to profile. Post-hoc subjective student feedback on the aspects of the programme they perceived to facilitate their confidence and skills with children Case vignette - more ‘objective’ measure of students’ increase in ability to apply knowledge to situations (though analysis is subjective….) Interviews – looking back, what was useful? How prepared were you? (in planning)

8. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.098 Vignettes Questions divorced from context make it difficult for students to draw upon knowledge available to them in a real life scenario. Hypothetical persons or situations provide a shared ‘reality’ from which students could contextualise their responses and make decisions appropriate to the circumstances - more likely to test students’ capacity to apply knowledge in situ (Hughes,1998). They set out and hold constant the contextual framework so increasing standardisability and reliability, & provide internal validity (Finch, 1987; Schoenberg & Ravdal, 2000). Provides a more ‘objective’ measure of students’ increase in knowledge about constituents of effective communication Analysis is subjective but could be improved by co-rating.

9. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.099 Collecting data at 4 points T1 – at the beginning of the programme T2 – just prior to the students undertaking the sub-module of 3 focused sessions on communicating with children and young people T3 – just after the students completed the sub-module of 3 focused sessions on communicating with children and young people T4 – just prior to the students completing the programme Proposal now for T5 – interviews at end of NQSW year

10. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0910 Methodological challenges Student time – justifiable? A learning tool too… Dilemmas of questionnaire fatigue – does modifying vignette each time affect validity? Standardising questions would have simplified analysis but previous pilot indicated it suggested answers. Open questions provide rich data but huge resource implications for analysis – how realistic for time-strapped programmes? Possibility of mapping the development of knowledge and skills of some individuals over time – although not all have completed all 4

11. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0911 4 measures indicate students’ participation in this programme has led to increased confidence in communicating with children and young people SCALE 012345678910 % SCORING T14 12 41628204 T241915842 11.5 T3 17 6 332217 T414273623 Key findings: Subjective rating scale regarding confidence at 4 points Key findings

12. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0912 Subjective measure: the focused teaching on skills with children was felt to be useful Exercise/teaching strategy % citing this% citing this in top 5 Mean for importance Contributions from student colleagues in the group 100%92%3.75 Direct teaching input85%54%2.9 Discussion on relationships85%54%2.7 Case study exercise77%69%2.2 Tutor style (e.g. Modelling, managing group process and discussions) 77%54%3.3 Experiential exercise reflecting on what was learned in professional experience 62%38%2.6 Experiential exercise role playing being a child 54%38%3.2 Role-play exercise regarding a child client 54%46%2.8 Research findings presentation46%8%2 Key findings

13. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0913 Perceived most important elements which enhanced the students’ learning Most important elements which enhanced the students’ learning % citing this n= 22 % citing this in top 5 N=17 Mean score for importance Direct practice with children and young people in placement 91%94%1.7 Course teaching on child development82%53%3.1 Sub-module on communication skills with children and young people 68%59%3.9 Pre-course professional experience with children68%59%2.5 Other personal experience with children64%53%3.2 Input from practice assessor/on-site supervisor55%47%3.1 Key findings

14. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0914 Vignette: key qualities, knowledge & skills about which students demonstrated MOST knowledge at T4 Posi- tion Those aspects about which students demonstrated most knowledge at T4 % OF STUDENTS DEMONSTRATING KNOWLEDGE 1*Knowledge about child development95% 1*Understanding of the purpose of communication in context95% 1*Knowledge of appropriate models, approaches, methods and skills in work with children95% 1*Child-centred communication95% 1*Use of play, symbolic, creative, non-verbal and expressive techniques95% 6*Interviewing techniques91% 6*Ability to recognise and use one’s own feelings (and counter-transference) in the work91% 6*Being able to work with depth processes in the work not just surface ones91% 9*Knowledge of how (adverse) experiences affect children86% 9*Actions to promote children feeling trust and safety86% 11Eliciting and taking into account children’s views & concerns81% Key findings

15. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0915 Vignette: Aspects of Knowledge which increased the most during the programme go across all 3 domains (Knowing-Being-Doing) Interviewing techniques  71% Being able to work with depth processes in the work not just surface ones  63% Feeling comfortable to work with children’s strong feelings  48% How inherited traits, capabilities or impairments affect child’s communication  41% Being able to use a variety of tools (e.g. ecomaps)  39% Providing information and explanations  25% Child-centred communication  19% Being playful  18% Key findings

16. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0916 Why was less knowledge shown at T4 about some aspects? AOP – non-judgemental attitude  45% Respectful,  27% Considering issues of confidentiality  26% Open and honest  17% Listening  16% Understanding of how the social work role and task impacts upon communication  15% Eliciting and taking into account children’s views & concerns  15% Understanding of the purpose of communication in context  5% Going at the child’s pace  5% Actions to promote children feeling trust and safety  2% Knowledge of appropriate models, approaches, methods and skills in work with children  1% Key findings

17. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0917 Some implications for the curriculum and further research Focused skills teaching important but practice learning opportunities with children essential Pre-course, paid work experience and personal contact with children to be encouraged (prior to programme?) Not all students will respond well to every learning method so variety important Importance of more evidence on effectiveness Enhancing validity of vignette tool through co-rating Much more data to analyse Follow-up interviews to look at what happens once in practice Engaging others in evaluating their own curriculum

18. Michelle Lefevre, University of Sussex, M.Lefevre@sussex.ac.uk, presentation 15.5.0918 References Lefevre, M., Tanner, K. & Luckock, B. (2008) Developing social work students’ communication skills with children and young people: a model for the qualifying level curriculum, Child and Family Social Work 2008, 13, pp 166–176 Luckock, B., Lefevre, M., Orr, D., Tanner, K., Jones, M. & Marchant, R. (2006) Knowledge Review:Teaching Learning and Assessing Communication Skills with Children in Social Work Education. Social Care Institute for Excellence, London. Luckock, B., Lefevre, M. & Tanner, K. (2007) Teaching and learning communication with children and young people: developing the qualifying social work curriculum in a changing policy context. Child and Family Social Work, 12, 192–201.