Evaluating Learning Using Vignettes Gillian MacIntyre Pam Green Lister Glasgow School of Social Work

Introduction  Our use of vignettes was:  Based on pilot work in Scotland  Evaluation of the social work degree qualification in England

Why use vignettes?  Vignettes are used to test how students’ demonstrate the necessary skills, knowledge and values informed by research  The use of vignettes in previous research

Developing the vignettes  Two vignettes  Represented children and families, adult services  Consultation with service user and carer advisory group  Students to provide written answers at time one and time two

Development of the vignettes (2)  Vignettes were designed to “test” certain areas  What are the most significant factors for you in this situation? (knowledge)  How might you explain what is going on in this situation? (application of knowledge)  What should happen now in this situation? (Assessment/ analysis)  As a social worker what would you do in this situation? (Action/ Skills)  Why would you do this? (linking theory to practice)

Data Collection  Pilot response rates  Administered to PG students prior to going out on placement and again after placement  22 students, time one  20 students, time two  14 direct comparisons at time one and time two  DH Evaluation response rates  Administered at start and end of programme  222 students, time one  131 students, time two  79 direct comparisons at time one and time two

Developing a Framework for Analysis of vignettes  Development of proforma and coding form  Refined using pilot data at a training day  Content of framework for analysis  Scale (emphasised - not mentioned)  Extent to which the account is descriptive, analytic or reflective

Criticality  Describing: answer focuses on stating what is happening and what practical tasks are necessary without giving any reason or undertaking them  Analytic: speculates on what is happening and why; no reason for their relevance is offered  Reflection: draws on theories and knowledge to explain what is happening; consequences of any intervention is discussed

Themes from vignette analysis  Attention to process of relationships  Non-pathologising/ pathologising continuum  Emphasis on helping  Task/ service oriented  Language  Understanding social work role  Use of theory  Personal/ situation continuum  Awareness of social/ structural/ political issues  Attention to risk and prioritising  Added at time two - service user and carer involvement

Changes in students’ approach to vignette one Not at allSome evidence Very % Time One % Time Two % Time One % Time Two % Time One % Time Two N Descriptive Analytic Reflective

Changes in approach to vignette one  Time one:  33 % of answers mentioned social exclusion or normalisation  68% mentioned independence and ascertaining David’s wants  80% made no mention of establishing a relationship  Time two:  78% mentioned social exclusion or normalisation  Little change in mention of establishing a relationship

Example of change between time one and time two  Theme: emphasis on helping  T1 - rush to resolve problems and helping, less attention to wider context, less recognition of potential complexities  T2 - shift in students’ emphasis in recognising potential impact of learning disabilities, recognition of complexity of situations

Discussion  Evidence of change over time in development of skills  Movement from reactive helper to a more discerning, reflective practitioner  Shown by: more consideration to wider context, greater appreciation of implications of intervention, more awareness of policy and legislation

Reflections: Advantages  Enables research to be carried out with relatively large numbers of students  Allows students’ to reflect on their learning  Less intrusive than observing practice  Thought-provoking exercise, cant get enough practice, needs more time to do it justice  I found the task helpful for myself to give me a set time to think about the issues and to come up with ideas for intervention

Reflections: Challenges  Time consuming for students  Attrition rates  Inter-rater reliability  Subjectivity and complexity in terms of isolating and attributing factors that contribute to learning

Considerations for future research  Incorporation of self-evaluation  Service user and carer involvement Further discussion in workshop this afternoon…!