What makes a difference What makes a difference? Research on the impact of good practice and how services can achieve excellence Donald Forrester Professor of Child and Family Social Work CASCADE Centre for Children’s Social Care Cardiff University forresterd@cardiff.ac.uk

Overview What IS good social work practice? What difference does it make? And how can we create great practice – the sort of practice that changes lives?

Child and family social work Context Practice Outcomes

Roll-Out of “Hackney Model” Overview of Studies Complex whole system change T1/T2 Before AND After 75 in each Systemic Units 5 LAs 75 families Move to MSW Impact of feedback for individuals and service 2/300 families T1 RCT of training in MI in 1 LA 134 families T1 104 T2 MI RCT Islington Project LA Reform Roll-Out of “Hackney Model”

Common data collection for c.500 families Follow up with families Observation (2nd visit : T1) Audio recording of meeting Interview (T1 and 20wks T2) Outcome measures: to be discussed! Including engagement, achievement of goals and standardised instruments Primary focus today the 134/102 from the RCT… with some info from others In addition over 250 qualitative interviews about change in LAs Referral and Assessment Children in Need team First SW visit 2nd or 3rd SW visit SW asks parent about observation Family enters study

What IS good social work practice?

What IS good social work practice? Hugely neglected in social work research But perhaps also in leadership and practice? Is there a clear vision of great practice in your organisation? Can you tell the person next to you what it is?

What IS good social work practice? We need to spend far more time and effort exploring what great practice IS and… Whether it actually makes a difference... And then reviewing what our vision of great practice is

We started with Motivational Interviewing (MI) MI is just a handy (evidence and value based) description of key elements of good practice This allows exploration of what good practice is and issues around behaviour change in social workers and in those we work with MI is a directive, client-centred approach to communication that attempts to elicit intrinsic motivation

Coding for Motivational Interviewing (MITI) MI Treatment Integrity (MITI) rates for: Collaboration Autonomy Evocation Empathy The first 3 are averaged for a MITI score But what is missed by this – what else is good practice?

Additional skill descriptions Purposefulness Clarity about concerns Focus on child

Our Practice Skill Coding Collaboration Autonomy Evocation Empathy Purposefulness Clarity about concerns Focus on child

Summary of coding Very poor practice Low levels of skill Adequate level of skill – ANCHOR POINT Good level of skill Excellence

What were skills of workers?

What were skills of workers? (n=134)

What were skills of workers? (n=134)

How were skills related to each other? Guesses? Empathy vs Focus on child? Collaboration vs Clarity about Concerns?

How were skills related? Correlations between coded skills in practice (N=127) for all but evocation (n=101) Evocation Collaboration Autonomy Empathy Purposefulness Clarity of concerns Child Focus .813** .519** .733** .307** .471** .300** Sig. <.001 .002 1 .665** .697** .308** .453** .381** .433** .138 .282** .285** .122 .001 .372** .485** .341** .434** .238** .007 .438** Child focus How were skills related?

Relationship between skills The “care” elements were strongly related The elements were ALL positively related to one another … so overall SKILL is more important than care vs control Put another way: good workers combine care and control…

Does worker skill predict parental engagement? Used Working Alliance Inventory (WAI): Involves three elements: Quality of relationship Agreement on goals Whether tasks are carried out Used widely in counselling – linked consistently to outcomes So do skills predict engagement at T1 – and at 20 weeks? And if so – which ones?

Does worker skill predict parental engagement?   Engagement (WAI) T2 Engagement (WAI) T1 MI Spirit total Correlation Sig. (2-tailed) N Empathy Purposefulness Clarity of concerns Child focus Does worker skill predict parental engagement?

Does worker skill predict parental engagement?   Engagement (WAI) T2 Engagement (WAI) T1 MI Spirit total Correlation .367** .344** Sig. (2-tailed) .007 .004 N 52 70 Empathy .357** .303** .002 .003 71 92 Purposefulness .183 .174 .129 .099 91 Clarity of concerns .263* .225* .028 .032 Child focus .197 .022 .102 .839 Does worker skill predict parental engagement?

How did skills relate to outcomes at 20 weeks? What ARE “outcomes” in child and family work? What measures might we use? What measures do YOU use?

Outcomes Measures: Tricky, Very Tricky The sheer range of issues – from unborn babies to 17 year olds etc Can we impose outcomes or should they be negotiated? Are the measures sensitive to change? Is change even necessarily the goal? What about maintenance or quality in its own right?

Outcomes Measures: What We Use Goal Attainment Scaling (GAS) Life Rating (10-points) Parent rating of worker and service Standardised instruments and z-scores: Parental anxiety/stress (GHQ) Child’s emotions and behaviour (SDQ) Drink and drugs (MAP) Social isolation (SSQ) Family functioning (SCORE-15) (includes arguments) DV (behavioural measures)

Overall how satisfied are you with your SW (T2)   Overall how satisfied are you with your SW (T2) Rate life now Parents score (+3 to -3) Engagement T2 (WAI) Pearson Correlation .825** .035 -.005 Sig. (2-tailed) .000 .729 .963 N 97 98 95 Engagement T1 (WAI) .292** -.072 -.056 .003 .475 .583 100 102 99 MI Spirit total .188 .171 .112 .174 .216 .426 54 53 Empathy .247* .135 .152 .034 .248 .197 74 75 Clarity of concerns .172 .227 .106 .147 .051 .371 73

How to make sense of these findings?

How did skills relate to outcomes? These relationships are relatively weak. However, they consistently point to a positive correlation between skill and outcomes – as strong as most “evidence based” methods They DO raise some issues about engagement… does it matter? But the link between skills and outcomes is mediated by a key variable… Which is obvious (when you think about it…)

8 or more visits only Rate life now Parents score (+3 to -3) MI Spirit total Pearson Correlation .216 .129 Sig. (2-tailed) .421 .648 N 16 15 Empathy .360 .315 .091 .153 23 22 Purposefulness .674** .155 .000 .491 Clarity of concerns .368 -.059 .084 .793

Some research issues with implications for the findings The quality of practice was tested in ONE interview There will be variation between this and the quality across all the work with a family – which will reduce the statistical relationships Direct work is only one part of good social work – work outside and assessment/ decision-making also crucial

Impact of good practice We need to think carefully about the impact of practice on outcomes generally – relatively small overall… Reasons for limited impact on outcomes: Most families did not have big problems… LIFE – other things happen in families People are active agents – trying to sort out problems How big a difference do specific interventions make? Nonetheless – encouraging signs that where serious issues then certain skills make a big difference

Donald’s big ideas put him in some illustrious company (at least that’s what he tells us). He’s never said that he thinks he is a genius, but he does often talk about himself in relation to other geniuses. One example of this is Einstein…

If P = R then Q I O = S This is Donald’s equation. If I’m not mistaken this means: Outcomes = practice x time over stuff. It’s a measure of Donald that you can have a 10 minute conversation with him about this, and come away thinking “hmm, yeah it kind of makes sense”.

Q I O = S If Problem = Real then uality of practice intensity utcomes This is Donald’s equation. If I’m not mistaken this means: Outcomes = practice x time over stuff. It’s a measure of Donald that you can have a 10 minute conversation with him about this, and come away thinking “hmm, yeah it kind of makes sense”.

Q I O = S If Problem = Real then uality of practice ntensity utcomes tuff O = S This is Donald’s equation. If I’m not mistaken this means: Outcomes = practice x time over stuff. It’s a measure of Donald that you can have a 10 minute conversation with him about this, and come away thinking “hmm, yeah it kind of makes sense”.

Implications What have learnt about good practice? What have we learnt about impact on families? What have we learnt about how to create good practice?

Work in Islington… Practice without training (c.60) Intensive 4 days plus 8 weeks of coaching (c.70) Embedded observation and feedback (c.100)

What impact on practice?

Are there differences between LAs?

Are there differences between LAs? And if there are – why?

Some comparisons…

Developing a theory about how to support excellent practice…

Developing a theory about how to support excellent practice… Key: A Vision of Practice Excellence

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence Training Supervision Selection Meetings Etc… Values

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence Training Supervision Selection Meetings Etc… Values Effective Initial Assessment

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence Training Supervision Selection Meetings Etc… Values Effective Initial Assessment Quality of Practice

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence Training Supervision Selection Meetings Etc… Values Effective Initial Assessment Quality of Practice Outcomes

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence Training Supervision Selection Meetings Etc… Values Effective Initial Assessment Quality of Practice Outcomes

Developing a theory about how to support excellent practice… Areas that create practice Key: A Vision of Practice Excellence Training Supervision Selection Meetings Etc… Values Effective Initial Assessment Quality of Practice Outcomes

Towards practice centred social work Practice is not primarily produced by individual workers being skilled or by training It is produced by organisations that set themselves to produce certain types of practice Local authorities therefore need a vision for the practice they want and then to create the organisation that delivers it

Towards practice centred social work I have very few answers.. But I think the most important thing is to be asking the right questions, such as: What IS great social work? What difference does it make? How can we make it happen?