1. What makes a difference
What makes a difference? Research on the impact of good practice and how services can achieve excellence
Donald Forrester
Professor of Child and Family Social Work
CASCADE Centre for Children’s Social Care
Cardiff University

2. Overview What IS good social work practice?
What difference does it make?
And how can we create great practice – the sort of practice that changes lives?

3. Child and family social work
Context
Practice
Outcomes

4. Roll-Out of “Hackney Model”
Overview of Studies
Complex whole system change
T1/T2 Before AND After
75 in each
Systemic Units
5 LAs
75 families
Move to MSW
Impact of feedback for individuals and service
2/300 families T1
RCT of training in MI in 1 LA
134 families T1
104 T2
MI RCT
Islington Project
LA Reform
Roll-Out of “Hackney Model”

5. Common data collection for c.500 families
Follow up with families
Observation (2nd visit : T1)
Audio recording of meeting
Interview (T1 and 20wks T2)
Outcome measures: to be discussed! Including engagement, achievement of goals and standardised instruments
Primary focus today the 134/102 from the RCT… with some info from others
In addition over 250 qualitative interviews about change in LAs
Referral and Assessment
Children in Need team
First SW visit
2nd or 3rd SW visit
SW asks parent about observation
Family enters study

6. What IS good social work practice?

7. What IS good social work practice?
Hugely neglected in social work research
But perhaps also in leadership and practice? Is there a clear vision of great practice in your organisation? Can you tell the person next to you what it is?

8. What IS good social work practice?
We need to spend far more time and effort exploring what great practice IS and…
Whether it actually makes a difference...
And then reviewing what our vision of great practice is

9. We started with Motivational Interviewing (MI)
MI is just a handy (evidence and value based) description of key elements of good practice
This allows exploration of what good practice is and issues around behaviour change in social workers and in those we work with
MI is a directive, client-centred approach to communication that attempts to elicit intrinsic motivation

10. Coding for Motivational Interviewing (MITI)
MI Treatment Integrity (MITI) rates for:
Collaboration
Autonomy
Evocation
Empathy
The first 3 are averaged for a MITI score
But what is missed by this – what else is good practice?

11. Additional skill descriptions
Purposefulness
Clarity about concerns
Focus on child

12. Our Practice Skill Coding
Collaboration
Autonomy
Evocation
Empathy
Purposefulness
Clarity about concerns
Focus on child

13. Summary of coding Very poor practice Low levels of skill
Adequate level of skill – ANCHOR POINT
Good level of skill
Excellence

14. What were skills of workers?

15. What were skills of workers? (n=134)

16. What were skills of workers? (n=134)

17. How were skills related to each other?
Guesses?
Empathy vs Focus on child?
Collaboration vs Clarity about Concerns?

18. How were skills related?
Correlations between coded skills in practice
(N=127) for all but evocation (n=101)
Evocation
Collaboration
Autonomy
Empathy
Purposefulness
Clarity of concerns
Child Focus
.813**
.519**
.733**
.307**
.471**
.300**
Sig.
<.001
.002 1
.665**
.697**
.308**
.453**
.381**
.433**
.138
.282**
.285**
.122
.001
.372**
.485**
.341**
.434**
.238**
.007
.438**
Child focus
How were skills related?

19. Relationship between skills
The “care” elements were strongly related
The elements were ALL positively related to one another
… so overall SKILL is more important than care vs control
Put another way: good workers combine care and control…

20. Does worker skill predict parental engagement?
Used Working Alliance Inventory (WAI):
Involves three elements:
Quality of relationship
Agreement on goals
Whether tasks are carried out
Used widely in counselling – linked consistently to outcomes
So do skills predict engagement at T1 – and at 20 weeks?
And if so – which ones?

21. Does worker skill predict parental engagement?
Engagement (WAI) T2
Engagement (WAI) T1
MI Spirit total
Correlation
Sig. (2-tailed) N
Empathy
Purposefulness
Clarity of concerns
Child focus
Does worker skill predict parental engagement?

22. Does worker skill predict parental engagement?
Engagement (WAI) T2
Engagement (WAI) T1
MI Spirit total
Correlation
.367**
.344**
Sig. (2-tailed)
.007
.004 N 52 70
Empathy
.357**
.303**
.002
.003 71 92
Purposefulness
.183
.174
.129
.099 91
Clarity of concerns
.263*
.225*
.028
.032
Child focus
.197
.022
.102
.839
Does worker skill predict parental engagement?

23. How did skills relate to outcomes at 20 weeks?
What ARE “outcomes” in child and family work?
What measures might we use?
What measures do YOU use?

24. Outcomes Measures: Tricky, Very Tricky
The sheer range of issues – from unborn babies to 17 year olds etc
Can we impose outcomes or should they be negotiated?
Are the measures sensitive to change?
Is change even necessarily the goal? What about maintenance or quality in its own right?

25. Outcomes Measures: What We Use
Goal Attainment Scaling (GAS)
Life Rating (10-points)
Parent rating of worker and service
Standardised instruments and z-scores:
Parental anxiety/stress (GHQ)
Child’s emotions and behaviour (SDQ)
Drink and drugs (MAP)
Social isolation (SSQ)
Family functioning (SCORE-15) (includes arguments)
DV (behavioural measures)

26. Overall how satisfied are you with your SW (T2)
Overall how satisfied are you with your SW (T2)
Rate life now
Parents score (+3 to -3)
Engagement T2 (WAI)
Pearson Correlation
.825**
.035
-.005
Sig. (2-tailed)
.000
.729
.963 N 97 98 95
Engagement T1 (WAI)
.292**
-.072
-.056
.003
.475
.583
100
102 99
MI Spirit total
.188
.171
.112
.174
.216
.426 54 53
Empathy
.247*
.135
.152
.034
.248
.197 74 75
Clarity of concerns
.172
.227
.106
.147
.051
.371 73

27. How to make sense of these findings?

28. How did skills relate to outcomes?
These relationships are relatively weak. However, they consistently point to a positive correlation between skill and outcomes – as strong as most “evidence based” methods
They DO raise some issues about engagement… does it matter?
But the link between skills and outcomes is mediated by a key variable… Which is obvious (when you think about it…)

30. 8 or more visits only Rate life now Parents score (+3 to -3)
MI Spirit total
Pearson Correlation
.216
.129
Sig. (2-tailed)
.421
.648 N 16 15
Empathy
.360
.315
.091
.153 23 22
Purposefulness
.674**
.155
.000
.491
Clarity of concerns
.368
-.059
.084
.793

31. Some research issues with implications for the findings
The quality of practice was tested in ONE interview
There will be variation between this and the quality across all the work with a family – which will reduce the statistical relationships
Direct work is only one part of good social work – work outside and assessment/ decision-making also crucial

32. Impact of good practice
We need to think carefully about the impact of practice on outcomes generally – relatively small overall…
Reasons for limited impact on outcomes:
Most families did not have big problems…
LIFE – other things happen in families
People are active agents – trying to sort out problems
How big a difference do specific interventions make?
Nonetheless – encouraging signs that where serious issues then certain skills make a big difference

33. Donald’s big ideas put him in some illustrious company (at least that’s what he tells us).
He’s never said that he thinks he is a genius, but he does often talk about himself in relation to other geniuses. One example of this is Einstein…

34. If P = R then Q I O = S
This is Donald’s equation. If I’m not mistaken this means: Outcomes = practice x time over stuff.
It’s a measure of Donald that you can have a 10 minute conversation with him about this, and come away thinking “hmm, yeah it kind of makes sense”.

35. Q I O = S If Problem = Real then uality of practice intensity utcomes
This is Donald’s equation. If I’m not mistaken this means: Outcomes = practice x time over stuff.
It’s a measure of Donald that you can have a 10 minute conversation with him about this, and come away thinking “hmm, yeah it kind of makes sense”.

36. Q I O = S If Problem = Real then uality of practice ntensity utcomes
tuff O = S
This is Donald’s equation. If I’m not mistaken this means: Outcomes = practice x time over stuff.
It’s a measure of Donald that you can have a 10 minute conversation with him about this, and come away thinking “hmm, yeah it kind of makes sense”.

37. Implications What have learnt about good practice?
What have we learnt about impact on families?
What have we learnt about how to create good practice?

38. Work in Islington… Practice without training (c.60)
Intensive 4 days plus 8 weeks of coaching (c.70)
Embedded observation and feedback (c.100)

39. What impact on practice?

40. Are there differences between LAs?

41. Are there differences between LAs?
And if there are – why?

42. Some comparisons…

43. Developing a theory about how to support excellent practice…

44. Developing a theory about how to support excellent practice…
Key: A Vision of Practice Excellence

45. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence

46. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values

47. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment

48. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice

49. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Outcomes

50. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Outcomes

51. Developing a theory about how to support excellent practice…
Areas that create practice
Key: A Vision of Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Outcomes

52. Towards practice centred social work
Practice is not primarily produced by individual workers being skilled or by training
It is produced by organisations that set themselves to produce certain types of practice
Local authorities therefore need a vision for the practice they want and then to create the organisation that delivers it

53. Towards practice centred social work
I have very few answers.. But I think the most important thing is to be asking the right questions, such as:
What IS great social work?
What difference does it make?
How can we make it happen?