2. John Bellamy & Anita Cuca
Research into Outcomes Among Users of an Integrated Family Support Service
John Bellamy & Anita Cuca

3. Presentation Outline Introduction – evaluating outcomes
Family Support Program/ Integrated Service Delivery described
Research into outcomes while people are with the Family Support Program
Research into outcomes after people have left the Family Support program
Conclusions

4. Evaluating Outcomes Among Service Users
Outcomes evaluation – a significant innovation
Outcomes defined: what has changed or been maintained for people through participation in a community services program
Why evaluate achievement of outcomes among community services programs?
Demonstrate that the program makes a difference
Better understand how it makes a difference
Identify potential service improvements.

5. Identifying Service User Outcomes
PLMs developed with staff
Client focus groups and surveys
Client Outcomes
How to measure outcomes?
Funding Bodies

6. Presentation Outline Introduction – evaluating outcomes
Family Support Program/ Integrated Service Delivery described
Research into outcomes while people are with the Family Support Program
Research into outcomes after people have left the Family Support program
Conclusions

7. Family Support Program Described
Home visiting, case-management support for vulnerable and disadvantaged families with children aged 0-17 residing in the Liverpool region.
Support is tailored to individual family needs with a focus on early intervention and crisis prevention.
Service user presenting needs: child protection, relationship breakdown, domestic violence, housing, household budgeting and financial difficulties.
Has a maximum of 85 service users at any one time.
Staffed by 10 f.t. and p.t. Family Support Workers.

8. Integrated Service Approach
Allows us to offer our service users easy access to all of our programs in order to meet their holistic needs.

9. Why Was the Integrated Approach Needed?
Five programs working in silos
but servicing the same client population
Multiple intakes and assessments for
the same client families
Evidence that integrated or holistic service models are better equipped to help vulnerable families (Scott 2008)
Preference in Anglicare for holistic service delivery. Scope for clients to receive more assistance beyond first presenting issue.
Potential savings in staff time on assessment and service duplication which could be used in direct service. Sharing of administrative loads across teams.
Potential increase in number of services and timeliness of service delivery for clients
Potential increase in levels of trust and cooperation between teams.

10. Presenting Issues Among Service Users
Household budgeting
“Budgeting has been difficult since husband's hours have been
reduced”
Parenting, Child behavioural issues
“I have learnt when I get stressed I take it out on my kids”
Partner relationship
“Stressed: having twins that are 3 years old, ex-partner whereabouts,
vigilant all the time about seeing ex-partner”

11. Presenting Issues Among Service Users
Legal, Health, Housing
“I feel stressed due to my medical issues as my pain can't be
controlled some days.”
Under-employment or unemployment
“I look for work but no luck.”
Child Protection issues
“I am involved with legal matters regarding getting my children
restored into my care.”
Social isolation
“My self-esteem is low and I feel socially isolated”; “I find it hard to
socialise with people because I don't trust them and I feel judged.”

12. Family Support & ISD - Video

13. What is the Program/ISD seeking to achieve with service users?
Service Aims and Outcomes
What is the Program/ISD seeking to achieve with service users?
Short- term
Medium-term
Long-term
Increased knowledge (eg. self-advocacy, tenancy)
Improved parenting and household management
Improved individual and family wellbeing
Reduction in stress/anxiety
Increased social inclusion and participation
Resilience, efficacy and independence
Speedy practical assistance and support
Improved tenancy literacy/ sustainable accommodation
Improved safety and family functioning
Integrated service response
Improved financial literacy

14. Presentation Outline Introduction – evaluating outcomes
Family Support Program/ Integrated Service Delivery described
Research into outcomes while people are with the Family Support Program
Research into outcomes after people have left the Family Support program
Conclusions

15. Evaluation of the ISD Approach
The Integrated Service Delivery approach was formally evaluated in 2013/14. Some 50 clients and 12 staff were surveyed.
Starting with this evaluation has led to two research exercises to measure client outcomes:
Pre and post surveys of people currently in the Family Support Program
Interviews among ex-service users who have experienced the integrated service delivery approach.

16. Pre and Post Surveys among current Family Support clients
Measuring Service User Outcomes
Pre and Post Surveys among current Family Support clients
Use of scales, often developed in the psychology field.
Scale completed on entry to the program (“Pre” survey), then again 3-9 months later, at time of case closure (“Post” survey)
Three scales to cover three key outcomes:
Financial Behaviour Scale (FBS) – Is there any measurable shift in self- reported budgeting and spending behaviour?
Parenting Sense of Competence scale (PSOC) – Is there any measurable change in sense of parenting competence among service users?
Resilience Scale for Adults (RSA) – Is there any measurable change in resilience?

17. Financial Behaviour Scale
Improved Financial Behaviour
Financial Behaviour Scale
5 items
Self-Rating scale:
‘Poor’ to ‘Excellent’

18. Parenting Sense of Competence Scale (PSOC)
Improved Parenting
Parenting Sense of Competence Scale (PSOC)
16 items, Likert scales. Eg:
“Being a parent is manageable, and any
problems are easily solved”
(Strongly agree to strongly disagree)
“Sometimes I feel like I’m not getting
anything done”
2 sub-scales:
Sense of Efficacy as a parent
Sense of Satisfaction as a parent

19. Resilience Scale for Adults (RSA)
Improved Resilience
Resilience Scale for Adults (RSA)
33 items
Semantic differential scales, eg:
I feel that my future looks:
Very promising _ _ _ _ _ Uncertain
6 sub-scales:
Perception of self
Perception of the future
Structured style
Social competence
Family cohesion
Social resources

20. Do you agree or disagree with the following statements?
Service User Satisfaction
Do you agree or disagree with the following statements?

21. Presentation Outline Introduction – evaluating outcomes
Family Support Program/ Integrated Service Delivery described
Research into outcomes while people are with the Family Support Program
Research into outcomes after people have left the Family Support program
Conclusions

22. Interviews Among Previous Clients
45 clients were identified who had been part of 2013/14 ISD Evaluation
29 interviews were obtained. Of 16 who weren’t interviewed, only 4 were refusals
Average length of time away from the program was 1 year and 10 months
Interview schedule contained common questions as well as questions tailored to the individual’s previous casework goals
In answer to the question, “Overall how have things been going for you since you left Anglicare?” – 14 were positive, 10 mixed and 5 negative.

23. Before, During and After Anglicare
Comparison with Evaluation Study Results
Before, During and After Anglicare

24. Do you continue to apply things learned at Anglicare?
Continuing to Apply Lessons Learnt
Do you continue to apply things learned at Anglicare?

25. Continuing to Meet Casework Goals

26. Common themes in interviews with ex-clients
Long-term Outcomes for Service Users
Common themes in interviews with ex-clients
Improved confidence and self esteem
Being supported
Financial help
Becoming better informed
Improved parenting and family management
“Being able to keep my children and look after them.”
“Most of all I feel more confident and stronger.”
“Emotional support to myself to keep up hope and ability to cope.”

27. Common themes in interviews with ex-clients
Long-term Outcomes for Children
Common themes in interviews with ex-clients
Positive parenting/happier home life
Referrals to services
Support
Stability/Safety
Financial or material aid
School and learning
“Having a stable and safe accommodation.”
“The kids are now regular at school.”

28. Presentation Outline Introduction – evaluating outcomes
Family Support Program/ Integrated Service Delivery described
Research into outcomes while people are with the Family Support Program
Research into outcomes after people have left the Family Support program
Conclusions

29. From interviews of previous service users:
Conclusions – Challenges for the Program
From interviews of previous service users:
Building resilience to adverse life events
Budgeting and long-term financial management
Ongoing goal achievement

30. From the pre and post surveys among current clients:
Conclusions – Outcomes Assessment
From the pre and post surveys among current clients:
Resilience, Parenting and Financial behaviour – preliminary results
Use of assessment tools

31. From interviews of previous service users:
Conclusions – Outcomes Assessment
From interviews of previous service users:
Improved confidence, self-efficacy and access to services
Improved parenting and family life
Improved stress management
Value of interviews