Whose outcomes are they anyway? Robin Miller Health Services Management Centre

What are we going to cover?  Background to Sandwell Integrated Support Service (SISS)  Background to the Partnership Outcomes Evaluation Tool (POET)  Bringing the two together – POET in action  Reflections on the process and the outcomes  Next steps for POET

Sandwell Integrated Support Service  Developed as a result of a long-standing wish to improve the co-ordination of services for disabled children and their families  Also hoped to address the difficulties experienced by young people with a disability as they left school  Initially championed by Children’s Trust but managed within Children’s Services

Sandwell Integrated Support Service  Early Years Support Service  Child Development Centre  Children with Disability Social Work Team  Sensory Impairment Team  Youth Services  Learning Disability Nurses and Psychology  Connexions Personal Advisors  Adult Social Work Team

Sandwell Integrated Support Service  Single Operational Manager co-funded by health, social care and education  Co-located in a refurbished base (with space for resource room and out-patients)  Managed ‘disability’ purchasing budget for children’s health and social care  Won award (APSE Public-Public Partnership) and praised in Joint Area Review report

We did have…….  ‘Single’ framework for assessment & care co- ordination with weekly referral meeting  Integrated structure with operational and professional leads identified  Monthly management meetings  Whole service meetings every 2-3 months  Accountability to Local Strategic Partnership through Management Group (which included family/carer rep)  Family Fun Days

We also had…….  Different assessment and care co-ordination process across all the teams  Some teams working with a small number of children with complex needs and others working with a larger number with less complex needs  Teams sitting in discrete groups  A management team with mixed views  Some personality clashes!  Tensions in wider partnership

The common sense of partnerships ‘To argue for the importance of partnerships is like arguing for ‘mother love and apple pie’. The notion of partnership working has an inherently positive moral feel about it and it has become almost heretical to question its integrity’. (McLaughlin, 2004: p. 103)

Six reasons why partnerships are created  Necessity – that is partnerships are mandated by law or regulation.  Asymmetry – one party wishes to exercise control over another.  Reciprocity – partners seek mutual benefit through cooperation.  Efficiency – partners may gain more efficiency through cooperation.  Stability – organisations can encounter less uncertainty through interaction.  Legitimacy – organisations may obtain or enhance their public image through cooperation. (Oliver, 1990)

National policy context (in theory) Integration Better Services? Better Outcomes? (Do they? How? For whom? In what contexts?)

The negatives of partnerships? Although collaboration may intuitively seem like a good thing:  For Alex Scott-Samuel collaboration is often better described as “putting mutual loathing aside in order to get your hands on the money” (quoted in Powell and Dowling, 2006, p.308).  For Powell and Dowling (2006, p.305) partnership working involves “the undefinable in pursuit of the unachievable”!  For Thompson and Perry (1998, p.409), “collaboration is like cottage cheese. It occasionally smells bad and separates easily.”

Lack of empirical evidence linking collaboration and service user outcomes  Despite the interest and efforts which have gone into collaborative activities internationally, there is little empirical evidence clearly linking this to improved service user outcomes.  Cameron & Lart (2003) Systematic review of 491 papers: Dearth of evidence to support the notion that joint working between the NHS and social services is ‘effective’ (see also Dowling et al, 2004; Dickinson, 2008).  Evaluations have tended to focus more on the processes of collaboration than the outcomes.

So is integration ineffective?  Lack of evidence of effectiveness doesn’t necessarily mean integration doesn’t work, it means we don’t know whether they work.  Integration is incredibly difficult to evaluate.  What outcomes should integration produce?  Schmitt (2001) undertook a systematic review of 25 years of collaborative working in the US, and concluded: ‘What is missing from these studies is the theoretical rationale for the choice of outcomes and measurement of the team characteristics that link the team intervention to improved functional outcomes’ (ibid: 53).

Different forms of outcomes  Change outcomes improvements made in physical, mental or emotional functioning. This includes improvements in symptoms of depression or anxiety that impair relationships and impede social participation, in physical functioning and in confidence and morale  Maintenance outcomes are those that prevent or delay deterioration in health, well-being or quality of life. This can include low-level activities such as living in a clean and tidy environment and having social contact.  Service process outcomes reflect the impact of the way in which services are delivered. This might include the degree to which service users are: treated as a human being; feel that their privacy and confidentiality are respected; or, treated as someone with the right to services.

Aims of POET project  Main question - do partnerships improve outcomes for service users?  Design a generic evaluation toolkit that can test this assumption  Strong theoretical underpinning

POET: Partnership Outcomes Evaluation Toolkit 1. Online staff survey - Tests the ‘health’ of the partnership - Identifies developmental and problem areas - Highlights areas for celebration - Surfaces all the underlying ‘partnership theories’. 2.User survey - Tests whether the ‘correct’ outcomes have been identified. - To what extent have these outcomes been met? - Retroductive techniques identify generative mechanisms See Dickinson (2007) for further detail

Why did we want to pilot in SISS?  Summative – enabling us to be clear about what we were hoping to achieve and how successful we had been to date  Formative – helping us to understand and make sense of our context, to highlight areas for celebration that could be built upon and identify areas that required strengthening

Agreement from management group Workshop for all staff in service On-line questionnaire Workshop with staff Family questionnaire & interviews

Personal Role  Role definition, management & development  Structure of service & how they fit in within this structure  Culture & professional identity  Communication, learning & innovation  What is positive and what should be changed

Organisational Role  Aims & objectives of service  Leadership & accountability  Ownership & trust  Push & pull  What is positive and what should be changed

Outcomes for people using service  Current outcomes and those which should be achieved in the future  Service user outcomes  Barriers to people accessing the service  Service process outcomes And a final question on ‘any concerns regarding management support’?

Family questionnaire and interviews  Evaluate the services received against key- objectives (e.g. treated with respect, services are co-ordinated, they are enabled to be ‘in control’)  What it is important to you from services (e.g. not repeating story, a welcoming building, have a lead professional to co-ordinate)  What makes it difficult to access support?  What information would you like?

What did we learn about the service?  On the whole, staff were positive about working in the integrated service and families were positive about the support received  A service ‘culture’ had not been developed (but early days) and there was evidence of a lack of ‘trust’ between disciplines  Outcome targets had not been clearly set and performance monitored  Families agreed on the whole with outcomes identified by staff

What did we learn about integration?  Staff find it hard to identify what outcomes should be delivered (beyond processes)  Families want better co-ordination and integration across all of the services that they receive  Evaluation is time consuming but does help to understand current position and set future direction  Integration is often fragile

What did I learn about ‘integrated leadership’?  Ambiguity provides an opportunity to adapt the message but also muddies vision  Improving the lives of the families is what inspires staff & overcomes barriers  Transformational leadership can be strengthened by clearer incentives and rewards  Succession planning needs to start the day that you start

Next steps for POET  Streamlined a number of process based questions and made it easier to articulate outcomes  Based on Q methodology – provide a list of outcomes that integration should achieve and ask participants to ‘sort’ these on basis of which they agree/disagree with  Process based report on how it feels to work in integrated teams and overview of local aims

References (1)  Cameron,A. & Lart,R. (2003) Factors promoting and obstacles hindering joint working: a systematic review of the research evidence. Journal of Integrated Care 11,  Dickinson H. (2008) Evaluating outcomes in health and social care. Policy Press, Bristol..  Dickinson, H., Glasby, J., Miller, R. & McCarthy, L. (2009) Whose outcomes are they anyway? Journal of Integrated Care 17,  Dowling,B., Powell,M., & Glendinning,C. (2004) Conceptualising successful partnerships. Health and Social Care in the Community 12,  McLaughlin,H. (2004) Partnerships: panacea or pretence? Journal of Interprofessional Care 18,

References (2)  Oliver,C. (1990) Determinants of inter-organisational relationships: integration and future direction. Academy of Management Review 15,  Powell,M. & Dowling,B. (2006) New Labour's partnerships: comparing conceptual models with existing forms. Social Policy and Society 5,  Schmitt,M.H. (2001) Collaboration improves the quality of care: methodological challenges and evidence from US health care research. Journal of Interprofessional Care 15,  Thomson,A.M. & Perry,J.L. (2006) Collaboration processes: inside the black box. Public Administration Review