1. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Monitoring Outcomes Doug Gosling

2. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important My messages Monitoring outcomes and the effectiveness of services is undeveloped Monitoring outcomes - individual and whole service - is not technically difficult There are complex barriers to effective monitoring There are important lessons from our wider lives!

3. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important My outcomes for today? Your immediate reaction to what I present, esp. enthusiasm, challenge? What you take back as useable ideas? How you monitoring of outcomes improves in (define period)? Understand (or be reminded about) chiasmus – fun but irrelevant?! It may be comparatively evidence light, but is intended to be idea rich

4. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Wider Life We are more experienced at buying outcome based services than we might think – though not in social care Examples Key Lessons: Trust, which grows over time Buyer has measure(s) of success

5. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Monitoring Outcomes CSIP survey (June 2006): No authorities (15) were monitoring outcomes CSIP think tanks (Autumn 2007): No authorities (30) were monitoring service outcomes Commissioning network request: limited response, still some confusion of outputs and outcomes

6. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Monitoring outcomes: the essentials Agree outcome(s) expected: service and individual Identify what shows they have been achieved Collect information to measure achievement - individual Collate individual information to obtain measure of service effectiveness USE THE RESULTS TO ACHIEVE CHANGE

7. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Monitoring outcomes: Connie Outcome: Wears her hearing aid without reminder Measure: Whether staff have had to help or remind her Information: Daily log records and reviews, where progress and achievement over period is recorded BUT, given her forgetfulness, will she ever acquire the skill and/or do it unprompted? (In which case, does it remain an output task: insert Connie’s hearing aid?)

8. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Collating outcome measures Representation of an individual's progress on several fronts, e.g. the outcome star Collating measures from several service users to monitor the effectiveness of the service

9. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Example of star Ref: http://www.mungos.org/about/Training%20&%20Consultancy.htmhttp://www.mungos.org/about/Training%20&%20Consultancy.htm And The London Housing Foundation and Triangle Consulting, via www.lhf.org.ukwww.lhf.org.uk

10. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Suggested Benefits For clients More accurate assessment of their progress Validates clients journey of recovery Client centred tool For staff Identifies appropriate interventions Guides the work Improves confidence For organisations Better assessment leads to better outcomes Improved effectiveness Demonstrability of results

11. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Service effectiveness All service users have outcomes, but variable number and focus Progress monitored on simple scale: 0= no progress 1= progress, less than 50% 2= progress above 50 but not complete 3= full achievement Effectiveness score and profile

12. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Service Effectiveness Service user Individual Outcomes Achievement Profile TOTAL SCORE MAX SCORE EFFECT’ NESS (%) 123456 Total 0 Total 1 Total 2 Total 3 1 2 3 4 TOTAL FOR SERVICE

13. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Service Effectiveness - example Service user Individual Outcomes Achievement Profile TOTAL SCORE MAX SCORE EFFECT’ NESS (%) 123456 Total 0 Total 1 Total 2 Total 3 133333-000515 100 20123--111161250 30303--200261250 4010200412031817 TOTAL FOR SERVICE 7238305753

14. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Barriers to effective monitoring? Contracts not based on purchasing outcomes, accepted across the whole system Care management – focus on outcomes User/informal carers – are they involved enough in identifying and agreeing individual outcomes? Contract monitoring resources – numbers and competences? Integration of different levels of outcomes, especially timescale and strategic – individual Knowledge barriers – cause and effect, satisfaction and effectiveness, outputs and outcomes OVERALL SHIFT IN CULTURE AND PRACTICE

15. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Remember the lessons from wider life Mutual trust Measure(s) of whether you have got what you paid for The supplier expects you to check you have a satisfactory outcome

16. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Have I been effective? Immediate – have more than 75% of you rated me X or higher on all scales? (Is this actually satisfaction?) Have you got greater knowledge and/or confidence about monitoring outcomes? How many of you? How much greater? How much have the systems you work with moved on in say six to twelve months? Are you monitoring service effectiveness? Sewing ideas: germination and growth are influenced by a multitude of other factors.

17. dugosling@aol.com 10 April 2008 20 May 2008 Measuring what’s important Finally: chiasmus You must measure what is important You must not make important what you can measure