1. A Beano Guide to extracting outcomes data from your information system
Alastair Macdonald
Clinical advisor
South London & Maudsley NHS Foundation Trust

2. Extract from the Beano glossary:
“Easy Peasy” adj. Hopelessly impossible and doomed to fail

3. Things to discuss What you want out When you want it out
What you will do with it when (if) you get it out
How do you set about getting a) and b)
How you achieve c) Focus here is on aggregate data rather than for individual service users

4. Things to discuss Start here What you want out When you want it out
What you will do with it when (if) you get it out
How do you set about getting a) and b)
How you achieve c) Focus here is on aggregate data rather than for individual service users
Start here

5. What you want to do with the data
mainly
What you want to do with the data /
Feed beasts
MHMDS
Commissioners
Managers (e.g. For appraisal or dealing with deficits)
Encourage reflective practice
Teams
Individual consultants

6. What you want to do with the data
mainly
What you want to do with the data /
Feed beasts
MHMDS
Commissioners
Managers (e.g. For appraisal or dealing with deficits)
Encourage reflective practice
Teams
Individual clinicians
Rest of this talk

7. What you want out
Outcomes data!

8. What you want out
Outcomes data! ... er?

9. Definitions of outcome
The attributable effect of an intervention (or its lack) on a previous health state.
UK Department of Health (1994)
a change in the health of an individual, group of people or population which is attributable to an intervention or series of interventions.
NSW Health Department (1992)

10. of clinical outcomes (Broadbent, 2001) All necessary: none sufficient
3 dimensions
of clinical outcomes (Broadbent, 2001)
All necessary: none sufficient
Intervention
Change in Health Status
over a period of time
Condition/ Context

11. of clinical outcomes (Broadbent, 2001) All necessary: none sufficient
3 dimensions
of clinical outcomes (Broadbent, 2001)
All necessary: none sufficient
Intervention
period of time
Change in Health Status
over a period of time
Condition/ Context

12. of clinical outcomes (Broadbent, 2001) All necessary: none sufficient
3 dimensions
of clinical outcomes (Broadbent, 2001)
All necessary: none sufficient
Intervention
Condition
Change in Health Status
over a period of time
Condition/ Context

13. of clinical outcomes (Broadbent, 2001) All necessary: none sufficient
3 dimensions
of clinical outcomes (Broadbent, 2001)
All necessary: none sufficient
Intervention
Context
Change in Health Status
over a period of time
Condition/ Context

14. of clinical outcomes (Broadbent, 2001) All necessary: none sufficient
3 dimensions
of clinical outcomes (Broadbent, 2001)
All necessary: none sufficient
Intervention
Change
Change in Health Status
over a period of time
Condition/ Context

15. of clinical outcomes (Broadbent, 2001) All necessary: none sufficient
3 dimensions
of clinical outcomes (Broadbent, 2001)
All necessary: none sufficient
Intervention
Intervention
Change in Health Status
over a period of time
Condition/ Context

16. Choosing a period of time of interest
Arbitrary e.g. Financial year
Period bounded by formally defined events such CPA reviews or cluster reviews
“Episode” of care with a particular team or unit
“Treatment episode” e.g. of psychological therapy
“Inpatient Episode” of continuous care of the same type e.g. Inpatient admission involving several wards
“Combisode”- episode of care by specifically linked teams (e.g. Inpatient ward and HTT)
“Spell” of continuous contact with any Trust team, unit or service

17. Choosing a period of time of interest
Arbitrary e.g. Financial year
Period bounded by formally defined events such CPA reviews or cluster reviews
“Episode” of care with a particular team or unit
“Treatment episode” e.g. Of psychological therapy
“Inpatient Episode” of continuous care of the same type e.g. Inpatient admission involving several wards
“Combisode”- episode of care by specifically linked teams (e.g. Inpatient ward and HTT)
“Spell” of continuous contact with any Trust team, unit or service
Rest of this talk

18. Period of interest=“Scenario”

19. Scenario data items Overall period start date
Overall period end date (blank if still active)
Description of secnario e.g. “Team/ward episode”, “Inpatient episode”
Identifiers of contributing teams
(if only two teams (“Combisode”), include the start date and end date for each team so that the proportional contribution of each team to outcome can be calculated)

20. “Combisodes” can be used to deal with service redisorganisation...
Community team 1 is closed and replaced by two new teams (a and b)
Some continuing cases from 1 go to a, some to b.
Create new episodes where overall start date is with 1 and end date is with a, and new episodes where overall start date is with 1 and end date is with b

21. Condition i.e. What’s wrong with the patient Problems with diagnosis
Two different meanings
shorthand for a hypothesis that attempts to explain the difficulties that the individual patient is experiencing. This hypothesis includes the likely course of the condition, and whether or not it might respond to interventions. Any clinician carrying out an intervention must have a hypothesis so must be capable of this sort of diagnosis!
A rule-based administrative label applied to groups of patients with similar requirements from the organisation. Not all clinicians are comfortable with these rules and labels.

22. So a patient with a long-term disorder such as schizophrenia (perhaps the most useful label for administration) may, if presenting with new affective symptoms, be given a diagnosis of a depressive disorder (which will be the most useful for the patient as it will determine the focus of treatment).

23. Which diagnosis?
For outcomes we use the latest diagnosis within the period of interest (diagnosis type 1) AND
The latest-ever diagnosis (found to be closest to validated diagnosis type 2)

24. Context Age Gender Ethnicity Deprivation index of home address
Employment1
Accommodation1
Marital status1
1 Not accurate in our data

25. Change in health status (Outcome)
% of spell

26. Change in health status (Outcome)
Measurement at start and end (CROM or PROM) of period of interest

27. Change in health status (Outcome)
Measurement at start and end (CROM or PROM) of period of interest
What if the measurements are not timely? RULES

28. Our current rules for attaching outcomes measurements to simple ward/team episodes
Ratings for simple team or ward episodes must come from the same team or ward.
The first rating is the first available between start of episode and 14 days after the end of the episode. If two such ratings are carried out on the same day then that with the highest cn_doc_id (i.e. the latest entered) is chosen.
The last rating is the last between 14 days after the end of episode and the first rating.
If two ratings straddle the end of the episode then the closest to the end is chosen, irrespective of whether it is before or after the end. If both are equidistant from the end of the episode then the one inside the episode is chosen.
First and last ratings cannot be on the same day, even if rated by different people. If there are only two ratings in an episode and they are on the same day the one with the highest cn_doc_id (i.e. the latest entered) is chosen as the first rating and the last rating is null.

29. Intervention We have little reliable structured intervention data
Even frequency of face-to-face contact is poorly recorded
The only way of telling that contacts or interventions have occurred is study of text items (progress notes, letters and discharge summaries)
This is being automated using natural language processing (the other NLP).

30. Our current position
Since 2011 we have had a SQL extract of all simple ward/team episodes from teams that use HoNOS or HoNOS65+
This does not allow analysis of inpatient episodes, trust spells or “combisodes”, nor of teams using any other measure
For 2015/16 commissioners set a CQUIN which included reporting on % episodes with first and last HoNOS-LD
This inspired (ha!) the Trust to fund a new SQL extract (as described here) that would be as future-proof as possible
Work is in progress.

31. episode table from referrals and discharges
Allocation of any outcomes ratings
Derive team and ward
episode table from referrals and discharges
Simple team/ward row?
First and last ratings from same team Y
Get context data for team and
ward episodes N
Trust Spell row?
First and last ratings from any team Y
Add Scenario rows
The new extracts for different purposes N
Inpatient episodes
Combisodes
Trust spells
Inpatient episode row?
First and last ratings from any team Y
Add context data from team and
ward episodes to scenario rows N
Nominated combisode row?
First and last rating from either team
Full context table with data Y

32. Each month the full context Table will be created in SQL
Each month certain key outcomes
ratings (first and last) will be
allocated by SQL procedures to designated rows in the full context table, creating the full outcomes table
Each month SQL queries will be run on the full outcomes table and results loaded into SPSS tables- one for each main outcome measure
Each month SPSS syntax is run on these tables to generate variables for analysis (e.g. Categorical change)
Whenever required SPSS syntax is run on these tables to generate reports and graphics for presentation to specific clinical teams or other audience

33. What you need (just to get data out)
A SQL analyst dedicated full-time for the duration*
SQL analyst available to maintain system once it is built and tested
An experienced IT person who is able to project manage
Access to reliable IT and database management support*
An experienced clinician interested in outcomes to make up rules (and, if there is time, to consult on these) and carry out analysis of results*
Accommodation*
Coffee (lots)**
SPSS or other data analysis program*
* We have these- the other items are sorely missed at present

34. How long and what does it cost?
We are doing this for £40k in 4 months
To do it properly I now think £100k and 8 months.
Ho Hum