RCSLT Outcomes Project TOMs CONNECT 17th November 2016 Kamini Gadhok Outcomes Project Sponsor, RCSLT CEO Kathryn Moyse Outcomes Project Officer, RCSLT Mark Bedwell Technical Partner, Different Class Solutions Ltd Dr Alison Stroud Head of Speech and Language Therapy, Aneurin Bevan Health Board

Context Challenges across healthcare, education and social care with regard to outcome measurement Use of terminology and consistency of definitions Historical focus on inputs, processes and outputs Outcome measurement not embedded - variable use of outcome measures, PROMS and PREMs Few validated outcome measures available Shift to outcomes-based commissioning in some parts of the UK Focus on national policies and frameworks promoting improvement in health outcomes

RCSLT work on developing an approach Developing consensus within the SLT profession about use of terminology: Inputs Processes Outputs Outcomes Adopting the Theory of Change framework to articulate the contribution of SLT interventions to ultimate outcomes https://www.youtube.com/watch?v=wKcxkUBrDkY

Selecting the approach May 2014 – December 2014 RCSLT Board of Trustees opted for a pragmatic approach: identifying an existing outcome measure to begin to gather consistent data for the SLT profession Commissioned a review of outcome measures used by SLTs to identify an existing tool for the data collection Developed criteria for appraisal of outcome measure tools, as suggested by members (see next slide) Appraised 60+ outcome measures, systems and frameworks against the criteria 4

RCSLT members’ ‘best fit’ criteria 5

Identifying an existing outcome measure December 2014 Therapy Outcomes Measure (TOMs) (Enderby and John, 2015) was identified as the measure most fit for purpose It was acknowledged that: The adoption of TOMs was a starting point for the profession’s journey on outcome measurement Use of TOMs by the profession is an ‘opt-in’ process TOMs would not be used as a ‘stand-alone’ option but employed alongside other measures or diagnostic assessments TOMs is not applicable across all clinical areas and settings (i.e. universal services/Public Health) (Parallel RCSLT work-streams are being established to consider how to fill these gaps)

Identifying an existing outcome measure January 2015 – March 2015 Communicated to RCSLT members that TOMs had been identified as the ‘best fit’ outcome measure for the recording reliable, comparable outcomes data across the profession, and invited SLT services to ‘opt in’ Scoped options for national data collection system RCSLT Board of Trustees decided on a stand alone model for data collection. A ‘Proof of Concept’ pilot was approved to develop an online tool using TOMs

‘Proof of Concept’ Pilot Aims To improve the collection of outcome data in order to identify the impact of therapy for individuals with speech, language, communication and swallowing needs To investigate and minimise the burden of data collection, collation and analysis on speech and language therapy service leaders and practitioners To investigate the practical challenges and benefits of benchmarking for quality assurance and identifying variation in outcomes associated with different services

‘Proof of Concept’ Pilot Aims To investigate the value of TOMs aggregated data reports in supporting service evaluation, quality assurance and business case development for individual services To evaluate the usability and value of a stand-alone TOMs web-based application for SLT services (via direct data input or data transfer from an existing system) compared to data collection and reporting systems currently in use

The ‘proof of concept’ online tool April 2015 – July 2015 Developed basic specification for the RCSLT online tool for measuring outcomes using feedback from members on data collection and reporting requirements The online tool is being developed to support practitioners with: Collecting and collating outcomes data using two methods: Direct input of TOMs data into the online tool Exporting of outcomes data from local systems to the online tool using a data transfer template Evaluating and reporting outcomes 10

The ‘proof of concept’ pilot August 2015 – present Identified SLT services with experience of using TOMs to pilot the online tool Representation from across the UK Range of clinical areas and settings Supported SLT services involved to develop relationships with key stakeholders within their organisation and with completing relevant information governance documentation and processes

Key ● Adult ● Paediatric ● Adult and Paediatric Pilot Organisations 12

Pre-intervention Post-intervention

Key challenges Two key areas of challenge Information sharing and information governance Data collection

Actions to address challenges Information sharing and information governance National stakeholder engagement (e.g. Information Governance Alliance, Information Commissioners Office, National Data Guardians) for advice and support on information governance Developing resources to support SLT services engaged in the project to navigate local information governance documentation and engage local IG leads (e.g. Information Governance Resource Pack)

Actions to address challenges Information sharing and information governance Developing resources to support SLT services with understanding legislation about information governance more broadly “What can I share? Exploding the myths about information sharing” https://www.rcslt.org/news/webinars/rcslt_webinars Webinar in collaboration with the Information Governance Alliance (IGA) and Information Commissioner’s Office (ICO)

Actions to address challenges Data collection National stakeholder engagement (e.g. NHS Digital, including UK Terminology centre) Embedding standardised clinical terminologies within the online tool (ICD10/SNOMED) Exploring different options for data export that will enhance the quality of the data and offer a more sustainable approach

Where are we now? Refining the online tool and aggregated data reports in response to feedback from SLT services in the pilot Iterative process ‘Agile’ methodology Currently, the RCSLT database contains outcomes data for 10,500+ complete episodes of care from six SLT services involved in the pilot (November 2016) Independent evaluation conducted by Bristol Speech and Language Therapy Research Unit to evaluate the “benefit vs. burden” RCSLT Board of Trustees will discuss next steps in December 2016

Thanks for listening! For more information, please contact: Kathryn Moyse RCSLT Outcomes Project Officer kathryn.moyse@rcslt.org https://www.rcslt.org/members/outcomes/RCSLT_outcomes_project 25

Reference slides Case Study - Paediatric speech and language therapy service outcomes

Change in TOMs scores between initial and final across each domain

Change in TOMs scores between initial and final across each domain

Change in TOMs scores between initial and final across multiple domains

Distribution of change in TOMs scores between initial and final across each domain

Average change in TOMs scores between initial and final across each domain