Using information from users of services in regulation Dr Alex Mears

The Care Quality Commission The Care Quality Commission is the regulator for England Created 1 st April 2009 (just over a year old) Remit covers all aspects of health (NHS and privately owned) and adult social care Does not include medical or clinical personnel (other bodies do that) Registration-based model- initial registration then ongoing monitoring of compliance Information is delivered to field staff through statistical risk model into the Quality and Risk Profile (QRP), indicating risk level for a number of outcomes for each service/ provider QRP used by field staff to prioritise regulatory activity QRP uses information from many sources including users of services

Information in regulation Information for regulation comes from many sources Can be numerical indicators, outcome or process, or qualitative experience information CQC views information from users of services as a crucial piece in the information jigsaw One of three key quality markers: safety, effectiveness and patient experience There is a need to show accountability in how the CQC uses information from users Programme of work to make better use of the ‘voice of the user’ Includes methodology development- driven by Intelligence Directorate

Current sources of user information Provider groupVersion 1.1Version 1.2, 1.3, etc NHS  Surveys: Survey of adult inpatients, Acute Outpatient Survey  Coded User Representative Groups (LINKs,OSC, Local Boards )  LINks annual reports  NHS Choices  Patient Opinion  Engagement forms  Data from MHAC visits In scoping….  Additional survey information  Patient views submitted from reviews of compliance (feedback mechanism)  Wider range of web 2.0 coverage  PROMs  Complaints  Ombudman resports Adult social care (still in scoping)  Coded User Representative Groups (LINKs,OSC, Local Boards )  LINks annual reports  Engagement forms  Complaints information  Potential additional information from iCAP  Web 2.0 feeds – e.g. Google maps, Yahoo local) Scoping yet to start Independent Health (still in scoping)  ISTC patient survey results  Coded User Representative Groups (LINKs,OSC, Local Boards )  LINks annual reports  NHS Choices  Patient Opinion Scoping yet to start

Additional survey sources External feeds – Web 2.0 Other sources CQC current sources CQC surveys (patient & staff) Care home inspection Health – core standards MHAC visits Voices into Action- CQC policy National Processing Centre National Contact Centre Complaints Additional external surveys ? Do we need to apply sector specific lenses? Are we able to make macro judgments with micro perspectives? Should we? How do we feedback the significance of how we use what we hear? Integrating the user voice into the regulatory model What information do we need? What information do other people have? How easily can we get others’ information? Where are the information gaps and how do we fill them? How do we weight information?

The People’s Voices programme Work programme to improve and extend how CQC makes use of information from users High priority for CQC in 2010/11 Cross- Directorate, sign up from all parts of CQC Intelligence Directorate, focus on development of analytical techniques and investigating new data sources

Intelligence directorate: Information Research and Policy  Development of best practice  Literature review  Methodology options paper  Secondary analyses of patient survey data  Periodic audit of user information sources  Developing new methodologies for use of information from users  Ongoing thematic content analysis  Early warning system/ patterns of performance  How best to feed information into regulatory activity  Identification and use of new information sources  Complaints information/ feedback  Web 2.0 feeds  PROMS  MHAC inspection programme

Secondary analyses of patient survey data Patient surveys are a key source of quantitative information from patients Cover all sectors periodically Good response rate, large volumes of data Are questions re how survey data relates to other key indicators  NHS staff survey  Mortality  Re-admission rates  HCAI infection rates  Waiting times  Inequality indicators  National Priorities & Existing Commitments IRP team will carry out secondary analyses to understand more about these relationships Key to understanding what factors are important to patients

Ongoing thematic content analysis Thematic content analysis is an accepted methodology for social science research Text is examined for key emerging themes. Raters agree on a coding frame and re-analyse Plan to develop a similar methodology for qualitative user information at CQC Methodological challenge: the information pool is changing- items become outdated and are removed, new information is added: therefore themes need to be fluid, not fixed

Early warning system/ patterns of performance Mortality outliers surveillance is accepted as a valuable early warning of potential service failure Requires triangulation Only applies to acute hospitals currently Plan to develop a system for user information, looking at patterns of performance Will use information from:  Patient surveys  Web 2.0 feeds  Other qualitative information

Feeding information into regulatory activity Currently, user information is embedded within the QRP at outcome level Possibility to extract all user information and present as separate page showing breadth of data sources Could also include media and other information about ‘experience’

Identifying and using new sources of information Use of information from users is not a static process: we need to make sure we are exploiting evolving new sources of information. The sources we are currently exploring:  Complaints information/ feedback  Web 2.0 feeds  PROMS  MHAC inspection programme

Complaints information Historic role of HC Importance as a source of information Three sources to explore  Quantitative from the IC  From the Ombudsman  Internal to CQC- feedback process

Web 2.0 feeds Development of the web Emphasis on ongoing development, interactivity and contemporaneous information sharing Examples are wikipedia and tripadviser Context for health: web 2.0 feeds for patient information  NHS choices  Patient opinion  Iwantgreatcare.com  Coverage is sporadic and potentially biased, but data is contemporaneous. Mental health under-represented.

Future issues for CQC Some areas that require consideration in future developments  Using the knowledge of the field staff, as a route to making best use of information  Disenfranchised groups- voice information over- represents the articulate middle-classes and under- represents the poor, minorities and those with English as a second language. The same groups tend so suffer disproportionately from health problems in addition  Under-represented sectors, in particular mental health  What are the issues specific to mental health that we need to be working on?

Other EPSO partners? CQC is keen to make best use of information from users Part of the information ‘jigsaw’ What are EPSO partners doing? No evidence that there has ever been cross-national comparison of approaches/ data use Proposal: to work with interested EPSO partners to gather information about what approaches exist Bring information to a subsequent EPSO meeting as a workshop

The EPSO project on user information Project objectives Obtain a pan-European view of how user information is used in regulation Form a typology of uses/ users Hold a workshop to promote information sharing and knowledge exchange

Proposed steps One option for moving forward: Gain support from interested EPSO partners to form working group Teleconference to consider best approach to data collection AM to lead on design of data collection tool? Circulation for comment and amendment Distribution of tool/ data collection (by England or other?) Analyses- compare and contrast approaches Writing of report  circulation and agreement Writing of research paper for journal submission

Lopp. Tänan teid!