1. Understanding outcomes in pharmacy regulation
Andy Jaeger
General Pharmaceutical Council

2. Previously at the 2015 conference…
Developed a logic model
About to commission evaluation
Interested in both process and impact
How well is our new approach to regulating registered pharmacies working?
What are the outcomes of our approach, for pharmacies and people using pharmacy services?

3. Key insights
3,747 respondents to a survey conducted by ICF International - representing 8% of people working in or responsible for community pharmacies
Types of respondent
Pharmacists 74%, pharmacy technicians 25%
Those working in multiples 33%, in independents (<= 5 branches) 67%
Inspected 38%, not yet inspected 34% (other respondents n/a or don’t know)
Role in pharmacy: Responsible pharmacist 45%, Superintendent 10%, Business owner 8% (respondents could choose multiple categories)
Overall our approach was shown to be working well
The standards underpinning the new approach are clear for community pharmacy professionals
Inspections are working well and help to promote awareness of the standards
Actions plans are well-received and ensure improvements to patient safety
Inspection reports and ratings ensure focus on the needs of patients and users of pharmacy services
Want to draw out three particular insights which I hope will contribute to our thinking at the conference today…

4. Engagement improves understanding
Q14: The Standards for registered pharmacies are grouped under five principles. Please indicate for each of the following principles the degree to which you consider you understand and have implemented the standards.
We see a significant uplift in understanding and implementation as a results of inspection activity
Most significant difference before and after inspection relates to principle 2 on staff competence (uplift of 13 percentage points)
Anecdotal feedback suggests conversations with inspectors helps pharmacies to “get it”
Source: ICF 2015

5. People are more important than processes
Q21: Please indicate the elements of inspection that are most important for meeting and further improving standards
Base: 1423 respondents inspected since 4 November 2013
98% rate feedback from the inspector and involving the whole pharmacy team as very or somewhat important
All results in the 90% range, but we do see a difference in relative importance – 62% & 66% strongly agree that evidence gathering and reporting is important, compared to 87% and 84% strongly agreeing about the more people-focused parts of the inspection process
Source: ICF 2015

6. Public reporting will have positive effects
Q30a: How do you think publishing inspection reports and ratings could impact on pharmacies and patients?
Some anxiety in the pharmacy sector about the potential impacts of publishing reports, but a stated strategy for the GPhC to increase transparency
Increasing accountability and improving sector performance rated most highly by pharmacies
Expect a limited market effect – much more limited agreement on whether reporting will allow pharmacies to take on new responsibilities, or will increase patient choice (neither of which are stated aims for publication)
Source: ICF 2015

7. Understanding outcomes
Evaluation focused primarily on how regulation is working within pharmacies
Important also to understand wider impact of regulation
Asked pharmacies to self-assess how well they deliver key outcomes for patients
Can compare with results of same question asked of the public in earlier GPhC/Ipsos Mori survey
Asked the public (and mirrored questions for pharmacy professionals): Thinking about the last time that you asked for help or advice at the pharmacy, to what extent do you agree or disagree that
Your privacy was maintained during your discussion with the pharmacy staff
You were asked questions by the pharmacy staff to make sure they gave you the best advice
You were treated with respect by the pharmacy staff
You were given enough time to speak with someone at the pharmacy
You were able to access the pharmacy services you needed
You were given the information or advice you needed by the pharmacy staff
The pharmacy staff you spoke to were knowledgeable and experienced
The pharmacy was clean and properly maintained

8. Leads us to some interesting questions to consider:
Three outcomes - maintaining privacy, being asked questions to ensure appropriate advice is given, and being treated with respect – where patients’ positive experiences of the outcomes are significantly lower than the outcomes pharmacies think they are delivering
On being seen by an experienced and knowledgeable pharmacy team, and the pharmacy being clean and well maintained, pharmacies are considerably harder on themselves than patients are
Leads us to some interesting questions to consider:
Do professionals need to do more to bridge the gap in understanding patient experience?
Is the focus of pharmacies in the right place, and what role as a regulator can we play in helping to rebalance that?
But we should equally be asking: after more than 6,000 inspection visits using our new approach, to what extent does the GPhC think these outcomes are being achieved?
Sources: ICF 2015, Ipsos Mori 2014

9. ? Standards Standards, inspections, action planning Pharmacies
Other influences
Business environment influences, NHS contract
Pharmacists, pharmacy technicians, pharmacy staff
People using pharmacy services ?
Comparing Ipsos Mori and ICF’s findings gives us an insight into the quality of the relationship between people working in pharmacies, and people using pharmacy services
But those conversations are happening in the context of pharmacies, which are also regulated
As a regulator we have an influence on both the professionals and the places we work in, through setting standards (and other regulatory levers)
But we also know there are other influences at work, most notably the business environment the pharmacy is operating in, including the NHS contract, and a wide range of other influences on people using pharmacy services
So is there anything else we know that can add to our understanding of what’s going on?

10. 1.8 Safeguarding 2.1 Staff numbers
Standards met by pharmacies
Outcomes experienced by people
Enough time to talk to staff
1.3 Defined roles
2.3 Professional judgement
3.2 Premises protect privacy
1.8 Safeguarding
3.5 Appropriate environment
2.1 Staff numbers
Are a proxy for the outcome?
When we developed our logic model we wanted to look at the relationship between standards put in place by pharmacies, and outcomes experienced by patients – but we did that at a high level.
Recently, we’ve been working to unpack those relationships at a greater level of detail, seeking to identify those standards that must or should be in place to create the conditions in which the outcomes have the potential to be delivered by pharmacies and experienced by people using their services (taking into account all the other influences)
But that analysis also opens up a further possibility for us – in more that 6,000 inspections to date, we’ve collected evidence about whether those standards are being met in individual pharmacies
So can we use our assessment of those standards as a proxy for the outcomes?
And so we’ve run some numbers – looking at the numbers of pharmacies that have met all of the relevant standards for each of the outcomes that we asked pharmacy teams and the public about
Standards evidenced in inspection

11. ‘ All models are wrong but some are useful - George E P Box
Some important notes of caution
We’re testing a model, as much as we’re testing the results of the model
We’re using this data experimentally at this stage, to see if this approach works and what it can tell us
All of the results from inspection are indicative only, and shouldn’t be taken as a guide to how we as a whole we think the pharmacy sector is performing
The figures we’ve derived from the data are a proxy – we think that there should be a logical relationship between the standards and outcomes, but we haven’t sought to evidence the outcomes directly
And the results may only indicate that the conditions are consistently satisfied (from a regulatory point of view) for the outcomes to be achievable – and there are other factors which are going to affect whether the outcomes are actually delivered by pharmacies or experienced by patients

12. Three observations when we add in the GPhC’s own data (based on the model we developed)
There is a broad correlation here – none of these results seem particularly extreme or surprising; we need to continue to refine our model, but we seem to be heading in the right direction
Overall, our assessment on outcomes seem more closely aligned with the views of pharmacies, rather than people using pharmacy services – at some level this isn’t surprising given the constraints around the way in which our inspections are conducted (individual inspectors, engaging largely with the pharmacy team, and with limited time and opportunities to observe interactions with people using the services)
On the outcome related to time, we can see very clearly the gap that appears between the implementation of a regulator’s standards, and the actual experience of outcomes by professionals and people using services – even if the conditions for the achievement of an outcomes are in place, all the other external factors influencing the outcome seem to make a significance difference
Sources: ICF 2015, Ipsos Mori 2014, GPhC data (not published)

13. Conclusion
We can do more on privacy, asking questions and treatment with respect
Person centred care is explicit in our new draft standards for professionals and for pharmacies
We need to consider how we ensure people’s voices are heard
If outcomes are important to us, how can we improve the way they are evidenced?
And are we being ambitious enough in describing the outcomes we think that healthcare can achieve for people?
Some more general conclusions

14. email andy.jaeger@pharmacyregulation.org web pharmacyregulation.org
facebook.com/The GPhC
linkedin.com/company/general-pharmaceutical-council
References
ICF Consulting Services, Evaluating the GPhC's approach to regulating community pharmacies: Final Report to the General Pharmaceutical Council.
Ipsos Mori, Public perceptions of pharmacies.