Does PPI make a Difference? Measuring the Impact of Involvement Professor Jonathan Tritter All Wales Joint PPI Conference Lampeter University 27 January 2010

2Warwick Business School What are we going to talk about? Defining Public and Patient Involvement Conceptualising Public and Patient Involvement What is the impact of Public and Patient Involvement Measuring Public and Patient Involvement The way forward

3Warwick Business School Ways in which patients or clients can draw on their experience and members of the public can apply their priorities to the evaluation, development, organisation and delivery of health and care services Patients/clients as individuals Carers on behalf of others Members of communities, localities and the public What is Involvement

4Warwick Business School Involvement in decisions about treatment and care Involvement in service development Planning, prioritising and commissioning services Involvement in the evaluation of service provision Regulation and public accountability not patient satisfaction Involvement in teaching Involvement in research At all stages of the research cycle Conceptualising Involvement

5Warwick Business School Individual and Collective Involvement Writing to an Assembly Member about the availability of a particular drug or treatment Joining a support group or community organisation Proactive and Reactive Involvement Volunteering at a local hospice Taking part in a local consultation on service reconfiguration One-off or Continuous Involvement Attending an open-meeting Building relationships within a system of involvement Forms of Involvement

6Warwick Business School Direct Involvement People play a role in making decisions Indirect Involvement People are sources of information which influences decisions Vast majority of involvement is Indirect Methods to solely to collect views and experiences Decision and justification reserved and not transparent Forms of Involvement

7Warwick Business School Patients and Service Users Individual treatment or care services Informed consent Carers Co-production Efficacy of treatment or care services Promote compliance Members of the Public: Potential service users Prioritization of health and care needs Responding to the broader public health agenda Competing justifications for involvement

8Warwick Business School Legitimacy Of decision Lessens conflict and resistance to change Relevance Different kinds of questions Focus on process and experience not just outcomes Impact Efficiency and effectiveness of decision More acceptable process for patients, members of the public and professionals Supporting co-production of wellbeing and compliance with professional advice Why involve members of the public?

9Warwick Business School Involvement is about creating accountability Promoting the transparency of decision-making Distinction between democratic accountability and other forms of accountability Democratic accountability not necessarily the highest form of accountability Both challenged by apathy Both challenged by lack of representativeness Why involve members of the public?

10Warwick Business School Social goals (Beierle 1999) Increasing the expertise of participants Incorporating public values into decision making Increasing trust in a decision Improving substantive quality of decision because of ‘local’ knowledge Expertise of organisations, professionals and managers Improves decision- making processes Greater care taken with procedures and underlying arguments in cases that generate greater public participation Indirect Effects of Public Involvement

11Warwick Business School Indicators What do you measure Outcomes of involvement How does involvement effect the organisation, staff, users and the public? Impact of involvement What is changed because of involvement? Are these changes improvements? How to Measure Involvement

12Warwick Business School The purpose of involvement The purpose of specific involvement activities The purpose of an involvement system The process of involvement Who participates The intensity of involvement activities The experience of involvement Dimensions of Involvement

13Warwick Business School The outcomes of involvement Identification of issues Suggested improvements The impact of involvement The implementation of issues emerging from involvement activities The evaluation of ‘changes’ Dimensions of Involvement

14Warwick Business School Indicators need to be: Realistic Measurable Specific “Each of these constructs lends itself directly to an evaluatory question: ‘did they listen?’, ‘did I get what I wanted?’ or ‘did the service change?’” (Crawford 2003: 15) Dimensions of Involvement

15Warwick Business School The Impact of Involvement Change that is made because of involvement activities Change that is ascribed to involvement How to Measure Involvement

16Warwick Business School What about the intrinsic benefits of involvement? “such as improvements to self esteem and changes in attitude” (Crawford 2003: 9) Should we try to measure these changes? Do these changes count? How to Measure Involvement

17Warwick Business School “The idea of citizen participation is a little like eating spinach: no one is against it in principle because it is good for you.” (Arnstein, 1969: 216) So why do we need to measure PPI? Measuring Involvement

18Warwick Business School Barriers to measuring involvement are different from the barriers to doinginvolvement An ethical issue; involvement is an inherent good so there is no need to measure it Legal requirement; there is no need to evaluate something that must be done anyway Participation is a right; there is no need to evaluate a right Barriers to Evaluating Involvement

19Warwick Business School Perceived costs and opportunity costs; money spent on evaluation means less can be spent to support involvement Power differentials have to be considered in the evaluation and make evaluations more difficult and potentially risky A challenge to existing hierarchies Cultures of some organisations may obstruct evaluation Ascription of outcome to involvement may be problematic Barriers to Evaluating Involvement

20Warwick Business School Part of the work of the Four Nations PPI Group Northern Ireland Funded by the NI Department of Health, Social Services and Public Safety and the participating Trusts Working with four Trusts across Northern Ireland We conducted 51 interviews with 66 respondents Reviewed Trust documents Observed Trust events Initiated by NHS Centre for Involvement Working with: Dr Sara Wilford, Dr Jayne Taylor and Connie Lord Parallel project in England reporting in early 2010 Evaluating the Impact of Personal and Public Involvement: Piloting a PPI Evaluation Framework

21Warwick Business School Evaluating the Impact of Personal and Public Involvement Evaluation of PPI using two tools PPI Self-Assessment Completed by PPI Operational Lead Reflective Qualitative PPI Performance Management Framework Organisational level Evidence-led Fixed format

22Warwick Business School PPI Self-Assessment Exploring two examples of PPI Successful Involvement – that made a difference Unsuccessful Involvement – did not make a difference Characteristics of two examples Aims Nature of activity Participants

23Warwick Business School PPI Self-Assessment Impact Staff, service users and members of the public Evaluation of impact Reflecting and learning from the process How would it be done differently if repeated?

24Warwick Business School Different Dimensions Meeting key performance indicators supported by evidence Infrastructure Organisational processes Impact on organisational decisions Impact on stakeholders PPI Performance Management Framework

25Warwick Business School Involvement is a process not an activity Learn through doing Involvement is predicated on collective not individual benefit The centrality of the co-production of public value Changing the ‘culture’ of organisational decision-making Relationships between the users and providers of public services Creating a different form of partnership The unvoiced and uninvolved Not a response to democratic deficit Reflections

26Warwick Business School We need systematic approach to evaluating PPI To document the benefits of involvement To document the costs of involvement To be able to make a business case for PPI We need to move to Evidence Based Involvement Reflections

27Warwick Business School By post Institute of Governance and Public Management Warwick Business School University of Warwick Coventry CV4 7AL By Get in touch