Resource 1. Involving and engaging the right stakeholders. Assess Plan Do Review Resource 1. Involving and engaging the right stakeholders. It is important to consider who will be interested in this service, evaluation or its findings. Engaging and involving all the right stakeholders right at the beginning is an important step not only in developing your service but also in ensuring a good evaluation. Stakeholders will be any one affected by the service or service change and this will include the users, providers and commissioners of the planned services. Engage all key stakeholders, including the patients and their carers, in the design, delivery and dissemination of the evaluation Who are my stakeholders? You may have already completed a stakeholder analysis as a part of your service planning, however if you have not completed one already then we recommend that you conduct a stakeholder analysis. This is ideally done as a group and could use existing forums such as the project team, steering group or advisory group. Don’t forget to involve your local and organisational experts in patient and public involvement, and your equalities and communications leads to help you. Once you have conducted the stakeholder analysis this can then be used to inform: Who to involve in the evaluation What expertise and people are available to support the evaluation (including expertise in data collection, data analysis, communication, patient and public involvement) Your project, evaluation and communication plans N.B. Make sure you consider the needs of those that are seldom heard and are vulnerable. If you have not already, consider completing an equalities impact assessment for the planned service or service change. NOTE 1: Could read ‘and your equalities and communications leads…’ I like the N.B. and also the link to INVOLVE. The NHS Institutes tool, part of its quality improvement tools, http://www.institute.nhs.uk/quality_and_service_improvement_tools/quality_and_service_improvement_tools/stakeholder_analysis.html will help you to conduct a stakeholder analysis. For more information about involving patients and the public then INVOLVE www.invo.org.uk, who are funded by the National Institute for Health Research (NIHR) to support public involvement in NHS, public health and social care research, are a useful resource. 

Resource 2. Understanding your service – Context Assess Plan Do Review Resource 2. Understanding your service – Context When considering your evaluation you first need to understand your service in terms of the outcomes you want the service to achieve and for whom (what need are you trying to address?), what will deliver these changes and in what context. The following are aspects of the service that are useful to consider when planning your evaluation. Aspect Description Implication Purpose What is the purpose of the service? What need is it addressing? What outcomes will it achieve? What is the purpose of the evaluation? How will it be used? Who is your audience? It is important to understand the purpose of your service (i.e. what changes it intends to make – outcomes – and how it intends to do this) and the evaluation (i.e. what answer does it need to address as this will impact on the type of evaluation you need). Evidence base What is the evidence base for the planned service and associated outcomes? What is the strength of evidence? How have similar services previously been evaluated? Understanding the evidence base for your planned service can help inform your evaluation approach and methods: from understanding the strength of the evidence available i.e. if well evidenced then you might focus on the implementation (process) evaluation; to identifying appropriate outcomes and outcome measures; to looking at how others have evaluated similar schemes before. Stage of development Is it new? Has it been in place for a while? Understanding the stage of development of your service will also impact on the type of evaluation you need to conduct. If it is new and being developed you will want to take a more formative (improvement) approach. Level of complexity and context Is it a single change? Are there multiple components? What is the context/environment in which it is working? Understanding how complex your service is and the context in which it is operating is also important. How many services do you know trying to reduce unplanned admissions? Timescales What are the timescales? Is it a one year or five year pilot? Timescales will impact on what you are able to evaluate and how, i.e. if a long term investment you are likely to be able to not only look at implementation (formative/process evaluation) but also outcomes.

Resource 2: Understanding your service – Theory of change Assess Plan Do Review Inputs Activities Outputs Intermediate Outcomes Impact Resource 2: Understanding your service – Theory of change A useful process for planning your evaluation is to develop a “theory of change” for your service. This can be useful way of articulating and providing a visual representation of the links between the various activities of service and how this will lead to the long term outcomes it is trying to achieve. To develop your “theory of change” working with your key stakeholders, some flip chart paper, pens and post-it notes consider the following questions1: Who the service is for? i.e. your case for change will usually set out the population group, their needs and characteristics, the problem it is trying to address What are the long – term outcomes you want to achieve? Then working backwards What are the intermediate outcomes (short and medium term) that will lead to these long term outcomes? What are the activities that the service will undertake to deliver these intermediate outcomes? What evidence (from research and local learning) is available to inform and support the links between activities and outcomes? i.e. what are your assumptions based on the evidence, expertise and learning? What other factors need to be in place to enable this service to work? i.e. what are your enablers? NPC guide to developing Theory of Change1 - http://www.thinknpc.org/publications/theory-of-change/ Kellogg Logic Model Development Guide: http://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide

Assess Plan Do Review Resource 2: Understanding your service – Accessing and reviewing the evidence base It is important to consider what evidence is available to inform your planning and decision making in terms of the service as well as the evaluation. You will be used to using a broad range of evidence from multiple sources including needs assessments, public health and performance data, evidence from research and best practice as well as expertise and local learning. This evidence can be used to inform your evaluation, help identify the outcomes you hope to achieve and the activities (processes) and outputs that will deliver these outcomes. Understand what evidence is already available to inform not only your service design, but the type and level of evaluation you need. Our evidence toolkit http://evidence.apcrc.nhs.uk can help you with accessing the best available published evidence from research, evaluation and the grey literature. How does this help me? Understanding your service, the context in which it is operating, providing a visual representation of your service and understanding the evidence and theory behind it can help you to identify what your should focus your evaluation on and the type of evaluation you might undertake. The diagram below tries to outline how this helps.