1. Introduction to Evaluation
Simon Bishop

2. Why evaluate?
Previous evaluations?
Theory based evaluation
Methods

3. Why evaluation? 3 good reasons
To evidence progress towards desired outcomes
To provide feedback to guide implementation
To test and develop existing ideas/knowledge/theories
(Kubish et al 1998)
+ Many other ‘less good’ ones!

4. Evaluations of Integrated Care
Torbay Evaluation Kings Fund (2011)
Longitudinal action research case study
Qualitative and quantitative measures of process and some broad outcomes
Overview of long term change
Davey et al (2005)
Comparative study of co-located and non co-located teams
Standardized interviews
Identify contextual factors that impact on team development and patient/client outcomes
DoH Integrated Care Pilots - Rand / Ernst and Young - (2012)
Comparison of 16 diverse care pilots
Qualitative (‘living documents’ + Interviews) and quantitative (HES)
Process improvements, but changes for patients mixed. Outcomes difficult to unpick
Lessons from Sedgefield- Hudson (2002, 2007)
Process of front line team integration
Mixed qualitative methods
Unpicks factors contributing to
positive outcome for the teams,

5. Theory Based Evaluation
Theory on relationship between programme, outputs and outcomes
Different schools
Theory driven evaluation
Theory of change
Realist evaluation
Acknowledges complexity
Take into account context
Making theory explicit improves shared understanding
Improves transferability of findings (as looks at relationship between conditions and outcomes)

6. ‘Assumptions’ and ‘Mechanisms’
Bring to light the underlying assumptions about why it is believed that elements are linked (for example how actions linked to outcomes)
Complex initiatives likely to involve a large number of assumptions
Unpick intermediate steps in the (assumed/hypothetical) causal chain
Impact of context on assumptions needs to be considered
Help to identify links between causes and assumed effects

7. Theories of Change
Identifying long-term goals
Mapping and connecting outcomes
Identifying Assumptions
Developing Indicators
Identifying Interventions

9. Why Integrate?
What are the underlying assumptions/ mechanisms behind integrated approaches to care? What contextual factors are these assumptions dependent upon?

10. Methods of Data Collection
Quantitative data
Outcome
Hospital Episode Statistics
Health/wellbeing indicators
Surveys
Process
Meetings
Teamwork
Qualitative
Staff interviews
Ethnographic methods
Documentary evidence
Reports / structured questions
A combination of different methods usually required to understand the process and outcomes of change …. or no change