Supervision by the Dutch Health Care Inspectorate: what are the effects on health? Sandra Oude Wesselink Erasmus Medical Center, department of Public Health 12 October 2012

2 Content  Effects of supervision  Example: supervision of care groups  Research designs  Requirements for effect research  Recommendations  Questions and discussion

3 Effects of supervision  Effects of supervision on quality of care  Structure variables: organisation of the care provider  Process variables: actions of the care provider  Outcome variables: health of the patient

4 Supervision of care groups Care groups: organisations of integrated care (diabetes care) Most care provided by GP en practice nurse Multidisciplinary teams, paid by bundled payment per patient Care standard as guideline for standard practice New system since 2007 (100 new care groups in NL) More money  more quality?

5 Supervision of care groups Supervisory activities  Visits to the care groups, followed by individual report  National report to all care groups

6 Research question What is the effect of the supervision on care groups by the Dutch Health Care Inspectorate on the quality of care? a)What is the effect of the visit? b)What is the effect of the national report?

7 Research method  Patient record research (1 pre measurement, 2 post measurements)  Structure, process and outcome variables: organisation, care provided and health patients  Intervention group: 10 visited care groups  Control group: 8 non-visited care groups Comparison: 1.Quality of visited groups vs. non-visited groups 2.Quality before and after publication of national report

8 Warning! All graphs are hypothetical and not based on results

9 Supervision of care groups

10 Supervision of care groups ReportVisits

11 Supervision of care groups VisitsReport

12 Supervision of care groups VisitsReport

13 Research designs: before and after O 1  X  O 2 O: observation (measurement) X: experiment (intervention: supervision) Observe trend in whole group  No control group available  Intervention assigned to all people in the group  Attribution of the effect to intervention uncertain

14 Research designs: randomised trial 1. O 1  X  O 2 2. O 1  –  O 2  Random assignment of groups  Not all care providers receive supervision  Attribution of effect to intervention

15 Requirements for effect research  Reliable data available  Supervision with measurable goals  Adequate timing of supervision and subsequent effect  Intervention only by Inspectorate (no other stakeholders involved)  Random assignment of intervention

16 Recommendations (1) Think about measuring the effect of supervision during the development of supervisory activities Possible to adjust (little) aspects of your supervision to make effect measurement possible For example (small adjustments):  Timeframe  Measurable aim  Random assignment of supervisory activities

17 Recommendations (2) If randomisation of supervision is not possible: stepwise implementation of the supervisory activities (stepped wedged design) From: Brown, Hofer, Johal, et al. (2008)

18 Take home message Measuring effect of supervision sounds easy…… …there are some practical problems… Nevertheless, you always have opportunities for effect measurement!

19 Questions

20 Discussion Inspectorates should aim to evaluate the effect of all their activities Random assignment of supervision is undesirable since all care providers should receive the same supervision Effect of supervisory on the structure and processes of care have no societal relevance, therefore effect evaluations should focus on outcomes (health of the patient)