1. Peer Evaluation of the Norwegian Board of Health Supervision (Statens helsetilsyn) First of its kind. Performed by EPSO Peer Evaluation Team Norway (July 2011- Jan 2012) Presentation by Jooske Vos – member of Peer Evaluation team

2. The Norwegian Peer Evaluation- Summary How did it start? (1 slide); Interesting starting points (3 slides); How did the PE-team work- procedures, norms, standards? (4 slides); What decisions were made and why?(2 slides); Peer evaluation in relation to other forms of assessment (review, external evaluation, accreditation, certification)(1 slide); Some interesting conclusions and recommendations (3 slides) Questions. EPSO European Partnership for Supervisory Organisations in health and social care 2

3. The Norwegian Peer Evaluation- How did it start? (1) Helsetilsyn asked EPSO to organise a Peer Evaluation of its organisation (16-3-2011) EPSO agreed and formed a team of Peers from England, Wales, Denmark, Finland and the Netherlands with background assistance from France EPSO European Partnership for Supervisory Organisations in health and social care 3

4. The Norwegian Peer Evaluation- Interesting starting points (1) a. In Norway every governmental organisation is obliged to regularly perform evaluations of their own activities (section 16 of the Regulation on economical governance); a Peer evaluation was seen as part of this obligation. EPSO European Partnership for Supervisory Organisations in health and social care 4

5. The Norwegian Peer Evaluation- Interesting starting points (2) b. It was up to EPSO to: to select a team; to elaborate the approach for evaluation (respecting formal conditions budget/legal); the interpretation of national requirements in a European context. EPSO European Partnership for Supervisory Organisations in health and social care 5

6. The Norwegian Peer Evaluation- Interesting starting points (3) c. The aim was : determine if NHBS’s work can be acknowledged as good supervisory practice; Find areas for improvement; Make a basis for standard setting by EPSO; EPSO European Partnership for Supervisory Organisations in health and social care 6

7. The Norwegian Peer Evaluation - How did the team work?(1) Norms and Standards: ISO /IEC standard 1720: 1998 (general criteria for bodies performing inspection - used as starting point for standard- setting: 13 key areas to evaluate and examine were identified (procedures and outcome): Flexible standards set by Peers instead of fixed norms to comply with; EPSO European Partnership for Supervisory Organisations in health and social care 7

8. The Norwegian Peer Evaluation - How did the team work?(2) Activities : PE-team was split in 2 teams - parallel sessions; NBHS was to invite stakeholders for interviews - on PE-request ; Structured questions based on 13 key areas; flexible discussions with stakeholders; unforeseen information; conclusions and recommendations; Key and strategic documents -reviewed; Senior management meetings - observed; Management; staff and stakeholders -interviewed (incl. individual and group discussions); Samples of work were reviewed. EPSO European Partnership for Supervisory Organisations in health and social care 8

9. The Norwegian Peer Evaluation- How did the team work?(3) Findings Findings - presented and discussed with the senior management team as a factual accuracy check (before presenting the report). Final findings reported: - as conclusions and recommendations (advice to learn and to improve); - from independent peers: without recognisable sources and without reference to recognisable interviewed persons. EPSO European Partnership for Supervisory Organisations in health and social care 9

10. The Norwegian Peer Evaluation- How did the team work?(4) Final report and Follow up The final report was presented to and discussed publicly with the full staff at the central office of NBHS; No feedback was given to the interviewed stakeholders (feed back was organised by NBHS). Follow up of the Peer evaluation was intended but not realized. EPSO European Partnership for Supervisory Organisations in health and social care 10

11. The Norwegian Peer Evaluation- What decisions were made and why? (1) Main debates in the PE team were about :  Standard setting including best practises and most preferred outcome; based on discussion between Peers and research on available best practises in Europe such as: - What is a good procedure in the national Norwegian context / what should be mentioned as possible improvement? - How is the context influencing working methods and good practice? - What is proportionate and adequate action by inspectorate in this context compared to working methods in other countries? EPSO European Partnership for Supervisory Organisations in health and social care 11

12. The Norwegian Peer Evaluation- What decisions were made and why? (2)  Separation between findings (facts) and recommendations; Findings not as a judgement / verdict but as conclusion and recommendation;  No individuals mentioned in the report; ‘Absolute Privacy Guarantee ‘of all stakeholders’; to guarantee an open mind set of interviewees;  Best practises mentioned if a possible improvement could be expected;  No observations on the budget - some remarks on legal context; EPSO European Partnership for Supervisory Organisations in health and social care 12

13. The Norwegian Peer Evaluation- related to other forms of assessment (1) Main Difference with other forms of assessment such as review, external evaluation, accreditation, certification: 1. Norms set by experienced Peers based on a European context; 2. Focused on a specific regulator in its political context; 3. Main aims are to learn and improve by advice, assistance, support of Peers; Not aimed at judgement based on previously set standards; 4. Flexibility of norms –to setting and possible outcome of improvement; 5. Not about people -always about the organisation in its political context. 6. Not only about compliance to procedures but also about results and effect (outcome); EPSO European Partnership for Supervisory Organisations in health and social care 13

14. The Norwegian Peer Evaluation- Some interesting conclusions and recommendations (1) 1. CONCL 9.14 ‘Helsetilsynet should consider selecting the organisations to be reviewed to be part of a planned countrywide system including organisations that range from excellent to poor (not only the risky/poor ones -RECOM 16) Reasons : identify and share good practice as inspection task ; not only ‘bad news’ also good news to politicians; prevent game playing by mediocre health institutions; Combine legal compliance with national quality improvement agenda. EPSO European Partnership for Supervisory Organisations in health and social care 14

15. The Norwegian Peer Evaluation-Some interesting conclusions and recommendations (2) 2. Helsetilsyn needs to reflect on whether its incident investigation processes could be enhanced to allow for a greater focus on refection and learning ; RECOM 21 Helsetilsynet reviews its incident investigation processes to ensure that individuals and organisations subject to investigation are given the opportunity to reflect and learn from the process in an environment that is non -threatening. Reasons: a process of investigation that does not allow for discussion in a non-threatening environment amongst peers and without legal input has reverse effect to which it aims to achieve ( underreporting, stifling of innovation,inability or fear to make judgements / take responsibility) EPSO European Partnership for Supervisory Organisations in health and social care 15

16. The Norwegian Peer Evaluation-Some interesting conclusions and recommendations (3) 3. RECOM 9 Helsetilsynet develops a patient and public engagement strategy that sets the framework for its engagement with patients and the public to inform all aspects of its work including forward planning which includes why various topics were chosen for inclusion in the work programme Reasons: A public and patient engagement strategy can be used to ensure that Helsetilsynet benefits from the knowledge and experience of patiënts and pubic when planning scoping and undertaking its supervisory work. EPSO European Partnership for Supervisory Organisations in health and social care 16

17. Questions about ‘operation’ Peer Evaluation Norwegian Board of Health Supervison? EPSO European Partnership for Supervisory Organisations in health and social care 17

18. More information on www.epsonet.eu http://www.epsonet.eu/presentations- 2.html EPSO European Partnership for Supervisory Organisations in health and social care 18