1. IAEA International Atomic Energy Agency The IAEA Integrated Regulatory Review Service (IRRS) IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Part 1: Initial IRRS Missions Moscow, 09 - 11 December 2014 Ahmad Al Khatibeh Section Head, Regulatory Infrastructure and Transport Safety Section, Division of Radiation, Transport and Waste Safety (NSRW) IAEA

2. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 2 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Contents: Assessment of IRRS programme performance Recurring safety issues IRRS as an integrated service IRRS and the Safety Standards IRRS as peer review IRRS recommendations and suggestions IRRS reporting of good practice IRRS review completeness IRRS review consistency IRRS Analysis - Conclusions

3. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 3 Introduction Every IRRS report carries the following statement on a page all to itself: “The number of recommendations, suggestions and good practices is in no way a measure of the status of the regulatory body. Comparisons of such numbers between IRRS reports from different countries should not be attempted”. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

4. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 4 Introduction IRRS programme first implemented in 2006 and by December 2013, more than 50 missions completed. Given this milestone, a comprehensive analysis of completed IRRS mission reports from the radiation safety perspective was commissioned to identify opportunities for continued improvement of the IRRS peer review programme. The analysis report; “IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective” has been distributed at this workshop and this presentation provides a summary of the report’s key findings and proposals. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

5. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 5 Introduction The consultancy comprised experienced international experts from Canada, Finland, Nigeria, Sweden, Ukraine and USA. The outputs of the consultancy would be: an analysis report drawing conclusions in two main areas: 1. The IRRS process; analysing strengths and potential weaknesses, and 2. Safety issues notably common to the majority of Member States. and with regards to follow up missions: 1. The impact of the IRRS on improving regulatory effectiveness in MS 2. IRRS report recommendations implemented and those not, with possible reasons. 3. Possible impact on the safety standards. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

6. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 6 Introduction The consultants developed their analysis according to Terms of Reference addressing two areas: The IRRS process: Preparation for IRRS, its implementation and follow-up; proceedings of meetings, scope, consistency, reliability, whether the programme works for every Member State, impact on global safety regime etc. Impact and effectiveness of the host country contribution (e.g. quality of ARM, performance of counterparts, logistics, etc.). Common / recurring safety issues identified by IRRS missions. Analysis of IRRS reports from 2011 to 2013 comparing findings with previous analysis results for missions from 2006-2010. Identification of common and recurring issues in the regulation of non-nuclear applications IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

7. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 7 Introduction The IRRS programme has been very successful, with more than 50 missions and follow-up missions at the end of 2013 and a heavy programme into the future. However, as the next slide illustrates, demand for IRRS varies significantly globally, with an ongoing high demand in Europe, low in Africa and very low in Asia, the Middle East and Latin America. The reasons for this significant regional variability may be a topic for discussion at this workshop, since it would appear the IRRS programme is not equally attractive to all Member States. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

8. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 8 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective June 2006 to November 2014: Global Distribution of IRRS Missions and Follow-Ups Implementation is uneven regionally: High in Europe Low in Africa Very low in Asia and Latin America IAEA MS: 162 (February 2014)

9. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 9 Assessment of IRRS Programme Performance The Analysis of IRRS reports highlights many programme strengths: Provides Host with an objective, evidence-based status of the legal, governmental and regulatory framework and regulatory infrastructure relative to applicable IAEA requirements. Provides the basis for an improvement programme. Promotes and strengthens implementation of internationally agreed standards for safety and provides input to the review and revision of IAEA Safety Standards. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

10. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 10 Assessment of IRRS Programme Performance Provides opportunity to exchange national experiences and practices. Promotes harmonisation, recognises regional and national differences and the unique approaches that contribute to safety. Relative to 2006-2010, IRRS reports between 2011 and 2013 reveal improved objective reporting and continuous improvement in evidence-gathering, report drafting and validation of host country self- assessment findings. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

11. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 11 Assessment of IRRS Programme Performance Despite the foregoing evidence of IRRS effectiveness, the mission reports analysed reveal numerous examples of inconsistencies in recommendations and variable interpretation of the requirements, from which lessons can be learned. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

12. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 12 Recurring safety issues The analysis reveals certain IRRS recommendations and suggestions frequently recur: Some issues recur throughout the analysis period of 2006 to 2013 Some issues are no longer seen after 2010, but conversely, in the period since 2010 there are new recurring issues. IRRS Guidelines have been revised and GSR Parts 1 and 3 were published, superseding earlier editions. These changes may have influenced the above. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

13. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 13 Recurring safety issues The existence of recurring issues suggests either: that countries put insufficient priority on their improvement and / or; the applicable IAEA requirements are not realistically achievable in the timescales set. Significantly, it is possible too, that issues of understanding and interpretation of the requirements (by both reviewers and hosts) plays a part. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

14. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 14 Recurring safety issues The IRRS process encourages reviewers to develop observations (recommendations, suggestions and good practice) specific to circumstances in the host country. This is fundamental to the concept of peer review, but leads to differing perspectives, even where addressing common and recurring issues. The consequence of this approach is variability in emphasis and interpretation of compliance with specific requirements IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

15. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 15 Recurring safety issues The foregoing suggests the need for discipline in drafting IRRS reports, such that: conclusions reached on recurring issues are consistent across every mission; similar observations in each country use the same bases; and IRRS observations directly contribute to the continuous improvement of IAEA safety standards and the global exchange of regulatory experience IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

16. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 16 IRRS as an integrated service: Addressing the full scope of the national legal, governmental and regulatory framework for nuclear and radiation safety. IRRS offers an integrated approach to the review of national frameworks for safety, thereby including in its scope: every organisation having regulatory responsibilities and functions; all facilities and activities involving ionising radiation above exemption levels. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective This analysis finds that since the inception of the IRRS programme in 2006, only a small minority of missions have fully met this objective.

17. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 17 IRRS as an integrated service: Addressing the full scope of the national legal, governmental and regulatory framework for nuclear and radiation safety. To illustrate the issue behind this important finding of the analysis: An IAEA NSTC joint working group found that 42% of Member States do not have adequate patient protection programmes. More than 50% have yet to implement adequate waste management and environmental protection programmes. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

18. IAEA International Atomic Energy Agency Average Status of TSAs 1 - 5 for MS in all Regions

19. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 19 IRRS as an integrated service: The host country commonly excludes certain facilities and activities and usually only one regulatory body is involved. Commonly, other national agencies assigned regulatory functions and responsibilities are not consulted. IRRS Guidelines state; “If the state wishes to exclude specific facilities and activities, a detailed explanation for the exclusion of these facilities or activities should be provided. The mission report will reflect this in both the title and content of the relevant report sections, including the Executive Summary”. The analysis indicates this Guidelines requirement has rarely been met (0nly Switzerland to date) IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

20. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 20 IRRS as an integrated service: Thus, the analysis team’s first ‘lesson-learned’ … IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson Learned LL5 LL5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and activities involving ionising radiation above exemption levels. Limiting the scope of IRRS missions or excluding national agencies assigned regulatory responsibilities and functions as defined by GSR Part-1, introduces potential to miss essential findings associated with nuclear and radiation safety. IRRS Guidelines emphasis on national framework for safety is not being met.

21. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 21 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson Learned LL5 LL5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and activities involving ionising radiation above exemption levels. LL5: 1 st Proposal for Consideration P7 Proposal: IRRS should review full extent of national, legal and governmental framework and regulatory infrastructure for safety, directly involving all organisations having oversight of all radiation facilities and activities in the country. IAEA should consider revising IRRS modular structure making optional Module 11 a ‘core’ module or explicitly incorporating all elements of Module 11 into core modules to ensure full extent of Host’s national framework for safety is addressed.

22. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 22 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson Learned LL5 LL5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and activities involving ionising radiation above exemption levels. LL5: 2 nd Proposal for Consideration P8 Proposal: Where the Host Country requests an integrated review as proposed above, then all agencies collectively comprising the national regulatory body and assigned responsibility and providing the full scope of regulatory responsibilities and functions (as defined in GSR Part-1) should be invited to participate in the review, with their various identities and roles confirmed at the preparatory meeting.

23. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 23 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson Learned LL5 LL5 Lessons-Learned: IRRS should include every organisation having significant relevant regulatory responsibilities and functions and review the oversight of all facilities and activities involving ionising radiation above exemption levels. LL5: 3 rd Proposal for Consideration P9 Proposal: When requesting a peer review which would not be integrated to the extent proposed, the Host Country should formally notify the IAEA of its specific requirements so that a more appropriate alternative mission can be offered.

24. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 24 IRRS and the Safety Standards: Analysis suggests IRRS Team Members and Host counterparts may not always share common understanding / interpretation of the safety requirements and their application. This is a complex issue that calls for a training programme to build better understanding of IAEA safety requirements, with expectations of what must be evidentially in place to demonstrate compliance. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

25. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 25 IRRS and the Safety Standards: ‘Lesson-learned LL6’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

26. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 26 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective LL6: 1 st Proposal for Consideration P10 Proposal: IAEA should develop mechanisms to facilitate common understanding of IAEA Safety Standards and relevant guidance throughout IRRS process. This could include training programmes, improved templates for self-assessment (including much improved ‘expectations’) and for drafting IRRS reports and developing improved guidance for preparation of advanced reference material (ARM). Lesson Learned LL6 LL6 Lessons Learned: IRRS Team Members and Host Country counterparts may not always have a common understanding of the relevant IAEA requirements and their proper application. In a few instances, recommendations and suggestions are not based on the IAEA Safety Standards.

27. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 27 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson Learned LL6 LL6 Lessons Learned: IRRS Team Members and Host Country counterparts may not always have a common understanding of the relevant IAEA requirements and their proper application. In a few instances, recommendations and suggestions are not based on the IAEA Safety Standards. LL6: 2 nd Proposal for Consideration P11 Proposal: IAEA should consider on-line training for both IRRS reviewers and Host Country addressing IRRS process, including drafting technical notes, formulating observations and selecting bases. Before embarking on a pre-IRRS self-assessment and preparation of ARM, Host Country should have access to IAEA online training tools to facilitate training on interpretation and application of the IAEA Safety Standards.

28. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 28 IRRS and the Safety Standards: ‘Lesson-learned LL7’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

29. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 29 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson Learned LL7 LL7 Lessons Learned: Analysis reveals wide-ranging inconsistencies in application and interpretation of standards and some recommendations and suggestions have no basis in the Standards. LL7: Proposal for Consideration P15 Proposal: The IAEA Deputy Coordinator’s role in ensuring correct use of IAEA requirements should be reinforced. IRRS Guidelines should assign to the Deputy Coordinator, responsibility for alerting IRRS reviewers and counterparts where misunderstanding / misinterpretation of a requirement is evident.

30. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 30 IRRS and the Safety Standards: ‘Lesson-learned LL9’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

31. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 31 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL9 LL9 Lessons Learned: No procedure for feedback from IRRS to support development and revision of IAEA safety standards. IRRS Team members accumulate considerable experience of application of IAEA safety standards. Post-IRRS, this asset may not be optimally utilised in providing feedback to improve standards, developing guidance and setting expectations with regard to their effective application. LL9: 1 st Proposal for Consideration P17 Proposal: The IAEA Deputy Coordinator (DC) should be assigned responsibility for gathering feedback on the application of IAEA safety standards and formally reporting it to IAEA. Special attention should be paid to requirements that appear to be interpreted variably.

32. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 32 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL9 LL9 Lessons Learned: No procedure for feedback from IRRS to support development and revision of IAEA safety standards. IRRS Team members accumulate considerable experience of application of IAEA safety standards. Post-IRRS, this asset may not be optimally utilised in providing feedback to improve standards, developing guidance and setting expectations with regard to their effective application. LL9: 2 nd Proposal for Consideration P18 Proposal: IAEA should ensure feedback from experienced IRRS experts and counterparts is formally taken into account during review and revision of IAEA safety standards.

33. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 33 IRRS as peer review: IRRS Guidelines state a peer review should be a validation of the host country’s self-assessment... … however, the analysis reveals little evidence that the Host self- assessment is actively referenced by reviewers during the mission. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

34. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 34 IRRS as peer review: ‘Lesson-learned LL10’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

35. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 35 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL10 LL10 Lessons Learned: Analysis of IRRS reports reveals little evidence that Host self-assessment is actively referenced by reviewers and few references to non-compliances identified during self-assessment already being addressed in Host’s action plan. LL10: 1 st Proposal for Consideration P19 Proposal: The IRRS Team’s initial impressions of self- assessment and action plan should be circulated to reviewers and the host immediately pre-mission and form an annex to the IRRS report.

36. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 36 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL10 LL10 Lessons Learned: Analysis of IRRS reports reveals little evidence that Host self-assessment is actively referenced by reviewers and few references to non-compliances identified during self-assessment already being addressed in Host’s action plan. LL10: 2 nd Proposal for Consideration P20 Proposal: IRRS report should clearly document validation (or otherwise) of host’s self-assessment as a preface to the reviewer’s findings. Where country has identified issues through self-assessment, this should be acknowledged in the IRRS report just as issues not revealed in the self- assessment are addressed in the IRRS report.

37. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 37 IRRS as peer review: IRRS Guidelines: Peer review should be an exchange of professional knowledge and experience. Analysis reveals that mission schedules typically assign only 2-4 days for interviews and observations (which include valuable professional discussions) whereas 8-10 days are allotted to clarification of findings and report writing. Hence, little time for mutual learning and exchange of regulatory experience. Strategies and processes to reduce report writing hours would benefit all parties - releasing mission time for exchange of experience, in the spirit of peer review. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

38. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 38 IRRS as peer review: ‘Lesson-learned LL11’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

39. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 39 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL11 LL11 Lessons Learned: Time available for interviews and observation is increasingly constrained, resulting in less time for mutual learning and the exchange of regulatory experience. LL11: Proposal for Consideration P22 Proposal: Having read Host ARM, IRRS Team members should pre-draft their sections of the IRRS report, using an IRRS report template. These ‘initial impressions’ should be shared with other Team members (and host) at least one week before the mission, accompanied by a list of issues of concern, where specific interviews and observations appear necessary.

40. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 40 IRRS as peer review: Comprehensive, accurate, but more succinct and consistently structured Host Country Advance Reference Material (ARM) including the self- assessment, may free up time for the exchange of experience during an IRRS mission. This goal may be facilitated by offering training, guidance and tools (such as ARM templates) so that both Host counterparts and IRRS reviewers can make more effective use of this comprehensive evidence of the current status of the Host’s infrastructure for safety. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

41. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 41 IRRS as peer review: ‘Lesson-learned LL12’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

42. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 42 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL12 LL12 Lessons Learned: More succinct and consistently structured Host Country Advance Reference Material (ARM) including self-assessment may improve IRRS mission effectiveness by facilitating discussions and freeing up mission time for exchange of experience. LL12: Proposal for Consideration P23 Proposal: IAEA should develop a template for consistent and straightforward compilation of ARM, possibly based on SARIS self- assessment report. At the same time, SARIS question-sets should be reviewed and abbreviated.

43. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 43 IRRS recommendations and suggestions: Analysis indicates that IRRS recommendations and suggestions are not always sufficiently specific or realistic to result in tangible improvements correlated with the Host’s action plan. The expectation is that the Host action plan will be revised to accommodate IRRS recommendations or suggestions. For this to be effective, recommendations and suggestions must: follow the SMART principle (i.e. Specific, Measurable, Achievable, Realistic and Timely); and recognise the circumstances of the host country and realism therefore, of its action plan for improvement. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

44. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 44 IRRS recommendations and suggestions: ‘Lesson-learned LL13’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

45. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 45 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL13 LL13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements correlated with the Host Country’s action plan. LL13: 1 st Proposal for Consideration P24 Proposal: The “SMART” test should be formally applied when developing recommendations and suggestions (i.e. IRRS Team Leader working with Team Coordinator, should ensure they are Specific, Measurable, Achievable, Realistic and Timely).

46. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 46 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL13 LL13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements correlated with the Host Country’s action plan. LL13: 2 nd Proposal for Consideration P25 Proposal: To the extent possible, recommendations and suggestions should be correlated with the Host Country’s existing action plan and refined, as appropriate to be realistically in accordance with the action plan.

47. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 47 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL13 LL13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements correlated with the Host Country’s action plan. LL13: 3 rd Proposal for Consideration P26 Proposal: Recommendations that may take a long time to be implemented or have a high priority, should include interim goals set out in the IRRS report and in due course, by agreement, transferred to the Host Country’s updated action plan.

48. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 48 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL13 LL13 Lessons Learned: Recommendations and suggestions in IRRS reports are not always specific, realistic or designed to result in tangible improvements correlated with the Host Country’s action plan. LL13: 4 th Proposal for Consideration P27 Proposal: Once an IRRS Report is complete and forwarded to the host government through official channels, IAEA should request an acknowledgement of receipt of the report and acceptance of its findings, including the timescales set for implementing the recommendations.

49. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 49 IRRS reporting of good practice: Although the term ‘good practice’ is unambiguously defined in the IRRS Guidelines, the analysis suggests the concept is over-used and frequently misapplied. The analysis also finds value in reporting notable practice ‘graded’ according to the circumstances of the Host Country. Thus, a two-level approach to reporting interesting, unique or otherwise worthy practices may be beneficial: first level of ‘notable practice’ reserved for practices which are effective in the circumstances of the Host Country but do not fully meet the criteria for an IRRS good practice; and second level of ‘good practice’ remaining as currently defined in the IRRS Guidelines. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

50. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 50 IRRS reporting of good practice: The analysis finds little evidence that good practice identified during IRRS is effectively distributed to the global community in manner that encourages its adoption. A good practice is notable only as long as it is worthy of sharing with the wider community. Once a good practice is established or widely implemented, it represents the status quo. Analysis reveals that essentially the same (or similar) ‘good practices’ are repeatedly reported over a long period. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

51. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 51 IRRS reporting of good practice: ‘Lesson-learned LL15’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

52. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 52 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL15 LL15 Lessons Learned: The concept and definition of a good practice is not applied rigorously. LL15: Proposal for Consideration P29 Proposal: IAEA should consider the following: Require all elements of a good practice be comprehensively addressed in the IRRS report. Unless all four criteria clearly met, concept of good practice should not be applicable; Define and encourage reporting of a second level of practice (e.g. ‘Notable Practice’) which acknowledges actions and processes notable in the circumstances of the Host but do not meet IRRS Guidelines criteria for good practice.

53. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 53 IRRS reporting of good practice: ‘Lesson-learned LL16’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

54. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 54 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL16 LL16 Lessons Learned: Dissemination of good practices occurs, but there is little evidence of their effective implementation worldwide. LL16: Proposal for Consideration P30 Proposal: IAEA should consider the following: A good practice should be documented only where it has not been seen before. This suggests development of an IAEA searchable database of good practice, easily accessible to IRRS reviewers and the global community. MS should be encouraged to report to the database if (and how) they adopt the good practices they find there.

55. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 55 IRRS review completeness: Analysis shows that partial application of an IAEA requirement together with inconsistent or variable interpretation of the requirement occasionally leads to an incomplete review with recommendations and/or suggestions that are not comprehensive. In some cases, a requirement is addressed in various separate parts of the IRRS report without due cross-referencing, so that the extent of compliance is not necessarily clear to the reader. Without a structured, explicit reporting of compliance with every element of a requirement, it is difficult to determine whether the unreported parts of the requirement are in fact, implemented by the Host Country. Not reporting (i.e. keeping silent on) elements of a requirement cannot be accepted as tacit validation of compliance with the requirement as a whole. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

56. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 56 IRRS review completeness: ‘Lesson-learned LL17’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

57. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 57 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL17 LL17 Lessons Learned: Partial application of IAEA requirements occasionally leads to an incomplete review because only the implemented elements are addressed during the mission. LL17: 1 st Proposal for Consideration P31 Proposal: IRRS report template should explicitly ensure that every element of each requirement is addressed in full. For each element of each IAEA requirement clear statements of compliance should be included for all regulated facilities and activities of the national framework for safety.

58. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 58 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL17 LL17 Lessons Learned: Partial application of IAEA requirements occasionally leads to an incomplete review because only the implemented elements are addressed during the mission. LL17: 2 nd Proposal for Consideration P32 Proposal: Where it is appropriate to use only part of a requirement as the basis for a recommendation or suggestion care must be taken to maintain the integrity and original intent of the requirement as a whole. This may be achieved if the over-arching requirement (printed in bold font in the more recent IAEA safety standards) is reproduced in full in IRRS report, together with relevant part of requirement (i.e. ‘shall’ statement) being used as the basis.

59. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 59 IRRS review completeness: Analysis shows that non-compliances noted in the body text of IRRS reports, occasionally do not result in a subsequent recommendation. The issue is often reflected in the text with terms such as; ‘concern’ or ‘advice’. The IRRS Guidelines clearly require that all identified non-compliances must lead to a recommendation. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

60. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 60 IRRS review completeness : ‘Lesson-learned LL18’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

61. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 61 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL18 LL18 Lessons Learned: Non-compliances are occasionally noted in the body text of IRRS reports which do not result in a subsequent recommendation. LL18: Proposal for Consideration P33 Proposal: IRRS reviewers should be reminded of the IRRS Guidelines statement that all identified non-compliances must lead to a recommendation.

62. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 62 IRRS review consistency : Analysis reveals that observations in any one IRRS report are occasionally inconsistent with each other. In some cases there is a statement of compliance in the findings and elsewhere in the same report, a recommendation or suggestion indicating the opposite. Observations do not always appear in the most appropriate part of the report. This may be the consequence of the current modular structure of the IRRS. All modules of the IRRS are interdependent. As such, it is important that findings or recommendations and suggestions in any one section of the report are drafted fully taking into account other areas under review. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

63. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 63 IRRS review consistency: ‘Lesson-learned LL19’ IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

64. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 64 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL19 LL19 Lessons Learned: Observations in the same IRRS report are occasionally inconsistent with each other, particularly where they’re made in another module. Additionally, in some IRRS reports analysed, observations do not appear in the most appropriate part of the report. LL19: 1 st Proposal for Consideration P36 Proposal: Guidance (in both the self-assessment and IRRS report template) should highlight those requirements that typically have an impact across several modules and indicate how they are linked.

65. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 65 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL19 LL19 Lessons Learned: Observations in the same IRRS report are occasionally inconsistent with each other, particularly where they’re made in another module. Additionally, in some IRRS reports analysed, observations do not appear in the most appropriate part of the report. LL19: 2 nd Proposal for Consideration P37 Proposal: Observations should be explicitly directed to the organisation responsible for their implementation (e.g. government, regulatory body, other agency etc.).

66. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 66 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Lesson-Learned LL19 LL19 Lessons Learned: Observations in the same IRRS report are occasionally inconsistent with each other, particularly where they’re made in another module. Additionally, in some IRRS reports analysed, observations do not appear in the most appropriate part of the report. LL19: 3 rd Proposal for Consideration P38 Proposal: To avoid repeatedly documenting the same observation when it addresses similar issues across a variety of IRRS modules, it may be reported in the IRRS module for which it has the most significance. Due reference to this recommendation or suggestion should then appear in all other modules of the report for which the observation is applicable (e.g. an observation on training inspectors might appear in Module 3 but be referenced in Module 7).

67. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 67 IRRS Analysis - Conclusions: Based on this analysis of 50 IRRS missions (including 7 follow-up missions) during the period 2006 to 2013, the following conclusions may be drawn: The IRRS is established as the prime peer review of national frameworks for nuclear and radiation safety. IRRS provides Member States with the opportunity to identify and prioritise necessary improvements to their national regulatory infrastructures for nuclear, radiation, radioactive waste and transport safety. The IRRS has given regulators subject to IRRS and government officers of participating States access to international advice and the exchange of regulatory knowledge and experience. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

68. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 68 IRRS Analysis - Conclusions: The IRRS is an integrated service. The term ‘integrated’ does not refer to the integration of IAEA services but specifically to the integration of all State organisations, functions and activities that comprise the national, legal and governmental framework and infrastructure for safety. Thus, IRRS should address the Host Country’s regulatory framework for protecting the public, patients, workers and the environment from radiation risks associated with all regulated facilities and activities in the country. In consequence, all national agencies that comprise the national regulatory body for all facilities and activities in the country should expect to participate in IRRS. By the IRRS Guidelines definition, an IAEA review of limited scope not addressing all elements of the State’s national, legal and governmental framework and infrastructure for safety should not be considered as an IRRS. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

69. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 69 IRRS Analysis - Conclusions: Where the State has good cause to ask for a limited scope the current IRRS Guidelines call for a written justification. The IRRS is under-utilised as a source of objective feedback for the improvement of IAEA safety standards. The self-assessment and experience exchange processes would benefit from improvement. Since the IRRS follow-up process appears not to be functioning in accordance with the Guidelines, alternative progress monitoring methodologies may be needed with greater flexibility according to Host needs and circumstances. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

70. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 70 IRRS Analysis - Conclusions: An ongoing programme of IRRS lessons-learned workshops with dissemination of lessons-learned, reporting IRRS results at international undertakings and conducting periodical analysis of the IRRS process would benefit the global safety regime. Since IRRS is a dynamic process, its efficiency and effectiveness can be further improved by using experience gained from previous missions, particularly in the following areas: Drafting and global dissemination of recommendations, suggestions and good practices. IRRS report completeness in terms of comprehensive review of all relevant IAEA standards and guidance. Consistency in the structure, style, content and emphasis of IRRS reports. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective

71. IAEA Contents END WORKSHOP ON “LESSONS LEARNED FROM THE INTEGRATED REGULATORY REVIEW SERVICE (IRRS) MISSIONS Moscow 09-11 December 2014 71 IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective