Acting Section Head, Regulatory Activities Section, NSNI, IAEA The IAEA Integrated Regulatory Review Service (IRRS) The IAEA Self-Assessment Methodology and Tools Moscow, 09 - 11 December 2014 Adriana Nicic Acting Section Head, Regulatory Activities Section, NSNI, IAEA Stephen Evans Project Officer, Regulatory Infrastructure and Transport Safety Section, Division of Radiation, Transport and Waste Safety (NSRW) IAEA

The IAEA Self-Assessment Methodology and Tools 2003: Origins lie in a project related to The ‘Radiation Safety and Security Infrastructure Appraisal’ (RaSSIA) programme. RaSSIA revealed the need for tools to facilitate national level self-assessment of regulatory infrastructures for safety. Thus … 2004: The IAEA self-assessment project established by NSRW. 2004/5: Based on long-standing quality management principles, the IAEA self-assessment methodology was devised and developed through a series of consultancies and a technical meeting. The methodology was developed for ongoing, cyclic application as a core part of any regulatory body’s management system. It was not intended for IAEA appraisal or peer review services.

The IAEA Self-Assessment Methodology and Tools 2005: First version of the Self-Assessment Tool (SAT) software introduced to implement the IAEA methodology in Member States. SAT pre-dated IRRS and therefore was not structured in accordance with the IRRS modular system. 2006: RaSSIA and IRRT replaced by IRRS. 2006-9: NSNI / NSRW work together to develop SAT into a more sophisticated tool for use by regulatory bodies seeking to periodically evaluate their current status against relevant IAEA requirements. 2009: SAT is replaced by Self-Assessment of the Regulatory Infrastructure for Safety (SARIS). 2010: Uprated SARIS introduced (based on user feedback). 2009 to date: SARIS required preparation for every IRRS

The IAEA Self-Assessment Methodology and Tools 2014: SARIS Guidelines published as IAEA Services Series 27 http://www-pub.iaea.org/books/IAEABooks/10697/SARIS-Guidelines-2014-Edition

The IAEA Self-Assessment Methodology and Tools SARIS is software built around the IAEA self-assessment methodology The SARIS platform can carry any question-sets on any topic. SARIS question-sets are devised using the SARIS Editor software, which has proven to be functional and effective. The software is reliable, stable and works on almost any PC and every edition of MS ‘Windows’ – an important issue for some States It has been widely adopted in IAEA Member States A number of variants of SARIS have been developed, including IRIS, the ‘Integrated Regulatory Infrastructure for Safety’ methodology A number of other products and variants are in preparation, all based on the SARIS platform

The IAEA Self-Assessment Methodology and Tools IRIS, the ‘Integrated Regulatory Infrastructure for Safety’ methodology was developed in 2011-2013 IRIS is based on SSG-16 “Establishing the Safety Infrastructure for a Nuclear Power Programme” IRIS is used in the tailored IRRS module for countries embarking on a nuclear power programme IRIS Guidelines were issued in 2014 as IAEA Services Series 28: http://www-pub.iaea.org/books/IAEABooks/10712/IRIS-Guidelines-2014-Edition SARIS and IRIS are fully aligned -

Self-assessment comments - Washington workshop 2010 The IAEA Self-Assessment Methodology and Tools Self-assessment comments - Washington workshop 2010 The value of the SAT tool has been recognized The conduct of a self assessment provides a benefit for improving safety even if an IRRS mission does not occur Questionnaires should be much simpler and less repetitive

It’s the question-sets! The IAEA Self-Assessment Methodology and Tools So what’s the problem with SARIS? It’s the question-sets!

The IAEA Self-Assessment Methodology and Tools So what’s the problem with SARIS? Current SARIS question-sets emulate the Safety Standards publications (such as GSR Part-1, GSR Part-3 and so on). Some safety standards have a great number of requirement statements – every one of which had to be incorporated in SARIS, re-phrased as closed primary questions. The result: Questions derived from many safety standards, and thus, often very similar. Almost every primary question generates a series of subsidiary questions, necessary to obtain the full evidence of compliance. For every primary question there’s a set of subsidiaries for a ‘Yes’ and another set for a ‘No’ response. The result is thousands of questions Why?

The IAEA Self-Assessment Methodology and Tools So what’s the problem with SARIS? Every requirement is just that …. a requirement. The IAEA cannot choose to include one and exclude another. It’s all or none! Although the context changes, many requirements are effectively duplicated in the various safety standards publications, particularly those associated with legislation, effective independence and the functions of the regulatory body. All these duplications are faithfully reproduced in SARIS Many requirements (especially those in the new structure of the standards) have a number of elements, each of which must be addressed during self-assessment. Some requirements are interpreted variably, or need further explanation. SARIS attempts to limit this tendency with still more subsidiary questions! SARIS is not just used for the IRRS. The result: 3,000+ questions to be waded through and they’re not even organised to match the modules of the IRRS

The IAEA Self-Assessment Methodology and Tools So what are we going to do about it? A review of SARIS question sets is taking place (recently reviewed question sets released in April 2014 include ‘EPR’ and ‘Research Reactors’). Initially, the SARIS questionnaire (and SARIS Report) are being aligned with the IRRS Modules, but for now, the number of questions remains unchanged. A comprehensive revision of SARIS question-set structure and content is planned. A new variant of SARIS will allow IRRS reviewers to draft the IRRS report in SARIS with direct reference to the host’s self-assessment made using SARIS – thus validating the self-assessment, as required by the IRRS Guidelines. Using this variant, reviewer’s technical notes will be presented alongside the host’s self- assessment and the host’s documentary evidence. The new SARIS will allow users in the host country to upload and archive the final IRRS report and present it against the original self-assessment and post-IRRS action-plan.

The IAEA Self-Assessment Methodology and Tools So what are we going to do about it? Other variants of SARIS are being developed - notably, the Regulations Comparator Tool (RCT). An e-learning video is now available to familiarise and update users in the effective operation of the SARIS system: http://gnssn.iaea.org/CSN/SA/SARIS%20Training%20and%20eLearning/Forms/AllItems.aspx

The IAEA Self-Assessment Methodology and Tools So what are we going to do about it? The challenge is to ensure IAEA requirements are properly and completely presented, but in a more user-friendly manner. A prime objective will be a substantial reduction in the number of questions and avoidance of similar questions or those that generate duplicated responses. It is likely that a series of consultancies will be commissioned to address these challenges and your feedback (particularly as a result of this ‘IRRS Lessons-Learned’ workshop) will be much appreciated.

IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective Thank you