1. Acting Section Head, Regulatory Activities Section, NSNI, IAEA
The IAEA Integrated Regulatory Review Service (IRRS) The IAEA Self-Assessment Methodology and Tools
Moscow, December 2014
Adriana Nicic
Acting Section Head, Regulatory Activities Section, NSNI, IAEA
Stephen Evans
Project Officer, Regulatory Infrastructure and Transport Safety Section, Division of Radiation, Transport and Waste Safety (NSRW) IAEA

2. The IAEA Self-Assessment Methodology and Tools
2003: Origins lie in a project related to The ‘Radiation Safety and Security Infrastructure Appraisal’ (RaSSIA) programme.
RaSSIA revealed the need for tools to facilitate national level self-assessment of regulatory infrastructures for safety. Thus …
2004: The IAEA self-assessment project established by NSRW.
2004/5: Based on long-standing quality management principles, the IAEA self-assessment methodology was devised and developed through a series of consultancies and a technical meeting.
The methodology was developed for ongoing, cyclic application as a core part of any regulatory body’s management system. It was not intended for IAEA appraisal or peer review services.

3. The IAEA Self-Assessment Methodology and Tools
2005: First version of the Self-Assessment Tool (SAT) software introduced to implement the IAEA methodology in Member States.
SAT pre-dated IRRS and therefore was not structured in accordance with the IRRS modular system.
2006: RaSSIA and IRRT replaced by IRRS.
2006-9: NSNI / NSRW work together to develop SAT into a more sophisticated tool for use by regulatory bodies seeking to periodically evaluate their current status against relevant IAEA requirements.
2009: SAT is replaced by Self-Assessment of the Regulatory Infrastructure for Safety (SARIS).
2010: Uprated SARIS introduced (based on user feedback).
2009 to date: SARIS required preparation for every IRRS

4. The IAEA Self-Assessment Methodology and Tools
2014: SARIS Guidelines published as IAEA Services Series 27

5. The IAEA Self-Assessment Methodology and Tools
SARIS is software built around the IAEA self-assessment methodology
The SARIS platform can carry any question-sets on any topic.
SARIS question-sets are devised using the SARIS Editor software, which has proven to be functional and effective.
The software is reliable, stable and works on almost any PC and every edition of MS ‘Windows’ – an important issue for some States
It has been widely adopted in IAEA Member States
A number of variants of SARIS have been developed, including IRIS, the ‘Integrated Regulatory Infrastructure for Safety’ methodology
A number of other products and variants are in preparation, all based on the SARIS platform

6. The IAEA Self-Assessment Methodology and Tools
IRIS, the ‘Integrated Regulatory Infrastructure for Safety’ methodology was developed in
IRIS is based on SSG-16 “Establishing the Safety Infrastructure for a Nuclear Power Programme”
IRIS is used in the tailored IRRS module for countries embarking on a nuclear power programme
IRIS Guidelines were issued in 2014 as IAEA Services Series 28:
SARIS and IRIS are fully aligned -

7. Self-assessment comments - Washington workshop 2010
The IAEA Self-Assessment Methodology and Tools
Self-assessment comments - Washington workshop 2010
The value of the SAT tool has been recognized
The conduct of a self assessment provides a benefit for improving safety even if an IRRS mission does not occur
Questionnaires should be much simpler and less repetitive

8. It’s the question-sets!
The IAEA Self-Assessment Methodology and Tools
So what’s the problem with SARIS?
It’s the question-sets!

9. The IAEA Self-Assessment Methodology and Tools
So what’s the problem with SARIS?
Current SARIS question-sets emulate the Safety Standards publications (such as GSR Part-1, GSR Part-3 and so on).
Some safety standards have a great number of requirement statements – every one of which had to be incorporated in SARIS, re-phrased as closed primary questions.
The result: Questions derived from many safety standards, and thus, often very similar. Almost every primary question generates a series of subsidiary questions, necessary to obtain the full evidence of compliance.
For every primary question there’s a set of subsidiaries for a ‘Yes’ and another set for a ‘No’ response.
The result is thousands of questions
Why?

10. The IAEA Self-Assessment Methodology and Tools
So what’s the problem with SARIS?
Every requirement is just that …. a requirement. The IAEA cannot choose to include one and exclude another. It’s all or none!
Although the context changes, many requirements are effectively duplicated in the various safety standards publications, particularly those associated with legislation, effective independence and the functions of the regulatory body. All these duplications are faithfully reproduced in SARIS
Many requirements (especially those in the new structure of the standards) have a number of elements, each of which must be addressed during self-assessment.
Some requirements are interpreted variably, or need further explanation. SARIS attempts to limit this tendency with still more subsidiary questions!
SARIS is not just used for the IRRS.
The result: 3,000+ questions to be waded through and they’re not even organised to match the modules of the IRRS

11. The IAEA Self-Assessment Methodology and Tools
So what are we going to do about it?
A review of SARIS question sets is taking place (recently reviewed question sets released in April 2014 include ‘EPR’ and ‘Research Reactors’). Initially, the SARIS questionnaire (and SARIS Report) are being aligned with the IRRS Modules, but for now, the number of questions remains unchanged.
A comprehensive revision of SARIS question-set structure and content is planned.
A new variant of SARIS will allow IRRS reviewers to draft the IRRS report in SARIS with direct reference to the host’s self-assessment made using SARIS – thus validating the self-assessment, as required by the IRRS Guidelines.
Using this variant, reviewer’s technical notes will be presented alongside the host’s self- assessment and the host’s documentary evidence.
The new SARIS will allow users in the host country to upload and archive the final IRRS report and present it against the original self-assessment and post-IRRS action-plan.

12. The IAEA Self-Assessment Methodology and Tools
So what are we going to do about it?
Other variants of SARIS are being developed - notably, the Regulations Comparator Tool (RCT).
An e-learning video is now available to familiarise and update users in the effective operation of the SARIS system:

13. The IAEA Self-Assessment Methodology and Tools
So what are we going to do about it?
The challenge is to ensure IAEA requirements are properly and completely presented, but in a more user-friendly manner.
A prime objective will be a substantial reduction in the number of questions and avoidance of similar questions or those that generate duplicated responses.
It is likely that a series of consultancies will be commissioned to address these challenges and your feedback (particularly as a result of this ‘IRRS Lessons-Learned’ workshop) will be much appreciated.

14. IRRS Missions 2006-2013: Analysis from a Radiation Safety Perspective
Thank you