IAEA International Atomic Energy Agency TOOLS AND METHODS FOR ENHANCING IRRS EFFECTIVENESS AND EFFICIENCY Presentation to the Workshop on Lessons Learned from IRRS Missions Moscow, Russian Federation 9-11 December 2014

IAEA Outline On the IRRS missions in general Continuous improvement of the IRRS programme Conclusions IRRS enhancements2

IAEA IRRS BASICS IRRS enhancements3

IAEA IRRS Process Prep-meeting about 6 months before Self-assessment by host First mission: 12 days Report finalization about 3 months Follow-up missions after 2-3 years, about 7 days IRRS enhancements4

IAEA IRRS Guidelines IRRS enhancements5 Published in May 2013 A series of Targeted Consultancies was held for collecting experience and advice for a thorough revision Novel edition will be compiled based on the consultancies

IAEA IAEA Safety Standards used in IRRS IRRS enhancements6

IAEA Basis for findings: GSR Part 1 Requirements IRRS enhancements7 IRRS Modules + other IAEA Safety Standards

IAEA Missions in IRRS enhancements8

IAEA Participation in the missions ( ) * No. of experts (including IAEA): 373 No. of participations: 707 From 59 countries IRRS enhancements9 * by October 2014

IAEA CONTINUOUS IMPROVEMENT OF THE IRRS PROGRAMME IRRS enhancements10

IAEA IAEA Nuclear Safety Action Plan IAEA is to “Strengthen IAEA peer reviews in order to maximize the benefits to Member States”. In specific: IAEA Secretariat to review the effectiveness of their peer reviews: IRRS, and therein Define effectiveness criteria, apply the criteria to a representative number of selected IRRS missions and prepare a report IRRS enhancements11

IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements12

IAEA First Basic IRRS Training Course First held in October participants from 38 MSs + EC In the framework of the EC-IAEA cooperation in the field of Nuclear Safety Covered all aspects of the IRRS programme, 16 Training Lectures, 3 Information Lectures Test Questionnaires for all Lectures IRRS enhancements13

IAEA Feedback on the course IRRS enhancements14

IAEA Feedback from the Test Questionnaires IRRS enhancements15

IAEA 2 nd Edition Held in October participants from 39 MSs + EC Even more favourable feedback IRRS enhancements16

IAEA On-site training By invitation from MSs Minimum 10 competent participants First held at ONR (Bootle, UK) 15 participants Excellent acceptance IRRS enhancements17

IAEA Refresher Training On the 0 th day of the missions Covers details important for the given missions: Overview of the IRRS Process Methodology of the review Evaluation and documentation of Observations Report writing IRRS Modules to review IRRS Mission Schedule Held in 2014 in the Netherlands and France IRRS enhancements18

IAEA Feedback IRRS enhancements19

IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements20

IAEA Mission Standards – ARM Report Template Under development / discussion IRRS enhancements21

IAEA Mission Standards – Team Assignment IRRS enhancements22

IAEA Mission Standards – Report Template Cross-contribution table Instructions for reviewers Issues to cover Format of observations IRRS enhancements23

IAEA Mission Standards – Schedule IRRS enhancements24

IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements25

IAEA Feedbacks on the IRRS process Questionnaire on the quality of the ARM – reviewers at mission start Questionnaire on effectiveness of the mission – reviewers at end of mission Round-table conclusions and suggestions – reviewers at end of mission Questionnaire on effectiveness of the mission – host after mission Questionnaire on effectiveness of the initial mission – host before follow-up mission Workshop for team leaders (January 2013) Workshop for experienced reviewers (April 2015) IRRS enhancements26

IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements27

IAEA Effectiveness and efficiency of IRRS NSAP requires establishing criteria for efficiency of IRRS missions 16 Performance Indicators introduced to characterize the E&E of a mission, such as: Team size; team experience; Time available to prepare for the mission; Feedback from team members and from hosts; Coverage of the Action Plan by findings; Balance of findings; Conciseness of mission report; etc. IRRS enhancements28 irrs/Pages/IRRS_pub_docs. aspx

IAEA Examples of Performance Indicators (1) IRRS enhancements29

IAEA Examples of Performance Indicators (2) IRRS enhancements30 Balance of findings: R*GP, R/S  const. Expectations: a)R and GP are negatively correlated b)R and S are positively correlated 1)The expected correlations are not present in all missions 2)The reasons in case of R and GP are: reviewers feel obliged to compensate by GP some topics (with possible issues) are more frequently considered than others (with GP) 3)The reasons in case of R and S are: R -> S: compromise between the team & host S -> R: the team tends to be stricter or the specificities of the given mission

IAEA E & E of past IRRS missions Note: The Performance Indicator values presented here characterize the effectiveness and efficiency of the IRRS process and represent no indication whatsoever on the safety performance of the host countries of the missions IRRS enhancements31 The distance from optimum tends to decrease, i.e. the E&E tends to increase

IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements32

IAEA Prompt evaluation report CONTENTS Basic mission data Feedback from the team Feedback from the hosts Overall evaluation of effectiveness and efficiency Discussion, conclusions Summary IRRS enhancements33

IAEA Evaluation of a mission via PI’s IRRS enhancements34

IAEA Reports on nuclear missions IRRS enhancements35

IAEA Reports on nuclear missions IRRS enhancements36

IAEA Reports on nuclear missions IRRS enhancements37

IAEA IN SUMMARY IRRS enhancements38

IAEA Conclusions IRRS is an effective means of reviewing compliance of RB status and activities with IAEA safety standards Regular invitation of IRRS missions by all IAEA MS is highly recommended The IRRS process is mature and effective – yet it is under continuous development by using novel standardized tools and feedback from participants Lessons learned from past missions as well as mission effectiveness and efficiency analysed Prompt evaluations of IRRS missions to nuclear countries were performed in a pilot phase IRRS enhancements39

IAEA IRRS enhancements40 THANK YOU FOR YOUR ATTENTION Photo: V. Friedrich This activity is conducted by the IAEA, with funding by the European Union. The views expressed in this presentation do not necessarily reflect the views of the European Commission