IAEA International Atomic Energy Agency TOOLS AND METHODS FOR ENHANCING IRRS EFFECTIVENESS AND EFFICIENCY Presentation to the Workshop on Lessons Learned from IRRS Missions Moscow, Russian Federation 9-11 December 2014
IAEA Outline On the IRRS missions in general Continuous improvement of the IRRS programme Conclusions IRRS enhancements2
IAEA IRRS BASICS IRRS enhancements3
IAEA IRRS Process Prep-meeting about 6 months before Self-assessment by host First mission: 12 days Report finalization about 3 months Follow-up missions after 2-3 years, about 7 days IRRS enhancements4
IAEA IRRS Guidelines IRRS enhancements5 Published in May 2013 A series of Targeted Consultancies was held for collecting experience and advice for a thorough revision Novel edition will be compiled based on the consultancies
IAEA IAEA Safety Standards used in IRRS IRRS enhancements6
IAEA Basis for findings: GSR Part 1 Requirements IRRS enhancements7 IRRS Modules + other IAEA Safety Standards
IAEA Missions in IRRS enhancements8
IAEA Participation in the missions ( ) * No. of experts (including IAEA): 373 No. of participations: 707 From 59 countries IRRS enhancements9 * by October 2014
IAEA CONTINUOUS IMPROVEMENT OF THE IRRS PROGRAMME IRRS enhancements10
IAEA IAEA Nuclear Safety Action Plan IAEA is to “Strengthen IAEA peer reviews in order to maximize the benefits to Member States”. In specific: IAEA Secretariat to review the effectiveness of their peer reviews: IRRS, and therein Define effectiveness criteria, apply the criteria to a representative number of selected IRRS missions and prepare a report IRRS enhancements11
IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements12
IAEA First Basic IRRS Training Course First held in October participants from 38 MSs + EC In the framework of the EC-IAEA cooperation in the field of Nuclear Safety Covered all aspects of the IRRS programme, 16 Training Lectures, 3 Information Lectures Test Questionnaires for all Lectures IRRS enhancements13
IAEA Feedback on the course IRRS enhancements14
IAEA Feedback from the Test Questionnaires IRRS enhancements15
IAEA 2 nd Edition Held in October participants from 39 MSs + EC Even more favourable feedback IRRS enhancements16
IAEA On-site training By invitation from MSs Minimum 10 competent participants First held at ONR (Bootle, UK) 15 participants Excellent acceptance IRRS enhancements17
IAEA Refresher Training On the 0 th day of the missions Covers details important for the given missions: Overview of the IRRS Process Methodology of the review Evaluation and documentation of Observations Report writing IRRS Modules to review IRRS Mission Schedule Held in 2014 in the Netherlands and France IRRS enhancements18
IAEA Feedback IRRS enhancements19
IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements20
IAEA Mission Standards – ARM Report Template Under development / discussion IRRS enhancements21
IAEA Mission Standards – Team Assignment IRRS enhancements22
IAEA Mission Standards – Report Template Cross-contribution table Instructions for reviewers Issues to cover Format of observations IRRS enhancements23
IAEA Mission Standards – Schedule IRRS enhancements24
IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements25
IAEA Feedbacks on the IRRS process Questionnaire on the quality of the ARM – reviewers at mission start Questionnaire on effectiveness of the mission – reviewers at end of mission Round-table conclusions and suggestions – reviewers at end of mission Questionnaire on effectiveness of the mission – host after mission Questionnaire on effectiveness of the initial mission – host before follow-up mission Workshop for team leaders (January 2013) Workshop for experienced reviewers (April 2015) IRRS enhancements26
IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements27
IAEA Effectiveness and efficiency of IRRS NSAP requires establishing criteria for efficiency of IRRS missions 16 Performance Indicators introduced to characterize the E&E of a mission, such as: Team size; team experience; Time available to prepare for the mission; Feedback from team members and from hosts; Coverage of the Action Plan by findings; Balance of findings; Conciseness of mission report; etc. IRRS enhancements28 irrs/Pages/IRRS_pub_docs. aspx
IAEA Examples of Performance Indicators (1) IRRS enhancements29
IAEA Examples of Performance Indicators (2) IRRS enhancements30 Balance of findings: R*GP, R/S const. Expectations: a)R and GP are negatively correlated b)R and S are positively correlated 1)The expected correlations are not present in all missions 2)The reasons in case of R and GP are: reviewers feel obliged to compensate by GP some topics (with possible issues) are more frequently considered than others (with GP) 3)The reasons in case of R and S are: R -> S: compromise between the team & host S -> R: the team tends to be stricter or the specificities of the given mission
IAEA E & E of past IRRS missions Note: The Performance Indicator values presented here characterize the effectiveness and efficiency of the IRRS process and represent no indication whatsoever on the safety performance of the host countries of the missions IRRS enhancements31 The distance from optimum tends to decrease, i.e. the E&E tends to increase
IAEA Ways to improve IRRS Training IRRS Mission Standards Feedbacks Measuring effectiveness and efficiency Prompt Evaluation Report IRRS enhancements32
IAEA Prompt evaluation report CONTENTS Basic mission data Feedback from the team Feedback from the hosts Overall evaluation of effectiveness and efficiency Discussion, conclusions Summary IRRS enhancements33
IAEA Evaluation of a mission via PI’s IRRS enhancements34
IAEA Reports on nuclear missions IRRS enhancements35
IAEA Reports on nuclear missions IRRS enhancements36
IAEA Reports on nuclear missions IRRS enhancements37
IAEA IN SUMMARY IRRS enhancements38
IAEA Conclusions IRRS is an effective means of reviewing compliance of RB status and activities with IAEA safety standards Regular invitation of IRRS missions by all IAEA MS is highly recommended The IRRS process is mature and effective – yet it is under continuous development by using novel standardized tools and feedback from participants Lessons learned from past missions as well as mission effectiveness and efficiency analysed Prompt evaluations of IRRS missions to nuclear countries were performed in a pilot phase IRRS enhancements39
IAEA IRRS enhancements40 THANK YOU FOR YOUR ATTENTION Photo: V. Friedrich This activity is conducted by the IAEA, with funding by the European Union. The views expressed in this presentation do not necessarily reflect the views of the European Commission