1. IAEA International Atomic Energy Agency PGEC Part IV The International System of Radiation Protection and the Regulatory Framework Module IV.10. Assessment of effectiveness of the regulatory programmes 10.2 Programme data & performance criteria Postgraduate Educational Course in radiation protection and the Safety of Radiation sources

2. IAEA This module will discuss the following aspects:  Management and assessment of the effectiveness of the Regulatory Programme, Management system  Collection and analysis of program data, Programme performance indicators, peer review, conducting an assessment, performance criteria; Overview of Module IV.10.2 2

3. IAEA Collection and Analysis of Programme Data Collection and Analysis of Programme Data is an important part of a Management System that aims to maintain an effective regulatory programme

4. IAEA  Reviewing a regulatory programme Analyzing data on the operation of a regulatory system is an important part of assessing the effectiveness of the programme That analysis is used as the basis for changes to regulatory body procedures and approaches to improve effectiveness Analysis of programme data

5. IAEA  Reviewing a regulatory programme Adjusting inspection frequency More or less depth to authorization procedure Update regulations More guidance for authorized users Analysis of programme data

6. IAEA Collection of Programme Data  Regulatory Body Information System Powerful tool to help review the effectiveness of regulatory infrastructure Provides useful, qualitative information All member states included in the Model Project received computerised system RAIS Developed by IAEA

7. IAEA l Laws/regulations and Regulatory Body l Notification l Authorization l Inspection l Enforcement l Emergecy Response Sources of Information l Investigation & follow-up l Technical Services l Coordination and cooperation l Staffing and training l Funding l Information Dissemination

8. IAEA Programme data  Accident Database Users should notify the Regulatory Body of accidents and incidents Results of analysis of accident database users, equipment manufactures Generic problem identified notifying other users at similar risk withdrawal from service of device at fault corrective action by manufacturer alterations to authorization requirements / regulations

9. IAEA Programme data  Inventory of radiation sources Number of sources at an installation Classified by practices and types of sources Installations of a given practice Installations that possess a specific model of equipment

10. IAEA Programme data  Authorization Administrative information Name, address, phone number, etc. History of authorizations Authorization status Dead lines for administrative actions History of sources that may have been in more than one installation

11. IAEA Programme data  Inspection and Enforcement Categorisation of practice according to risk Frequency of inspections according to category Schedule of inspections in a given period of time History of inspections of a given installation Enforcement actions Action dead lines

12. IAEA Programme data  Occupational Dosimetry Persons of a given installation that required personal control Persons exceeding investigation level or dose constraint Person exceeding dose limits Average doses for a given practice Classification of individuals practice, age, sex

13. IAEA Performance Indicators  Performance of Regulatory Body Average time to process application for authorization (by type of practice) Number of authorizations pending Turnover of personnel and training of personnel Inspections per year and per inspector Installation scheduled for inspection, but not inspected in given period

14. IAEA Performance Indicators  Performance of individual user installations average dose (by practice) within the installation doses exceeding constraints, investigation levels or limits (by practice) history of non-compliance, violations and sanctions list of incidents or accidents by category

15. IAEA Performance Indicators  Global indicators Number of doses exceeding investigation levels or limits, per practice Number of accidents (by category and practice) Number of enforcement actions and penalties

16. IAEA Continuous assessment by programme management Continuous assessment is part of management functions Assessment of performance is a vital part of Management & Quality Systems Demonstrates to staff importance of procedures Provides vital feedback to ensure that standards are maintained or improved

17. IAEA Continuous assessment by programme management Should involve all levels of staff and management Includes regular management supervision special supervisory audits periodic reviews of regulations, guides and polices

18. IAEA Peer Review Additional formal periodic assessments Conducted by persons who are independent of regulatory programme Results reported to level of management above those having a direct responsibility for the programme Resources for such reviews should be budgeted for in regulatory programme IAEA TECDOC 1217

19. IAEA International Audits International audits allow assessments to be conducted by persons experienced in regulatory activities in other countries Considered as on-site training opportunity, particularly during earlier stages of developments Directly related to real cases and situations

20. IAEA Conduct of an Assessment  Preparing for the assessment preparation essential Review Team only likely to be available for 1-2 weeks  Steps to prepare for assessment define areas to be reviewed appoint Review Team and designate Leader identify principal contact with Regulatory Body / Government

21. IAEA Preparation for Assessment  Steps (cont.) reach agreement on review schedule and conduct designate programme areas to Review Team members Review Team request advance information Advanced information reviewed and questions/issues identified for on-site visit Agree review plan and logistical arrangements

22. IAEA Conducting the Assessment Entrance meeting with Regulatory Body management and principal staff involved Review Team acquainted with Regulatory Body staff and familiar staff organization and responsibilities facilities and equipment Visits made to appropriate facilities of users and technical support services Inspectors’ performance observed during inspections

23. IAEA Conducting the Assessment Review elected files, procedures, etc. authorization inspection investigation enforcement Interview staff Review quantitative information Define the findings Formulate conclusions and recommendations

24. IAEA Conducting the Assessment Discuss initial findings with Regulatory Body management and principal staff Conduct exit meeting with Regulatory Body management and principal staff for discussing any discrepancies or additional issues describing preliminary findings, conclusions and recommendations prioritising each recommendation

25. IAEA Reporting the Findings Final report of Review Team should be available as soon as possible after on-site assessment Sufficient detail for a reader to understand basis for any recommendations Report should identify good practices areas for improvements Report sent to Government at level that requested it copy to head of Regulatory Body

26. IAEA Recommendations Value of assessment lies in improved programme effectiveness arising directly out of implementation of recommendations Implementation may have resource implications Review Team may recommend scaling back some areas of regulatory programme to enable funding of higher priority recommendations Prioritisation of recommendations Essential, Important, Advised

27. IAEA Performance Criteria A key part of conducting an Assessment of a Regulatory Body is to compare the performance with criteria against which the status of each element of the regulatory programme can be assessed IAEA-TECDOC 1217 includes criteria that take into account peer review missions carried to over 14 Member States participating in the Model Project – these will now be outlined.

28. IAEA Judgement of Effectiveness Performance indicators provide qualitative or quantitative information that can be compared with performance criteria Performance Criteria allow judgements to be made Performance criteria should be established for each area of regulatory infrastructure

29. IAEA Performance Criteria  Laws/Regulations and Regulatory Body Is a Regulatory Body Established and empowered? Resourced to fulfil its role? Effectively independent? Are Regulations Promulgated? Do laws/regulations define scope & provide for exemption?

30. IAEA Performance Criteria  Laws/Regulations and Regulatory Body Do regulations provide for safety & security of radiation sources? Are regulations revised and updated based on operational experience & international recommendations (eg. the BSS)? Does Regulatory Body apply a systematic approach to the fulfillment of its responsibilities? Do regulations provide occupational, public and patient protection?

31. IAEA Performance Criteria  Laws/Regulations and Regulatory Body Can Regulatory Body access information on doses to workers & discharges to environment? Has Regulatory Body established procedures for quality assurance & analysis of programme data? Is Regulatory Body self-sufficient in specialist expertise, or is it available through expert advisers/advisory committees?

32. IAEA Performance Criteria  Notification During developmental phases, has Regulatory Body prioritised sources and practices? identified and listed likely users? informed users of notification requirement? followed up non-responders? followed up contradictory information?

33. IAEA Performance Criteria  Notification During operational phase, does Regulatory Body document and log sources and users? periodically update inventory? check on accuracy of inventory ? (especially high risk sources or practices) Can Regulatory Body identify and locate radiation sources?

34. IAEA Performance Criteria  Authorization (licensing/registration) Does the Regulatory Body have criteria for the level of authorization required for sources/practices? Are application forms and guidance available? Are priorities established for reviewing applications? Is guidance developed/adopted for assessment of applications? Is such guidance easily available?

35. IAEA Performance Criteria  Authorization (licensing/registration) Does assessment of completed authorizations indicate: guidance is being followed, inspection history reviewed and radiation protection/safety issues properly addressed? clear indication of what is authorized and what requirements user must meet? such requirements appropriate? the process is adequately documented? the assessment process is completed in a timely manner?

36. IAEA Performance Criteria  Authorization (licensing/registration) Does the Regulatory Body have a register of authorization, by category of practice? Does the Regulatory Body request from users periodic radiation protection and safety reports? Has the Regulatory Body established lines of communication with senior management of authorized users?

37. IAEA Performance Criteria  Inspections Have inspection priorities, frequencies and schedules been established? in relation to hazards in consideration of past performance Are guidance documents available to inspectors and followed during inspections? Are inspections addressing correct issues? Are inspection reports clear and consistent with guidance?

38. IAEA Performance Criteria  Inspections Do inspection reports adequately address radiation safety issues and non-compliance? Are the findings communicated to in user in a clear and timely manner? Are the findings communicated to in appropriate Regulatory Body staff in a clear and timely manner? Is there documented follow-up to non-compliance (including follow-up inspections)?

39. IAEA Performance Criteria  Inspections Are inspections being performed within specified frequencies? Is there a database derived from inspection reports? reviewed and analyzed? Is there a database on occupational exposures? reviewed and analyzed? Are actions taken on the findings of such analysis?

40. IAEA Performance Criteria  Enforcement Has enforcement policy/guidance been established? Is there interaction between authorization and inspection staff regarding enforcement actions? Do inspection findings lead to timely enforcement actions? Are users required to provide details of corrective actions taken?

41. IAEA Performance Criteria  Enforcement Are user corrective actions appropriately evaluated? Are (legal) challenges to enforcement actions successfully defended? Have criteria for the application of sanctions, if appropriate, been established?

42. IAEA Performance Criteria  Emergency Response Are the roles and functions for emergency management defined and documented? Is there a national emergency plan? defines roles of national and local organization Is Regulatory Body’s plan integrated with national plan? Are there regulations/guidance on reporting of accidents and incidents?

43. IAEA Performance Criteria  Emergency Response Are adequate procedures, training programmes, equipment, facilities and support available? Are actions levels established? For severe types of accidents, are emergency exercises conducted to assess effectiveness of emergency plan? ensure capabilities remain effective? Are emergency response plans modified in accordance with lessons learned following exercises?

44. IAEA Performance Criteria  Investigation and follow-up Have dose investigation levels been established? Have procedures for investigation of accidents been established? Documentation follow-up procedures Is response to accidents timely, appropriate, in accordance with procedures? Is level of effort commensurate with health & safety significance?

45. IAEA Performance Criteria  Investigation and follow-up Are findings documented? Is appropriate corrective and/or enforcement action taken? Is effectiveness of subsequent corrective and/or enforcement actions verified? Are accidents analyzed for ‘lessons learned’ and information disseminated? Is there participation in international reporting and dissemination?

46. IAEA Performance Criteria  Technical Services Dosimetry external, internal, accredited Laboratory services Calibration services, traceable to standards Radioactive waste management facilities Training services Expert technical assistance

47. IAEA Performance Criteria  Co-ordination and Co-operation Have areas where co-ordination and co-operation is needed been identified? customs, emergencies, transport, public health Have appropriate procedures been adopted and implemented? Is there a specific arrangement with customs to prevent unauthorized import of sources? does it work ? Are international co-ordination and co-operations agreements in place?

48. IAEA Performance Criteria  Staffing and training Are staff plans, qualification requirements and training plans: in place? appropriate? funded ? Are there sufficient staff numbers? Is staff turnover rate adversely affected by compensation? lack of satisfaction? management practices?

49. IAEA Performance Criteria  Funding Is funding independent of income from authorizations, inspections and fines? Are resources adequate for Regulatory Body to perform its functions? Are facilities and equipment adequate for required tasks?

50. IAEA Performance Criteria  Information dissemination Are there procedures for collection of important radiation safety information? dissemination of such information to users, manufacturers, suppliers, international organizations? Are there procedures for rapid dissemination of information in the wake of an accident? Are there procedures for making information available to members of the public / media?

51. IAEA http://www-ns.iaea.org/default.asp IAEA Safety Requirements BSS GSR Part 3 (Interim) IAEA 2011 http://www-pub.iaea.org/MTCD/Publications/PDF/p1531_startpage.pdf IAEA Safety Requirements: Governmental, Legal and Regulatory Framework for Safety. GSR Part 1 IAEA 2010 IAEA Safety Requirements: The Management System for Facilities and Activities; IAEA Safety Standards Series No. GS-R-3 : IAEA 2006. IAEA, Organization and Implementation of a National Regulatory Infrastructure…, IAEA-TECDOC-1067, Vienna (1999). IAEA, Assessment by Peer Review of the effectiveness of a regulatory programme for radiation safety IAEA- TECDOC-1217 (2001). IAEA Safety Guide: Building Competence in Radiation Protection and the Safe Use of Radiation Sources RS-G- 1.4 IAEA 2001 IAEA Inspection of Radiation Sources and Regulatory Enforcement TECDOC 1526 (2007) IAEA-TECDOC- 1090 Quality assurance within regulatory bodies, IAEA, 1999 Further information 51