1. International Atomic Energy Agency Definition and overview of required safety documentation (e.g., safety case and safety assessment) Phil Metcalf Workshop on Strategy and Methodologies for the Development of Near Surface Disposal Facilities April 7-11, 2014, Amman Jordan

2. International Atomic Energy AgencyOVERVIEW Safety Assessment Assessment context Development and justification of scenarios Models Data Uncertainties Analysis of results Use of safety case and safety assessment results Documentation of safety case and safety assessment 2

3. International Atomic Energy Agency

4. System Description Site and waste characteristics, Safety Functions, Design Options Safety Case Context Safety objectives Safety principles Regulations Safety Strategy Isolation, Containment Passive systems, robustness Defence in depth, demonstrability Integration of Safety Arguments Demonstration of robustness, defence in depth system understanding, monitoring, etc Non-radiological Environmental Impact Operational Safety Site / Engineering Safety Assessment  Stakeholder & Regulatory Involvement  Limits, controls and conditions Management System Management of Uncertainty Iteration & Design Optimization  Management System  Post Closure Radiological Impact Scenarios Models Calculations 4CNSC Workshop 2010 - Canada

5. International Atomic Energy Agency

6. SPECIFICATION OF ASSESSMENT CONTEXT Assessment context sets the scope and content of the safety assessment Assessment context provides clear statement as to: what is being assessed why it is being assessed what decisions are to be made Strongly influenced by: stage of development of the facility regulatory and stakeholder requirements 6

7. International Atomic Energy Agency SPECIFICATION OF ASSESSMENT CONTEXT Provides information concerning: purpose of the assessment regulatory framework assessment end points assessment philosophy waste characteristics disposal system characteristics assessment timeframes 7

8. International Atomic Energy Agency DESCRIPTION OF DISPOSAL SYSTEM Includes information relevant to assessment context on: near field - e.g. waste types, waste forms, waste inventories, disposal practices, engineered barriers (chemical and physical characteristics), facility dimensions geosphere - e.g. lithologies, flow and transport characteristics biosphere - e.g. exposure pathways, climate characteristics, human habits and behaviour 8

9. International Atomic Energy Agency DEVELOPMENT AND JUSTIFICATION OF SCENARIOS Safety assessment needs to consider the performance of the disposal facility under current and future conditions Scenarios describe alternative evolutions of the disposal system based on information from the assessment context and system description What-if scenarios may be used to assess robustness and defence in depth A range of safety indicators can be considered (e.g. dose, risk, concentrations, fluxes) 9

10. International Atomic Energy Agency DEVELOPMENT AND JUSTIFICATION OF SCENARIOS Several scenario generation techniques (expert judgment, fault tree analysis, generic scenarios) No one technique is the best; the technique used should be fit for purpose Any technique should ensure appropriate level of transparency and comprehensiveness Common features: initial construction/audit of list of features, events and processes (FEPs) influencing disposal system and migration/fate of radionuclides within it scenarios screened in light of assessment context and system description 10

11. International Atomic Energy Agency NEA INTERNATIONAL FEP LIST

12. International Atomic Energy Agency CONCEPTUAL MODELS DEVELOPMENT - BACKGROUND - Once scenarios have been developed, need to analyze their consequences Do this by developing conceptual model(s) of: disposal system associated release, transport and exposure mechanisms and media Use information from the assessment context, system description and scenario development steps of the assessment approach 12

13. International Atomic Energy Agency CONCEPTUAL MODELS DEVELOPMENT - BACKGROUND - Conceptual models: describe model’s basic FEPs consider relationship between FEPs consider model’s application in spatial and temporal terms Mathematical models and computer tools: algebraic and differential equations with empirical and/or physical basis solved using computer tools using analytic and/or numerical techniques 13

14. International Atomic Energy Agency CONCEPTUAL MODELS DEVELOPMENT - BACKGROUND - Model data: disposal system parameters (e.g. facility dimensions, flow path lengths) human exposure parameters (e.g. food produce consumption rates, occupancy rates) radionuclide/element dependent parameters (e.g. sorption coefficients, transfer factors, dose coefficients) 14

15. International Atomic Energy Agency CONCEPTUAL MODEL DEVELOPMENT (1) Need to be aware of conceptual model uncertainties: data are sparse models are incomplete projections are into the unknown future Need to consider a range of credible models Models need to be fit for purpose 15

16. International Atomic Energy Agency CONCEPTUAL MODEL DEVELOPMENT (2) Interaction Matrix: Divide system into constituent components Main components go into leading diagonal elements of matrix Interactions are noted in off diagonal elements Workshop Islamabad 2014 16

17. International Atomic Energy Agency CONCEPTUAL MODEL DEVELOPMENT (3) Example Interaction Matrix

18. International Atomic Energy Agency CONCEPTUAL MODEL DEVELOPMENT (4) Influence Diagram Define system barriers Select relevant FEPs Represent on diagram Identify influences between FEPs Document FEPs and influences 18

19. International Atomic Energy Agency MATHEMATICAL MODELS DEVELOPMENT - BACKGROUND - Mathematical models are required for two primary purposes: to describe evolution of disposal system (e.g. chemical evolution in near-field, impact of climate change on disposal system) to describe transfer of radionuclides through the evolving disposal system 19

20. International Atomic Energy Agency MATHEMATICAL MODELS DEVELOPMENT - BACKGROUND - Translate assumptions of conceptual models into sets of coupled algebraic, differential and/or integral equations with appropriate initial and boundary conditions in a specified domain Equations are solved to give the temporal and spatial dependence of the quantities of interest (e.g. radionuclide concentrations and doses to humans) 20

21. International Atomic Energy Agency MATHEMATICAL MODELS DEVELOPMENT - COMPLEXITY - Particular mathematical representation of a conceptual model depends on assessment context and on process-level understanding of the ways in which FEPs can be represented As the understanding of the system is developed, more detailed models may be needed to adequately represent the system BUT models should be simple enough to be compatible with available data 21

22. International Atomic Energy Agency COMPLEXITY OF MODELS (1) Factors affecting model complexity 22

23. International Atomic Energy Agency COMPLEXITY OF MODELS (2) Some simplification is generally required to translate the concepts of conceptual model into mathematical terms Can take several forms: simplification of geometry or structure (e.g. considering transport in 1D and homogeneous and isotropic medium) omission/simplification of processes (e.g. neglecting kinetic terms in chemical reactions) 23

24. International Atomic Energy Agency ANALYTICAL METHODS Can provide exact solutions to the flow and transport equations Useful for: screening level assessments when site data are sparse and uncertain sensitivity and uncertainty analysis verifying more complex models BUT only developed for simple cases with homogeneous spatial domain, steady flow and 1-D advection/dispersion 24

25. International Atomic Energy Agency NUMERICAL METHODS Discretize the spatial domain into cells and the resulting set of algebraic equations is solved by iteration, matrix methods or a combination of the two Advantages include: Easy handling of spatial and temporal variability and complex geometry and boundary conditions 2-D and 3-D transport problems can be solved BUT resource (time, money and data) intensive 25

26. International Atomic Energy Agency COMPUTER CODES (1) Solution of mathematical models is usually achieved by implementing one or more computer codes Need to consider software design of code(s) - should be conducted within an appropriate software quality assurance system 26

27. International Atomic Energy Agency COMPUTER CODES (2) Codes may be: proprietary codes - advantage of being previously developed and checked, history of application to a range of cases, but not necessarily appropriate to the problem modified codes - need to be developed and checked, however tailored to the needs of the specific problem specifically developed for implementation of the chosen mathematical models – same advantages/disadvantages as modified codes 27

28. International Atomic Energy Agency SELECTION OF COMPUTER CODES (1) Need to ensure that the code(s) used are fit for purpose Factors to consider: Assessment context (scoping vs detailed calculations) Resource availability (time, money and data) Nature of the processes to be modelled (e.g. fractured vs porous medium) Relative importance of the processes 28

29. International Atomic Energy Agency SELECTION OF COMPUTER CODES (2) Ideally the codes should be chosen to be consistent with the conceptual and mathematical models, and not vice versa However, for practical reasons, the conceptual and mathematical models are often developed with the code already selected If this happens then it is important to document the constraints that the pre-selection of the code places on the conceptual and mathematical models 29

30. International Atomic Energy Agency VERIFICATION ‘Test problems’ show that equations are solved satisfactorily in the computer codes Verify calculation methods Can be feasible and should be used for confidence building BUT a model is only verified for the specific problems which have been considered 30

31. International Atomic Energy Agency CALIBRATION Compare model estimates with site specific field observations. Can include modifying: input data and boundary conditions conceptual and mathematical models Limitations: A model can only be calibrated for the same temporal and spatial scales as the field measurements Different conceptual models may show similar agreement with observed data 31

32. International Atomic Energy Agency VALIDATION Production of credible results under a range of conditions Validation over limited timescales is achievable Not possible for long term evolution of a specific site, since there are insufficient data Natural analogues and other assessments/ experience may be useful 32

33. International Atomic Energy Agency DATA COLLECTION (1) Many kinds of data needed during the modelling process: Waste inventory and form Design data Site data (geosphere, biosphere) Flow and transport data Data are often unavailable: Many kinds of data cannot be collected or are very costly Data relate to small space and time scales Information about the future unavailable 33

34. International Atomic Energy Agency DATA COLLECTION (2) Collection of all data is impossible and any attempt would lead to an enormous expenditure of resources Recognition that safety assessment has unique characteristics: Goal of safety assessment is to make decisions, not to predict the future The decision is about reasonable assurance of safety Predictive capability is unnecessary/impossible 34

35. International Atomic Energy Agency SOURCES OF UNCERTAINTIES 35

36. International Atomic Energy Agency DATA/PARAMETER UNCERTAINTY Uncertainty in data and parameters used as inputs in modelling Lack of specific data Natural variability (spatial and/or temporal) variability in some parameters 36 Workshop Islamabad 2014

37. International Atomic Energy Agency MODEL UNCERTAINTY Uncertainty in conceptual, mathematical and computer models used to simulate the disposal system behaviour and evolution 37 Workshop Islamabad 2014

38. International Atomic Energy Agency FUTURE UNCERTAINTY Uncertainty in evolution of disposal system over timescales of interest 38 Workshop Islamabad 2014

39. International Atomic Energy Agency SUBJECTIVE UNCERTAINTY Uncertainty due to reliance on expert judgment 39 Workshop Islamabad 2014

40. International Atomic Energy Agency 40 APPROACHES FOR UNCERTAINTY MANAGEMENT Awareness – be aware of all major locations of uncertainty Importance – determine relative importance of various sources of uncertainty using sensitivity analysis Reduction – reduce uncertainties, e.g. through further data collation Quantification – quantify effects of uncertainties on model output using sensitivity analysis Workshop Islamabad 2014

41. International Atomic Energy Agency MANAGEMENT OF SCENARIO AND MODEL UNCERTAINTY Scenario analysis: describes alternative futures and allows for a mixture of quantitative analysis and qualitative judgements Conceptual model: consider alternative conceptual models and collect further data Mathematical/computer model: use model verification, calibration and validation, and range of models 41

42. International Atomic Energy Agency MANAGEMENT OF MODEL UNCERTAINTY Example Results from BIOMOVS II 42 Workshop Islamabad 2014

43. International Atomic Energy Agency MANAGEMENT OF DATA/PARAMETER UNCERTAINTY Approaches can be used: conservative/worse case approach best estimate and what if sensitivity analysis probabilistic Can also be used to address future and model uncertainties 43 Workshop Islamabad 2014

44. International Atomic Energy Agency PROPAGATION OF UNCERTAINTIES 44 Workshop Islamabad 2014

45. International Atomic Energy Agency CONSERVATIVE / WORST CASE APPROACH Use pessimistic parameter values to overestimate impact Danger of being so pessimistic as to be worthless and misleading Difficult to define the worst value, and prove that this is the worst one. Not always obvious what is conservative for a particular combination of parameters, exposure pathways and radionuclides 45 Workshop Islamabad 2014

46. International Atomic Energy Agency MODELS – SUMMARY AND CONCLUSION Need to make process of formulating and developing models formal, defensible, and transparent to independent review Process consists of: generation of conceptual models using information from the assessment context, system description and scenario generation representation of conceptual models and associated processes in mathematical models implementation and solution of mathematical models in computer codes Workshop Islamabad 2014 46

47. International Atomic Energy Agency MODELS – SUMMARY AND CONCLUSION Models should be: as simple and easy to use as possible whilst including enough detail to represent the system’s behaviour adequately for the purpose of the assessment consistent with assessment context consistent with data availability A simple modelling approach is likely to be more efficient, easily understandable and justified Workshop Islamabad 2014 47

48. International Atomic Energy Agency ANALYSIS OF RESULTS Compare results with criteria defined in assessment context Analyze the extent and implications of uncertainties Confidence can be built using various approaches (e.g. transparent, logical and well documented reasoning, multiple lines of reasoning, treatment of uncertainties) Workshop Islamabad 2014 48

49. International Atomic Energy Agency ITERATION The safety assessment process is iterative, although iteration need only proceed until the assessment is judged fit for purpose Iteration promotes: consideration of improvements to and optimisation of the disposal system confidence in the understanding of the main safety related parameters and the robustness of the disposal system under the assumed scenarios collection of relevant data Workshop Islamabad 2014 49

50. International Atomic Energy Agency DOCUMENTING THE SAFETY CASE Executive summary Introduction and the safety case context Safety strategy Safety assessment Synthesis and conclusions Follow-up programmes and actions Public involvement Requirements on the documentation of safety assessment Traceability Workshop Islamabad 2014 50

51. International Atomic Energy Agency USE OF THE SAFETY CASE Comparison of Options Prioritization of Site Characterization and R&D Facility Design and Operation Licensing Derivation of Limits and Conditions Monitoring Staff Training Workshop Islamabad 2014 51

52. International Atomic Energy Agency REGULATORY REVIEW PROCESS Objectives and attributes of the regulatory review process Managing the review process Conducting the review and reporting review findings Workshop Islamabad 2014 52

53. International Atomic Energy Agency