1. Overview of Deterministic Safety Analysis: Sensitivity & Uncertainty Analysis, Q.A. (Part. 3) IAEA Training Course on Safety Assessment of NPPs to Assist Decision Making Workshop Information IAEA Workshop Lecturer Lesson IV 2_3 City, Country XX - XX Month, Year

2. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 2 Sensitivity & Uncertainty Analysis –A best-estimate analysis must include an uncertainty analysis (UA). –Results calculated in the analysis are given an “uncertainty interval”, i.e. one in which it is expected with high probability to find the “true” result. –Uncertainties in the input data of the analysis and in the calculational devices (codes) are propagated to the calculated results.

3. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 3 Sensitivity & Uncertainty Analysis –Sensitivity analysis (SA): “what-if” analysis. Impact of changes of some attribute (input data, models, numerics, etc) of an analysis or calculation on the results. –Relation among SA and UA.

4. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 4 Sensitivity & Uncertainty Analysis –Basic ingredients of an UA: Establishing the input uncertainties:  From input parameters  Associated with the calculational device(s): from models and calculational patterns. Combine input uncertainties to find the results uncertainty. –When the number of uncertain attributes is very high  selection of those most influential (on important results). One tool  Sensitivity analysis.

5. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 5 Sources of Uncertainty in a Safety Analysis –U on initial (i.e. previous to the accident) conditions U of the plant, and on boundary conditions in the transient. –U on physical parameters describing the plant, its systems and materials: geometry, state and transport properties, etc. –U coming from the inaccuracy of physical models and correlations, including errors due to scaling and extrapolation. –Errors coming from numerics of the codes. –Errors due to omissions in the models or the noding. –U due to the computer and the user.

6. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 6 Sources of Uncertainty in a Safety Analysis –De-compensation of errors, arising from scaling or extrapolation.

7. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 7 CSAU Methodology –USNRC sponsored the development of a methodology for uncertainty quantification in complex codes calculations: Code Scaling, Applicability and Uncertainty Evaluation Methodology (CSAU). –Formerly developed for BE LBLOCA analyses, but can be applied to other scenarios. It is a study of: Capability of the code to scale up phenomena from small- scale facilities to nuclear plants. Applicability of the code to a particular scenario + plant. Uncertainty on the important calculated results (e.g. PCT).

8. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 8 CSAU Methodology Systematic approach with three succesive phases : 1. REQUIREMENTS AND CAPABILITIES Scenario specification: temporal windows and most important phenomena. Plant selection. Phenomena and processes identification and ranking (by impact on the safety criteria for the scenario). PIRT is established: it will guide the uncertainty quantification. PHENOMENA and processes found by examining exp data and code simulations. RANKING through experts opinion, decision-making methods (AHP) and calculations.

9. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 9 CSAU Methodology (Req. and capabilities) Frozen code version selection: changes allowed for correction only. Provision of complete code documentation, including MC/QE: Models and Correlations Quality Evaluation. Code’s capabilities are established:  Field equations : can model global processes (such as multi-D flows, boron injection, etc) ?.  Closure equations: do they enable the code to calculate important processes ?.  Numerics.  Structure and nodalization: do they allow the modelling of important design characteristics ?. Determination of code applicability, by comparing code’s capabilities to scenario requirements

10. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 10 CSAU Methodology (Req. and capabilities) 2. ASSESSMENT AND RANGING OF PARAMETERS –Establishment of Assessment Matrix, that must include both separate and integral effects tests. Necessary for evaluating: Code accuracy to calculate important phenomena (isolated or interacting). PIRT is used. Code capability to scale-up the phenomena to plant conditions. Counterpart tests are quite useful. Influence of nodalization. –Plant nodalization definition: compromise between economy and capture of the phenomenology. An iterative process is employed. Where the data base is insufficient to judge nodalization, a separate bias may be added in determining uncertainty.

11. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 11 CSAU Methodology (Assess. & Rang of parameters) –Definition of code and experimental accuracy: Individual uncertainty contributions (arising from code and experiment) are estimated from experimental data. The actual probability distribution may not be easily quantified. The individual uncertainty of each contributor is input to the plant model, and the effect upon important safety criteria evaluated by separate plant calculations. SET and IET simulations can be a confirmatory support of these estimations. –Determination of effect of scale, quantifying it for bias and deviation as a contributor to overall uncertainty.

12. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 12 CSAU Methodology (Assess. & Rang of parameters) –Determination of effect of scale, quantifying it for bias and deviation as a contributor to overall uncertainty. Lack of data  bounding analysis to provide a separate uncer. Bias MC/QE document + code assessment reports  identification of closure relations and evaluation of the capability to scale- up phenomena from PIRT. Test facility scale distortions do not have the same effect throughout a transient. Results not affected by scale are used directly to evaluate the total uncertainty; those affected are subject to further examination.

13. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 13 CSAU Methodology 3. SENSITIVITY AND UNCERTAINTY ANALYSIS –Determination of the effect of reactor input parameters and state Uncertainties come from plant operating state at the initiation of the transient (e.g. State of fuel is a function of the burnup history and of the original manufacture tolerances) and also from process variables Uncertainties quantified through experimental data and/or analytical studies. Biases and distributions, or sparate (bounding) biases. –Performance of plant sensitivity calculations: sensitivity of primary safety criteria parameters to various plant operating conditions that arise from uncertain initial state; also to scale and process variables.

14. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 14 CSAU Methodology (SA & UA) –Determination of combined bias and uncertainty: The individual uncertainties (coming from code limitations, scale effects, input variations, etc) are combined. Proven technique: pdf determination of a safety parameter through Monte Carlo sampling of a response surface that “substitutes” the code. Economic considerantions can recommend take an individual uncertainty as a separate bias based on bounding sensitivity calculations. –Determination of total uncertainty: an error band, or a statement of probability for the limiting value, is given for each safety criteria parameter.

15. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 15 CSAU Methodology (SA & UA)

16. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 16 CSAU Methodology (SA & UA)

17. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 17 CSAU Methodology (SA & UA)

18. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 18 – – Uncertainty Methods Study (UMS): supported by Committee on the Safety of Nuclear Installations (CSNI) from OECD Nuclear Energy Agency. Aimed to compare several uncertainty analysis methodologies, from: AEA Technology (UK) GRS (Germany) CEA (France) ENUSA (Spain) Pisa University (Italy) Uncertainty Analysis - UMS

19. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 19 – – Five methods: Pisa method: extrapolation from integral experiments. GRS, IPSN, ENUSA methods: identify and combine input uncertainties, using subjective pdfs. AEAT method performs a bounding analysis. – – The five methods have been applied to the simulation of a cold leg SBLOCA experiment in the ROSA-IV Large Scale Test Facility (LSTF). Uncertainty Analysis - UMS

20. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 20 Uncertainty Analysis - UMS

21. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 21 Uncertainty due to User Effect –Any differences in calculations that use the same code version and the same specifications (e.g. Initial and boundary conditions) for a given plant or facility. –Code users have a significant influence on calculation results. –User effects have been identified in numerous publications as the origin of many calculation failures. –In the many Standard Problems proposed by CSNI appear the influence of the code user. –The new generation of the advanced computer codes have reduced, but by no means suppress the user effect.

22. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 22 Uncertainty Due to User Effect – – Reasons: Code user guidelines not fully detailed or comprehensive Experienced users may overcome code limitations adding “engineering knowledge” to the input deck. System nodalization: many times, the user must model 3D geometries using 1D components. Application of steady state qualified models to transient conditions. Lack of understanding of the code capabilities and limitations.

23. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 23 Uncertainty due to User Effect – – Reasons: Code options and physical model parameters: there are several options (model and correlations) for the user to choose. Uncertain parameters (e.g. Pressure loss coefficients) must also be specified by the user. Effect of compiler and computer Lack of information about facilities and experiments Error bands and the values of initial and boundary conditions badly defined

24. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 24 Uncertainty due to User Effect – – Reasons: Input parameters for system characteristics: e.g. Distribution of heat losses. Input parameters for system components: a number of empirical models (pumps, valves,etc) are specified by the user, sometimes based on extrapolation from scale devices. Specification of initial and boundary conditions: sometimes users fail to obtain a steady state previous to the transient. Specification of state and transport properties.

25. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 25 Uncertainty Due to User Effect – – Reasons: Time step size choice: if a code is using an explicit numerical scheme, the results may vary significantly with the time step size. QA guidelines: should be followed to check the correctness of the values in the input decks, despite the automatic consistency checks provided by the code.

26. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 26 Quality Assurance in Development of Evaluation Models EVALUATION MODEL DEVELOPMENT AND ASSESSMENT PROCESS (EMDAP) – – Draft Regulatory Guide DG-1096, released Dec. 2000, gives recommendations for the process of development and assessment of Evaluation Models (transient and accident analysis methods). – – One of the basic principles during the EMDAP is following an appropriate Quality Assurance (QA) protocol. – – QA standards are a key feature of the development and assessment process.

27. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 27 Quality Assurance in Development of Evaluation Models – – QA Plan: covers the procedures for design control, document control, software configuration control and testing and error identification and corrective actions used in the development and maintenance of the EM. – – The QA Plan also ensures adequate training of personnel involved with code development and maintenance as well as those who perform the analyses.

28. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 28 Quality Assurance in Development of Evaluation Models – – An appropriate QA protocol must be established early in the development and assessment process. – – Development, assessment, maintenance and application of an EM are activities related to the requirements of the Appendix B (“Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants”) to the US code of federal regulations 10 CFR 50: Section III (”Design Control”) of Appendix B is a key requirement for this activity, stating that design control measures must be applied to reactor physics, thermal, hydraulic and accident analyses.

29. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 29 Quality Assurance in Development of Evaluation Models – –Section III states: “The design control measures shall provide for verifying or checking the adequacy of design, such as by the performence of design reviews, by the use of alternate or simplified calculational methods, or by the performance of a suitable testing program”. – –Section III also states that design changes should be subject to appropriate design control measures. – –Section V: requires documented instructions, e.g., user guides – –Section VI: address Document Control – –Section XVI (“Corrective Action”): errors must be promptly identified and corrected, and corrective actions must be taken to preclude repetition. All this must be documented.

30. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 30 Quality Assurance in Development of Evaluation Models Section XVII: address QA Records – – In order to fulfill the App. B requirements, independent peer reviews should be performed at key steps in the process. This is an important point when complex codes are involved. – – PEER REVIEW: an evaluation technique in which software requirements, design, codes or other products are examined by persons whose rank, responsibility, experience and skill are comparable to that of the authors.

31. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 31 Quality Assurance in Development of Evaluation Models – – Composition of a peer review team: programmers, developers and users, and independent members with recognized expertise in relevant engineering and science disciplines, code numerics and computer programming. – – Experts in the Peer Review team not directly involved in the EMDAP can enhance the robustness of the EM and can be valuable in identifying deficiencies that are common to large system analysis codes.

32. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 32 Quality Assurance in Development of Evaluation Models – – In the early stages of EM development it is recommended that a review team be convened to review EM requirements, which are the basis of the EMDAP. Such requirements are: Analysis Purpose, Transient Class and Power Plant Class. Figures of merit (i.e. Quantitative standards of acceptance) used to define acceptable answers for a safety analysis. Systems, components, phases, geometries, fields and processes that must be modeled. Ranking of key phenomena and processes.

33. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 33 Quality Assurance in Development of Evaluation Models – – Configuration control practices throughout the development process: Adopted to protect program integrity and allow traceability of both the code version and the plant input deck. Responsibility for these functions should be clearly established. At the end of the process, only the approved, identified code version and plant input deck should be used for licensing calculations.

34. IAEA Training Course on Safety Assessment of NPPs to assist Decision Making 34 Quality Assurance in Development of Evaluation Models – – The QA Plan documentation must include process that address all of the topics described. – – A Safety Analysis must be adequately documented: All sources of data referenced and documented. Whole process recorded and archived to allow independent checking. Results of the SA structured and presented in an appropriate format to provide a good understanding of the course of the events, and to allow easy checking of the individual acceptance criteria.