1. www.inl.gov RELAP5-3D 2011 Solver Improvements Dr. George L Mesina RELAP5 International Users Seminar 2011 July 25-28, 2011

2. Outline of Presentation Ten BPLU User Problems Categories of Problem & Size Solutions Pre-Transient Algorithm, Issue, and Fix 198 & 199 cards

3. Previously Unresolved BPLU User Problems (06007, 1/06) - Takashi Nagae, INSS – On version 2.2.4, the code runs for both the initial calculation and the restart calculation. On version 2.3.6, the initial calculation runs. Restart fails in BPLU pre-transient processing; with message: 'inconsistency in reordering, check dimensions', 'bpform -1'. (06010, 1/06) - Walt Weaver, INL – DESCRIPTION: The BPLU solver overwrites data in scratch space when executing a large problem coupled to Fluent. Problem traced to having more than 4 PVM coupling junctions. (06013, 2/06) - Cliff Davis, NL – DESCRIPTION: While working on UP#06007, it was discovered that problems with large MULTID components (8x8x8 and 6x10x10) would not run with the nearly-implicit solution scheme with BPLU.

4. Previously Unresolved BPLU User Problems (06035, 6/06) – DESCRIPTION: In running a system calculation using 3D hydro components, calculation fails in BPLU pre-transient processing. – STATUS: Diagnostics in subroutine BPARAM reveal the arrays are dimensioned too small. Workaround is to use the MA18 solver. (07042, 10/07) - Angelo Frisani, Yassin Hassan, Texas A&M – DESCRIPTION: A calculation with six 3D hydro components fails with both default (BPLU) and MA18 solvers. With FAST common variable 'lfsiz' increased from 3.5M to 22M, code runs through input processing, but fails on first time step with either solver. (08004, 1/08) – DESCRIPTION: MA18 solver overwrites scratch database for a large problem coupled to a CFD code. MA18 was employed because more coupling junctions can be used with it than BPLU (see UP#06010).

5. Previously Unresolved BPLU User Problems (08025, 8/08) – DESCRIPTION: A PVM problem fails in subroutine EQFINL in version 2.4.1. Traced BPLU subroutine BORBND; there appears to be a data overwrite of the variable ‘ihld1’. – STATUS: Created input decks (modification of installation run/Pvm directory files pvmcorex.ii, pvmcorep.i, pvmcorec.i to use 6 coupling junctions). Related to UP 07042, more than 2 PVM coupling junctions. (09024, 5/09) - Paul Bayless, INL – DESCRIPTION: The LOFT L2-5 input deck has a 2-D component for the downcomer and 3-D component for the core. The nearly- implicit with the 1-D option (CCC0001 word 7=1), ran on version 2.9.2 if the flag was set for just the 2-D component, or for both components. However, with the flag set for only the 3-D component, it failed at the beginning of the transient. This may affect all earlier versions.

6. Previously Unresolved BPLU User Problems (09047, 8/09) - Peter Cebull, INL – DESCRIPTION: Version 2.9.2 fails on an ATR SBLOCA deck with a message from BPLU input subroutine BPPART written to the screen: “Chain connects to a previous chain.” The problem runs to completion using MA18. The problem occurs also in version 2.4.1, so it predates the F90 conversion. Affects some earlier versions. (10030, 3/10) - George Mesina, INL – DESCRIPTION: In running a modified 3dflow.i installation problem (3dflow.i, that has 9 junctions connected to 1 time-dependent volume), version 2.9.4 fails with a thermodynamic property failure at minimum time step with both BPLU and MA18. All 18 cases run with the MA18 on RELAP5/MOD2.5. Indicates a non-solver issue. BPLU (with the fix to UP#06061) fails on vapor case 14 and MA18 fails on vapor case 8. All vapor cases with both solvers have large cross flow and take more time steps than the other cases.

7. Categories of Unresolved BPLU User Problems CategoryNumber Insufficient memory (overwrite)06010, 08004, 08025 Inconsistent reordering @ pre-transient06007, 06013, 06035, 07042, 09047 Error during transient09024, 10030 1. The insufficient memory issues resolved by Fortran 95 conversion. – BPLU had been fit into insufficient MA18 arrays by Wagner – BPLU now has its own memory, sized to its own needs. – 06010, 08004, 08025 resolved 3. Transient failures – UP 09024 is resolved. – UP 10030 is solved. – Physics issue remains: runs with MA18 on MOD2.5 but not 3.0 Decks unavailable for 06010, 08004

8. Pre-Transient Processing Algorithm Character of BPLU reordering (renumber the variables) – Create Arrow-shaped matrix for special BPLU solution method – Reorderings differ by arrowshaft BW and arrowhead thickness – Estimate reordering’s Operation Count (OC). Include green fill-in – Smallest OC – fastest runtime BW X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X

9. Pre-Transient Processing Issues Original pre-transient algorithm creates many reorderings – Seeking lowest operation count – fastest runtime – Kills entire run if any single reordering is inconsistent Inconsistent means two variables are given same number in reordering – Variables are either volume ΔP (SI) or junction velocities (NI). – Inconsistent implies some row of matrix is all zereos Singular – Cannot solve system Graph theoretic BPLU output (for an inconsistent renumbering) makes no sense to average user. – Was meant to help debug code errors.

10. Fix for BPLU Pre-Transient Failure Ignore bad reordering(s), proceed to next iteration of BW loop – Also, do not calculate Opcount for bad renumbering Implementation via 2 logical variables – Required replacement of STOP statements at error messages Use RETURN after setting a logical variable – Suppress graph theoretic messages (for inconsistent renumbering) – Code only stops with error message if no consistent renumbering is found. Modification resolves UP 06007, 06013, 06035, 07042, 09047 Deck unavailable for 07042

11. Improvements to BPLU Pre-Transient Reduce pre-transient processing time. – Can take longer to set-up matrix than to run the transient in some 3D input models. Improvement 1 – Leave BW loop if matrix shape is all shaft with no arrowhead (NBORDer = 0) Improvement 2 – minimum memory renumbering elimination – Original algorithm calculates and saves parameter pair that uses least memory. – Prevented overrunning old FA array of Fortran 77 coding – Unnecessary in Fortran 90 coding, overrun detected and prevented by allocate statement status These changes do not change in matrix – Calculations will remain the same.

12. 198 Card – Less Time in Input Processing For deck with unchanged TH nodalization, matrix is same every time. – BPLU parameters in printed-output file for each system – Why not reuse parameter pairs? Format: 198 W1 W2 W3 W4 W5 – W1 = ASCII name for solver (BPLU, PGMRES,...) – W2 = System identifier (TH1, TH2, etc.) – W3 = Parameter 1 (for BPLU, this is degree) – W4 = Parameter 2 (for BPLU, this is bandwidth) – W5 = Search length (BPLU: Reexamine bandwidths W4 ± W5/2) – The format repeats in groups of 5. Use zero for unneeded value.

13. 198 Card Continued New subroutine, RSOLVER, called from RNEWP Modify BPARAM – If W5 zero, simply build renumbering with user’s values – If W5 non-zero, restrict bandwidth search loop to values indicated Will be used also for PGMRES and other solvers CAUTION, WARNING, DANGER – Must remove 198 card if nodalization changes!

14. 199 Card – Generic Debugging Could be used to activate and deactivate Doug Barber’s detailed-log Generic 199 card is a standard RELAP5-3D input card. Its format is: 199Word1Word2Word3Word 4 Word 1 Character string that names a subroutine (group) Word 2Keyword that indicates what aspect (to debug) Word 3Advancement number to activate (debugging) Word 4Advancement number to terminate (debugging) This feature was developed for the DTSTEP Test Matrix, but can be applied by a developer for debugging any program features.

15. Summary Solver-related User Problems have been solved The pre-transient processing is being improved to run faster The 198 card will allow user interaction with the solvers through the solver parameters. 199 card could control the detailed log activation.