TRANSP and Frameworks Presented by D. McCune at the Joint ORNL/IU Workshop on Computational Frameworks in Fusion, June 6, 2005

Outline of Talk TRANSP Background Information: –What is TRANSP; Applications of TRANSP. –Users, Developers, Funding profile. FusionGrid TRANSP. TRANSP Code Generators – Python. TRANSP Makefile Generators – Development Environment. Requirements for Frameworks.

What is TRANSP? “1.5D” time dependent tokamak transport simulation tool: –2d axisymmetric tokamak reduced to 1d by using standard “flux coordinate” methods. Major application: analysis of experimental data from tokamaks. Major strengths: –Physics: e.g. validated fast ion models. –Operations: effective workflow & support. Under continuous development since 1978!

Applications of TRANSP Transport and confinement studies (with or without ITB, H-mode, etc.). Neutral beam heating simulation; computation of fast ion distributions. RF heating and current drive simulations. Diagnostics simulations. Predictive calculations for experimental proposals.

Applications of TRANSP (cont.) Many groups report use of TRANSP results in conference papers and refereed journal articles. Two examples follow: –Tritium Transport on JET (I. Voitsekovitch). –NPA Simulations on NSTX (S. Medley). Numerous other examples could have been chosen.

TRANSP at JET Trace Tritium: Thermal Tritium Transport: 14 MeV neutrons measured along 19 chords (left) are reproduced in TRANSP by using a proper adjustment of Tritium transport coefficients (right, 6 bottom chords are shown) 2 techniques have been used: (1) Dt and Vt are adjusted in TRANSP; (2) novel use - combined TRANSP/SANCO analysis - plasma reactivity is extracted from TRANSP using 20 runs and transmitted to SANCO for automatic estimation of Dt and Vt by minimising  2 -value Publications: K.-D. Zastrow and JET team (EPS 2004, PPCF, 2004 in press), I. Voitsekhovitch, et al., Trace Tritium transport in H-mode JET plasma with different density, JET pinboard, waiting for clearance, to be submitted to Phys. Plasmas and also in EPS 2004; J. Mailloux et al, EPS 2004; P. Belo et al., EPS 2004; D. Stork and JET team, IAEA 2004; D. Stork and JET team, APS 2004

MHD-Induced Fast Ion Losses in NSTX S. Medley et al., Nucl. Fusion 44(2004)

TRANSP Developers and Users PPPL TRANSP team: –R. Andre, A. Brooks, F. Dahlgren, E. Feibush, K. Indireshkumar, S. Klasky, L. P. Ku, C. Ludescher, D. McCune, L. Randerson ~5 FTE for TRANSP & related efforts. –Major focus on production & support… User Sites: –Culham (MAST) –GA (DIII-D) –IPP/Garching (Asdex-U) –JET –MIT (C-Mod) –PPPL (NSTX) –PPPL (Collaborations)

FY05 PPPL TRANSP Funding NSTX/MAST (approx FTE, 60%) DIII-D collaboration (0.9 FTE, 25%) C-Mod collaboration (0.25 FTE, 7%) JET collaboration (0.25 FTE, 7%) Tokamak Projects Total: 3.55 FTE R&D Projects (1.45 FTE): –PTRANSP (predictive upgrade), 0.6 FTE –SciDAC FusionGrid (2 year renewal grant), 0.85 FTE

Outline of Talk TRANSP Background Information. FusionGrid TRANSP: –TRANSP as a Computational Service. –TRANSP Workflow; Workflow Automation. –Future: Linked, Distributed Services. TRANSP Code Generators – Python TRANSP Makefile Generators – Development Environment. Requirements for Frameworks.

FusionGrid TRANSP TRANSP runs as a service on PPPL Linux cluster machines. –User authentication via Globus. –Credit to SciDAC Fusion Collaboratory Project. Users submit run requests via supported client software tools. Users can monitor system load and watch progress of runs, access log files, etc. Website:

TRANSP Client Tools, Services Prepare Input Data. Submit runs. Monitor run progress; view log files. Fetch interim data. Option to cancel run prior to completion. Explore, visualize output data. Crashed runs debugged by TRANSP support staff at PPPL.

access via

FusionGrid TRANSP Annual Run Production *FusionGrid TRANSP Operational Since September 2002

FusionGrid TRANSP Workflow Preliminary data Analysis and Preparation (largely automated) Diagnostic Hardware Experiments (CMod, DIII-D, JET, MAST, NSTX) signals {f(t), f(x,t)} Plasma position, Shape, Temperatures, Densities Field, Current, RF and Beam Injected Powers. TRANSP Analysis*: Current diffusion, MHD equilibrium, fast ions, thermal plasma heating; power, particle and momentum balance. Experiment simulation Output Database ~ signals {f(t), f(x,t)} Visualization Load Relational Databases Detailed (3d) time-slice physics simulations (GS2, ORBIT, M3D…) *FusionGrid TRANSP now executes on PPPL servers; GS2 and other grid services are planned for the future. Pre- and Post-processing at the experimental site…

Workflow Automation User-programmable automated data preparation is key to system productivity. –IDL (many legacy tools in fusion program). –UREAD/SGLIB and other legacy fortran tools*. Also important: –convenient tools for submission and web- based monitoring of runs. –User-programmable automated visualization, database loading, and further post-processing. D. McCune 23 Apr 2004 *available at

TRANSP Workflow Caveats The software technology used is ancient. –But: familiar to current users. The workflow volumes (dataset sizes) are small, so far. –So, no motivation yet for transition to high performance solutions. The potential for generation of larger datasets exists. –Current practice may be a limiting factor.

TRANSP FusionGrid Future: Linked, Distributed Services. Serial TRANSP at PPPL Parallel RF Wave Solver (TORIC at MIT) Parallel Monte Carlo Fast Ion Model (NUBEAM at PPPL) Multi-site Globus Forwarded Authentication Possible issues: Firewalls Resource sharing Queue wait time. Data transfer times.

Outline of Talk TRANSP Background Information. FusionGrid TRANSP. TRANSP Code Generators – Python: –Ad hoc data specification languages. –Python code generators. TRANSP Makefile Generators – Development Environment. Requirements for Frameworks.

Code Generators Many i/o interfaces of TRANSP are written by code generators (Python scripts). –Measured once to be ~25% of fortran lines. Code generator input: list of data items. Code generator output: –Anything needing to be driven by the list: Fortran namelists, “state” or restart file i/o routines, documentation, allocate/deallocate routines, default/initialization routines, plot labeling routines, Fortran-90 type definition modules, debugging…

Ad Hoc Languages Custom, ad hoc, requirements-driven “language” for each list. Example: NTCC NUBEAM “nbspec.dat”*: –Sections: Array_Dimensions, Constants, Inputs, Internal, Outputs. –“nbigen.py” writes code for internal data structures, public modules defining input/output data structures, input default settings, state file i/o, html documentation, … *see

Ad Hoc Languages Section: Inputs # from nbspec.dat… … … Block: Atomic ! Atomic physics controls (defaults shown). D xdepmod = 1.0d0 ! Anomolous deposition opacity adjust. L nlbbcx =.TRUE. !.FALSE. to disable beam-beam ch.ex. … … Section: Outputs … … orb%DAS sorbn0(mj,mig,mibs) ! Thermal neutral source ! due to deposition or charge exchange recapture of fast ! neutrals, vs. x, thermal ion, fast ion; “orb” MPI_reduce. Key: D means REAL*8, L means LOGICAL A means ARRAY, S means “in State File”,…

Code Development Automation Code generators organize i/o and “book- keeping” aspects of code. –They make it easy to add inputs & outputs: (a) edit specification file. (b) run generator python script. –Less error prone than hand-maintained methods, where list-driven requirements exist. –Allows greater focus on physics coding. Key to long term success of TRANSP.

Python Chosen for Code Generators Early TRANSP code generators were written in fortran-77 or C, but… Much easier / faster to work with Python. –Python “dictionary” and “list” data structures perfect for code generator application. –Fast prototype / debug cycle. –Python has excellent error reporting. –Python has excellent tutorial and reference documentation (e.g. O’Reilly books).

Outline of Talk TRANSP Background Information. FusionGrid TRANSP. TRANSP Code Generators – Python TRANSP Makefile Generators – Development Environment: –Makefile generator system. –Resulting benefits for code developer. –Issues / Limitations. Requirements for Frameworks.

TRANSP Developers do not edit Makefiles… Makefiles are written by makefile generator for: –Subroutine libraries (179). –Executables (220). Supported by script database with information on various fortran compilers… Library makefiles based on contents of library source directory: –F90 module dependency analysis included.

TRANSP Makefile Generator Linkage for executables defined by separate file containing ordered list of objects and libraries: > cat foo_exe.link foo_main.o foo_sub1.a foo_sub2.a $L_NETCDF $L_LAPACK $L_FFTW Elements:.o – from TRANSP source..a – from TRANSP source. $L_ -- non-TRANSP dependency.

Command Line Interface cmsmms -- generate makefile for library. uplib -- update (build) library. makelink -- generate makefile for executable. uplink -- update executable. These commands build “standard” (i.e. non-debug) libraries and executables.

Debug builds dbxadd -- add single source to debug list. –F90 module dependencies added automatically. dbxaddlib -- add entire library to debug list. uplink debug – build debug executable. –Makefile for custom debug executable is built; only named items are debug compiled.

Benefits Relieves developer of makefile maintenance requirement. –Hand built makefiles become nightmares… Precise control of generation of debug executables. Easy to learn. Saves time.

Drawbacks / Limitations Monolithic – not easily transferable outside the TRANSP project. Too much dependency on ancient csh scripts (awkward to maintain). Certain peculiarities, due to a lapsed VMS compatibility requirement, persist… We are looking at SCONS for possible upgrade or replacement… Advice of CS experts would be welcome!

Outline of Talk TRANSP Background Information. FusionGrid TRANSP. TRANSP Code Generators – Python TRANSP Makefile Generators – Development Environment. Requirements for Frameworks. –Generic Requirements. –Requirements Checklist. –Compatibility with CS Research (?).

Generic Requirements Non-invasive; light-weight. –No motivation for massive project to convert an existing, working system. –Evolutionary method needed for introduction of new technology. Simple things must be simple, complicated things must be possible. Well engineered. –Robustness, ease-of-use, documentation…

Requirements Checklist Technical assessment: –Computational resources needed. –Data management issues to be faced. Sociological assessment: –Define benefit to end user. –Define cost to end user (e.g. skills to learn). –Define benefit to code developer. –Define cost to developer (e.g. skills to learn).

Requirements vs. CS Research Are the deployment requirements for a production scientific application compatible with CS research constraints? –How are engineering issues handled: robustness, error handling, documentation… –Who finances such expensive work? Shouldn’t production deployment be viewed as the “experimental program” for software technology research?

Example of adoption of new software technology in the TRANSP project: introduction of Python for code generator applications.

The End. Hopefully, the perspective derived from TRANSP experience will be helpful. But, it is only one perspective…