Developing I/O module of KIAPS Framework with Test-Driven Development Sang-Yoon Jun, Ki-Hwan Kim, Junghan Kim, In-Sun Song, Seoleun Shin, Youngsu Lee, and Dongchan Joo Systems Development Team, Korea Institute of Atmospheric Prediction Systems 2013 Spring Meeting of Korean Meteorological Society / KIAPS Workshop

Numerical model is a Complicated Software module relationship diagram for CAM-SE shallow water model

Developing complicate software: Software Engineering Software Requirement Software Design Software Construction Software Testing Software Maintenance Software Configuration Management Software Engineering Management Software Engineering Process Software Engineering Tools and Methods Software Quality The IEEE Computer Society tried to define and clarify the software engineering via the Software Engineering Body of Knowledge (SWEBOK) project. Object-Oriented Programming, Component Based Programming, Framework-Based Development, CVS, SVN, … Numerical model development utilizes

Software testing and conventional approach Write code Compile code Run code Test code Testing code is usually performed by checking result (log or visualized output) under different conditions or inserting validating routines into source Tests that verify software functionality - detects unintended changes - improves developer confidence - decreases risk from change Clune, 2012

Software Testing – testing complicated software Increasing in a number of implementation layer yields a complexity of software and a cost of verification/validation. Conventional testing method can be painful for the complicate software.

Solution for testing complicated software: Unit-Testing Add tests to each unit functionality (module, or class), and check reliability of software through these unit-tests. Unit-testing could help testing complicate software. As software becomes larger and more complex, tests as well. Writing and maintaining tests could be another heavy task.

change a paradigm: Test-Driven Development Integrates developed module to other module write code compile code run code test code Write test that fails Make test code to work normally Refactor code Conventional Development Process Test-Driven Development (TDD) The TDD gives benefits: - High reliability - Excellent Test Coverage - Less time spent debugging - Productivity - Tests act as maintainable documentation Unit-Testing and Test-Automation are the keys for Test-Driven Development

Solution for TDD: xUnit Testing Framework Test Case Test Fixture / Context Test Suites Assertion ABAP, ActionScript / Adobe Flex, Ada, AppleScript, ASCET, ASP, BPEL, C, C#, C++, Cg, CFML (ColdFusion), Clojure, Cobol, Common Lisp, Curl, Delphi, Emacs Lisp, Erlang, Fortran, F#, Groovy, Genexus, Haskell, HaXe, HLSL, ITT IDL, Internet, Java, JavaScript, Lasso, LaTeX, LabVIEW, LISP, Lua, MATLAB, MySQL,.NET programming languages, Objective-C, OCaml, Object Pascal (Free Pascal), PegaRULES Process Commander, Perl, PHP, PL/SQL, PostgreSQL, PowerBuilder, Progress 4GL, Prolog, Python, R programming language, Racket, REALbasic, Rebol, RPG, Ruby, SAS, Scala, Scilab, Scheme, Shell, Simulink, Smalltalk, SQL, TargetLink, Tcl, TinyOS/nesC, Transact-SQL, TypeScript, Visual FoxPro, Visual Basic (VB6), Visual Lisp, XML, XSLT Test written in language of target software Organize tests Provide verifying functions Automate tests xUnit testing framework provides functionalities based on each language

xUnit testing framework for Fortran: pFUnit Available at: pFUnit is a Fortran testing framework created by developer from National Aeronautics and Space Administration (NASA) and The Analytic Sciences Corporation (TASC). Distributed under NASA Open Source Agreement (NOSA) pFUnit targets at software written in Fortran - Supports using TDD in standard Fortran (INTEL, GNU, NAG, IBM, and PGI) - Supports testing of parallel software based on MPI - Extensive support for multidimensional FP arrays pFUnit targets at software written in Fortran - Supports using TDD in standard Fortran (INTEL, GNU, NAG, IBM, and PGI) - Supports testing of parallel software based on MPI - Extensive support for multidimensional FP arrays Used to develop some packages in NASA GISS-E model

Developing KIAPS I/O module with pFUnit Requirement of I/O module - provide interface routines, which read input and write output, to other module. - I/O methods could be changed and refined in order to increase model performance under various runtime condition. Requirement of I/O module - provide interface routines, which read input and write output, to other module. - I/O methods could be changed and refined in order to increase model performance under various runtime condition. Design of I/O module divide I/O module into 2 levels : high-level layer which provides interface to other modules, and low-level layer which handles input and output files. Design of I/O module divide I/O module into 2 levels : high-level layer which provides interface to other modules, and low-level layer which handles input and output files.

First Step: Writing test TestKiapsIoModule TestDriverMain SUBROUTINE TestInitIoSystems SUBROUTINE TestCreateFile SUBROUTINE TestWriteState … CALL AssertEqual(l_ierr, IO_NO_ERROR) … END SUBROUTINE SUBROUTINE TestCloseFile SUBROUTINE TestFinIoSystems PROGRAM TestDriverMain … suite_all = TestSuite('Test') CALL add(suite_all, io_tests()) CALL Run(suite_all, result) smy_stmt = Summary(result) … END PROGRAM TestDriverMain $> make unittests make[1]… … mpirun -n 4./tests.x 4 failures in Test::KiapsIoModuleTest::testDefineDimension: - (pe: 0) (pe: 0) rtion failed: Strings of unequal length. Differences begin at position 5. Expected: "time_invalid" but found: "time" - (pe: 1) String assertion failed: Strings of unequal length. Differences begin at position 5. Expected: "time_invalid" but found: "time" - (pe: 2) String assertion failed: Strings of unequal length. Differences begin at position 5. Expected: "time_invalid" but found: "time" - (pe: 3) String assertion failed: Strings of unequal length. Differences begin at position 5. Expected: "time_invalid" but found: "time" 9 run, 9 failed 0.08 seconds make[2]: … make[1]: … $> _ These tests will be failed.

Second Step: Make test to work normally KiapsIoModule PioIoModule TestKiapsIoModule TestDriverMain $> make unittests make[1]… … mpirun -n 4./tests.x 9 run, 0 failed 0.08 seconds make[2]: … make[1]: … $> _ Make test to pass by implementing I/O Module.

Third step: Refactor codes KiapsOutputModuleKiapsInputModule KiapsCommonIoModule PioIoModule TestKiapsIoModule TestDriverMain $> make unittests make[1]… … mpirun -n 4./tests.x 9 run, 0 failed 0.08 seconds make[2]: … make[1]: … $> _ We can refactor and refine code because passing tests guarantee normal working of I/O module.

Further works with the TDD KiapsOutputModuleKiapsInputModule KiapsCommonIoModule KiapsHistoryModule OutputVariableManagerModule PioIoModule Other Module TestKiapsIoModule TestDriverMain pNetCDFIoModule

Summary The Systems Development Team in KIAPS develops I/O modules by adopting Test-Driven Development. Test-Driven Development with pFUnit testing framework gives: - High reliability - Excellent Test Coverage - Less time spent debugging - Flexibility Test-Driven Development with pFUnit testing framework gives: - High reliability - Excellent Test Coverage - Less time spent debugging - Flexibility We consider using TDD to develop other modules in KIAPS model.

Thank you for your attention!