SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Pipeline Test methods Lars Kr. Lundin - CPL developer - NACO and VISIR developer - SPHERE Contact Person

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Presentation Outline Static code analysis Unit test Integration test Regression test Testing via nightly pipeline builds

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Static Code Analysis Static Code Analysis: The analysis of the source code without program execution. Find programming errors and non-conforming code. Tools:  gcc: Various warning-options (-Wall, -Wextra, …)  splint  Other C-compilers (e.g. Solaris)

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Unit test: Validate the individual units (i.e. functions) of the source code. Each unit test should be implemented before the functionality it is validating: Its initial failure will validate its testing ability. Benefits:  Exemplifies the intended usage of the tested function  Reduces complexity of integration testing  Errors in later changes will be exposed (regression bugs)  Unit testing can be automated Unit Test

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Unit test - build system iiinstrument/tests/Makefile.am : check_PROGRAMS = iiinstrument_my1st_module-test \ iiinstrument_my2nd_module-test iiinstrument_my1st_module_test_SOURCES = \ iiinstrument_my1st_module-test.c iiinstrument_my2nd_module_test_SOURCES = \ iiinstrument_my2nd_module-test.c TESTS = iiinstrument_my1st_module-test \ iiinstrument_my2nd_module-test

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Unit test – code example iiinstrument/tests/iinstrument_my1st_module-test.c: int main (void) { /* Test error handling and normal call */ assert( iiinstrument_my1st_function(NULL) == CPL_ERROR_NULL_INPUT); assert( iiinstrument_my1st_function(“OK”) == CPL_ERROR_NONE); return 0; }

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Integration Test Integration test: Test the combination of individual software components as a group, i.e. a complete pipeline recipe with a front-end, i.e. esorex and gasgano. Involves (manual) validation of the science products. Science validation is time consuming, thus done only after all automated tests are successful. Creates reference set of validated pipeline output data for subsequent regression testing.

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Regression Test Regression test: Uncover regression bugs, i.e. when functionality that used to work as required stops doing so. Regression bugs are typically introduced as an unintended consequence of a software change, possibly in a support library. Regression testing becomes essential once the development proceeds beyond the first working version.

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Regression Test Continued Partial automation of regression tests using the esorex regression test tool, esoreg.pl:  Execution of multiple tests (during lunch, night/week-end).  Detection of crashes, failures, missing products, memory-errors (valgrind).  Find differences in product-headers.  - report only largest change in a numerical FITS card.  Find largest change in every data unit.  Filter out insignificant changes (DATE related keys).

SDD/DFS L.K.Lundin VLT 2 nd Generation Instrumentation Pipelines, 19 Apr Nightly Pipeline Builds Nightly Pipeline Builds:  Tests run every night using latest CPL release  Unit tests  Simple integration test with esorex  Various static checks (splint, staticcheck)  Memory errors in Unit and Integration tests (valgrind)  Multi-platform tests (Linux, Solaris, MacOS, HP-UX)  Multi-compiler tests (new and older gcc, Solaris/HP-UX cc) 