1. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 1 Antonio Longinotti Testing Instrumentation Software Applications

2. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 2 Presentation Objectives 1.Why it is important to write and exercise test Software 4.How tests should be executed and implemented 3.What should be tested 2.When tests should be defined and implemented

3. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 3 Objective 1 1.Why it is important to write and exercise test Software 4.How tests should be executed and implemented 3.What should be tested 2.When tests should be defined and implemented

4. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 4 Why testing ? Best way to validate basic assumptions: The Software works The Software does what expected It is a powerful instrument to measure quality and readiness of the Software Costs associated to testing are compensated by the savings in maintenance and re-engineering

5. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 5 Objective 2 1.Why it is important to write and exercise test Software 4.How tests should be executed and implemented 3.What should be tested 2.When tests should be defined and implemented

6. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 6 When testing ? During implementation in parallel to the code Why ? Early Defect Detection Higher Return on Investment (ROI)

7. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 7 Objective 3 1.Why it is important to write and exercise test Software 4.How tests should be executed and implemented 3.What should be tested 2.When tests should be defined and implemented

8. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 8 Engineering Level Tests to be implemented ICS Single device functional tests Subsystem tests (all devices in parallel) Robustness and reliability tests (long duration) DCS Single exposure standalone, mostly used readouts MS Test OB exercising all technical templates

9. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 9 System Level Tests to be implemented OS System startup/shutdown Single exposures for each Instrument mode Templates Test OB exercising all science templates

10. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 10 Test documentation Template for Acceptance Test Plan (ATP) VLT-PLA-ESO-17240-2266 ATP draft at FDR, to be kept up-to-date till PAE Template for Acceptance Test Report (ATR) VLT-TRE-ESO-17240-3162 Write a Test Report after each formal session You need an approved Test Plan

11. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 11 Objective 4 1.Why it is important to write and exercise test Software 4.How tests should be executed and implemented 3.What should be tested 2.When tests should be defined and implemented

12. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 12 Automatic or manual Tests ? Automatic whenever possible Manual tests is inefficient  Time consuming  Error prone  Tester Signature Why ?

13. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 13 No manual tests at all ? 4LGSF lab Necessary e.g. when safety is an issue

14. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 14 INS Software framework for Testing Module insc provides a set of standard scripts/utilities. They are based on tat (part of VLT common Software) Example of complete set of tests for an Instrument is available in the Template Instrument No need to code: just configuration files

15. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 15 Insc utilities inscClean Create conditions to rebuild the instrument Sw from scratch inscPickObjectSimStart Start simulating a loop of pick object user actions on rtd inscBuildCode Execute the build code phase inscPickObjectSimStopStop simulated pick object loop inscBuildEnvExecute the build environments phase inscSelfTestIt implements the code for testing a sub-system inscStartEnvStart all environmentsinscSelfTestDCSSelf test of all Instrument DCSs inscStopEnvStop all environmentsinscSelfTestICSSelf test of all ICS devices and ICS itself inscDetSlcuPerform clean/build/start operations on the DCS SLCU inscSelfTestOSSelf test of OS inscMidasStartStart an on-line Midas sessioninscSelfTestTPLSelf test of templates inscMidasStopStop an on-line Midas sessioninscSelfTestMSSelf test of MS inscTcssimStartStart TCS simulation and get it tracking inscTcssimStopStop TCS simulation

16. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 16 DevelopmentIntegrationOperations Site A Site B Site C Garching VCM Paranal Different hw conditions at different sites Same Instrumentation application must run at different sites under different hw conditions

17. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 17 Installation strategy Same code used at all locations Differences only in configuration files Based on pkgin and TARGET environment variable (see VLT-MAN-ESO-17240-1913) Main site (e.g. Paranal) TARGET is not defined cmmCopy uvins pkginBuild uvins pkginBuild does not look for any TARGET specific file Other sites (e.g. Garching VCM) export TARGET=CM_FULL cmmCopy uvmgar pkginBuild uvmgar pkginBuild looks for file uvmgarTARGET_CM_FULL.cfg Example for UVES

18. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 18 Example: Install UVES inVCM File uvmgarTARGET_CM_FULL.cfg contains: # LCU CPU type for UVES in VCM INSTALL.MODULE.CPU "MC68040“ # Do not build BLUE SCCD and TCCD LCU environments: they are simulated on WS INSTALL.ENVS.AVAIL "$INSTALL.RTAPENV1.NAME $INSTALL.LCUENV1.NAME $INSTALL.LCUENV4.NAME“ # Remote operations on FIERA RED LCU (Sparc) INSTALL.HOOK26.NAME "BEFORE_CLEAN"; INSTALL.HOOK26.PLUGIN "inscDetSLcu -env $INSTALL.DET2.CCDLENV -ccd $INSTALL.DET2.CCDNAME -clean" INSTALL.HOOK27.NAME "AFTER_STOP_ENV"; INSTALL.HOOK27.PLUGIN "inscDetSLcu -env $INSTALL.DET2.CCDLENV -ccd $INSTALL.DET2.CCDNAME -stop" INSTALL.HOOK28.NAME "AFTER_BUILD_ENV"; INSTALL.HOOK28.PLUGIN "inscDetSLcu -env $INSTALL.DET2.CCDLENV -ccd $INSTALL.DET2.CCDNAME -build" INSTALL.HOOK29.NAME "AFTER_START_LCUENV"; INSTALL.HOOK29.PLUGIN "inscDetSLcu -env $INSTALL.DET2.CCDLENV -ccd $INSTALL.DET2.CCDNAME -start“ # use VCM TCS INSTALL.SCAN7.REMOTEENV "wt0tcs"

19. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 19 Test strategy Default (Main location = Paranal). File uvmcfgTEST.cfg # Placeholder for TCS simulation OCS.TPL.TESTPAR1 "" # Bring the instrument online OCS.TPL.TESTCMD7 "ONLINE“ # start loop on red slit viewer OCS.TPL.TESTPAR13 "msgSend wuves uvoSVCtrl_RED START \"\"" # Execute Self Test OB OCS.TPL.TESTPAR10 "inscExecOB UVES_gen_tec_SelfTest.obd" # Bring the instrument to STANDBY OCS.TPL.TESTCMD12 "STANDBY" # Show the results. OCS.TPL.TESTPAR14 "sh -c \"slxCatFits `ls $INS_ROOT/SYSTEM/DETDATA/UVES_RED_STD*.fits`\"" Garching VCM. File uvmgarTEST_CM_WS.cfg # Start TCS simulation OCS.TPL.TESTPAR1 "inscTcssimStart NB -restart -insId UVESslit“ Inherit default sequence. Overload only the necessary steps Same code. Differences only in configuration files Based on insc and TARGET environment variable Example for UVES Templates test

20. ESO – A.Longinotti ESO Instrumentation Software Workshop 2008 - 20 Example: Test UVES in the VCM 1.Login as user uvesmgr 2.> export TARGET=CM_FULL 3.> cmmCopy uvmgar 4.> cd uvmgar/test 5.> tat The following sequence of tests is executed: Clean INTROOT, INS_ROOT etc Retrieve code from cmm Archive Build code Build environments Start environments Start and test all DCSs Start ICS and test all its devices Start OS and test it Test all templates Test MS Stop environments