Discussion on Testing Simantics Technical Board Meeting

Organizing Test Suites Suggestion: new plug-in org.simantics.tests – Contains test suites for different testing activities: Regression TDD Performance Stress – Composes test suites for the SDK from around the codebase, doesn’t contain test cases – Most suites are automatically tested via Jenkins

Testing Activities Regression – First step is to move all currently existing and passing tests into regression tests TDD – Development-time, moved into regression when development is complete, manually executed Performance/Scalability – Focus on performance-critical parts of the SDK right now: databoard, DB, graphics, history Stress – For uncovering hidden unstabilities, mainly relevant for DB

Tools to Incorporate Squish (finally) EMMA (code coverage for unit tests) Possibly static analysis tools

Static Analysis Several (free) tools available for Java [1] that have Eclipse & Jenkins plug-ins :[1] – PMD, FindBugs, Checkstyle AFAIK, none of us has been actively using any of these with Simantics Relatively cheap to automate Need to test the tools first to say anything definitive of usefulness

What to Test? org.simantics.tests: everything that is part of the platform – not products built on top of it (sysdyn, modelica, etc.) A separate test suite for every product UI testing – Difficult without a Workbench product – Use movie tutorial ? – Perhaps create the planned diagram editor for it ?

Static Analysis Several (free) tools available for Java [1] that have Eclipse & Jenkins plug-ins :[1] – PMD, FindBugs, Checkstyle AFAIK, none of us has been actively using any of these with Simantics Relatively cheap to automate Need to test the tools first to say anything definitive of usefulness

Reporting Jenkins provides unit/squish test reports Performance testing needs different logging – Instrument all tested code with necessary performance logging (See Eclipse tracing facility)Eclipse tracing facility – See about using Apache JMeter (Jenkins plug-in)Apache JMeterJenkins plug-in For stress tests it is important to have proper logging enabled and relevant data kept for failed tests (databases, etc.)

Epics (prioritized) Create and automate org.simantics.tests Sysdyn product test suite with UI tests List most important testing topics Prevent publishing of builds with broken tests & ensure is delivered about breakage – Try to force developers to react to acute problems

References [1]