Created by Jan Medved Integration & Test Strategy for Lithium

Created by Jan Medved  OpenDaylight is made of multiple projects -> we have to test them individually and together  Projects are quite heterogeneous: core, tools, plugins, apps, etc… -> they have different test needs  OpenDaylight has a common CI process for all projects -> tests should in general be part of the CI  We leave users freedom to select features -> we need an strategy to test a non-rigid distribution Facts 2

Created by Jan Medved  Forget about verifying projects, focus on features  Test single feature enabled to identify issues in a feature itself  Test all features enabled to identify feature interferences  For a given project, tests are called after:  A change (merge) in an upstream project. This triggers build and system test  A change in the project itself. We miss system test on code submit  Be selective with the tests we run  A change in a “stem” project will trigger test on many features  A change in a “leaf” project will trigger test on few concerned features High Level Strategy 3

Created by Jan Medved We define 2 integration features for test purposes:  compatible-with-all -> useful for all features system test  all = compatible-with-all + incompatible features -> useful to detect feature/bundle interferences We generate 2 distributions (no feature enabled at startup):  Regular distribution with all projects feature indexes (except integration)  Test distribution = Regular distribution + integration feature index Distribution Assembly 4

Created by Jan Medved Build test occurs during project code build. Examples are Junit, PAX-EXAM or Karaf feature install test Every project in OpenDaylight defines 3 build jobs:  Verify -> This job builds the project code and pass build tests upon code submit  Merge -> This job builds the project code and pass build tests upon code merge. In addition it will upload generated artifacts to Nexus repository.  Integration -> This job will build the project code and pass build tests after a merge in an upstream project Build Test 5

Created by Jan Medved System test requires a framework that installs controller and executes tests against network tools. Example is the Robot test. For every project user-facing feature we will create 2 system test jobs:  feature-only test -> Install single feature with required dependencies and run feature planned test  feature-all test  If feature is compatible -> Install odl-integration-compatible- with-all and run feature planned test  If feature is not compatible -> Install odl-integration- compatible-with-all + incompatible feature and run feature planned test System Test 6

Created by Jan Medved Test Triggers 7 Code Merge Release creation Merge build feature-only test feature-all test feature-only test feature-all test For all available features For all features in downstream Code Submit Verify build *Optional (requires project distribution or some automation to update project artifact) Integration build For all projects in downstream feature-only test*