Explicit arbitration specification binding does not scale Issue# UMLTP21-4 (https://issues.omg.org/browse/UMLTP21-4) Related issues - UMLTP21-2 Abstract TestDesignDirective concepts into abstract TestDirective (https://issues.omg.org/browse/UMLTP21-2)

Current situation UTP offers a mechanism to override the default AS for test cases, test sets (or test execution schedules) and procedural elements Default AS is always set, when no other AS are explicitly specified for the given element Three different situations are possible Default AS binding Static AS binding Dynamic AS binding (override during runtime) Every element that produces an arbitration result has a reference for static AS binding Some element may have to specify overriding AS for dyanmic binding COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Current situation (2) Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Current situation (3) Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Current situation (4) Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Current situation (5) Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Opinion Mechanism for binding explicit arbitration specification to elements is simple, but does not scale There can only be one single AS bound to such an element, which literally changes the test case physically If more than one AS shall be bound, one must copy the entire test case Introduce a binding mechanism that loosely couples an arbitration specification with a specific element Declare the current (in-line) binding mechanism deprecated New binding mechanism should be based on the TestDirective concept (see issue UMLTP21-2) COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Requirements The new arbitraton binding mechanism… shall support dynamic and post-execution comparison Dynamic: Verdict calculation happens during test execution instantaneously (capability of the test execution tool) Post-execution: Verdict calculation happens after test execution based on the test logs sometime shall enable the definition of different arbitration configurations by reusing and combining existing arbitration specification shall enable reuse of arbitration specifications across suitable arbitration targets (e.g., test sets, test cases, procedural elements) shall allow for binding multiple arbitration specifications to the very same corresponding arbitration target in different arbitration configurations COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Simplifying diagram – general view Arbitration Facility Test Logs (post-execution) Config Input Arbitration Bindings Test artifacts (dynamic) output COP for test design - web site, forum, group, learning structure, topics project, career Arbitration Result Doc#:

Simplifying diagram – post-execution Arbitration Facility Test Logs (post-execution) Input Config Arbitration Bindings verdict Test Case Arbiter Test Case Log Arbiter for expected Procedural Element Actual Procedural Element verdict COP for test design - web site, forum, group, learning structure, topics project, career Arbiter for expected Procedural Element Actual Procedural Element verdict Doc#: Arbitration Result

Simplifying diagram – dynamic Code generation facility Test artifacts (dynamic) Input Config Arbitration Directive Test Case Test Set Test Execution Schedule Procedural Elements output COP for test design - web site, forum, group, learning structure, topics project, career Test Script with Arbitration Information Verdict calculation process belongs to execution tool Doc#:

Our proposal: Cascading AS bindings Couple AS with arbitration targets on different levels Global: AS effectively bound to all arbitration targets of the input Test Set: AS effectively bound to all arbitration targets of that particular test set of the input Test Case: AS effectively bound to all arbitration targets of that particular test case of the input Procedural Element: AS effectively bound to either one particular procedural element or all procedural elements of a specific type (e.g., ExpectResponseAction) of the input Different scopes for the arbitration binding and evaluation process! AS bindings on more special scope override AS bindings on more general scope (like in cascading style sheets) Cascading AS bindings allows for fine-grained override Reference implementation in Fokus!MBT COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Example: AS Binding Scopes Test Set AS 1 Test Set 1 Test Set AS Binding 1 Test Case 1 Test Case AS Binding 1 Test Case AS 1 Test Case 2 Test Set AS 1 COP for test design - web site, forum, group, learning structure, topics project, career Test Set 1 Test Case 1 Test Case AS 1 Test Case 2 Default Test Case AS Doc#:

Example: AS Binding Scopes (2) Test Set 1 Test Set AS Binding 1 Test Set AS 1 Test Case 1 Test Case AS Binding 1 Test Case AS 1 ERA 1 ExpectResponse AS 1 ERA 2 ERA ASB 1 Test Case 2 Test Set AS 1 ERA 3 Test Set 1 Test Case AS 1 Test Case 1 COP for test design - web site, forum, group, learning structure, topics project, career ERA 1 ExpectResponse AS 1 ERA 2 Default ExpectResponse AS Test Case 2 Default Test Case AS ERA 3 Doc#:

Example: AS Binding Scopes Test Set 1 Test Set AS Binding 1 Test Set AS 1 Test Case 1 Test Case AS Binding 1 Test Case AS 1 ERA 1 ExpectResponse AS 1 ERA 2 ERA ASB 1 Test Case 2 ExpectResponse AS 2 ERA ASB 2 ERA 3 COP for test design - web site, forum, group, learning structure, topics project, career Test Set 1 Test Set AS 1 Test Case 1 Test Case AS 1 ERA 1 ExpectResponse AS 1 Default Test Case AS ERA 2 Test Case 2 ExpectResponse AS 2 ERA 3

Arbitration Facility Design Based on TestDirective/TestInstruction-Framework (see UMLTP21-2) Introduces new stereotypes ArbitrationDirective (subclassing TestDirective) ArbitrationSpecificationBinding (subclassing TesttInstruction) TestSetASBinding TestCaseASBinding ProceduralElementASBinding ArbitrationTarget (superclass of stereotypes TestSet, TestExecutionSchedule, TestCase and ProceduralElement) Used to harmonize the elements that may produce a verdict COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Sytax: ArbitrationTarget COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Sytax: Recap Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Sytax: ArbitrationDirective COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Syntax: ArbitrationBindings subclasses COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Syntax: TestSetASBinding COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Syntax: TestCaseASBinding COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Abstract Syntax: ProceduralElementASBinding COP for test design - web site, forum, group, learning structure, topics project, career Doc#: