1. Explicit arbitration specification binding does not scale
Issue# UMLTP21-4 ( Related issues - UMLTP21-2 Abstract TestDesignDirective concepts into abstract TestDirective (

2. Current situation
UTP offers a mechanism to override the default AS for test cases, test sets (or test execution schedules) and procedural elements
Default AS is always set, when no other AS are explicitly specified for the given element
Three different situations are possible
Default AS binding
Static AS binding
Dynamic AS binding (override during runtime)
Every element that produces an arbitration result has a reference for static AS binding
Some element may have to specify overriding AS for dyanmic binding
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

3. Current situation (2) Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

4. Current situation (3) Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

5. Current situation (4) Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

6. Current situation (5) Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

7. Opinion
Mechanism for binding explicit arbitration specification to elements is simple, but does not scale
There can only be one single AS bound to such an element, which literally changes the test case physically
If more than one AS shall be bound, one must copy the entire test case
Introduce a binding mechanism that loosely couples an arbitration specification with a specific element
Declare the current (in-line) binding mechanism deprecated
New binding mechanism should be based on the TestDirective concept (see issue UMLTP21-2)
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

8. Requirements The new arbitraton binding mechanism…
shall support dynamic and post-execution comparison
Dynamic: Verdict calculation happens during test execution instantaneously (capability of the test execution tool)
Post-execution: Verdict calculation happens after test execution based on the test logs sometime
shall enable the definition of different arbitration configurations by reusing and combining existing arbitration specification
shall enable reuse of arbitration specifications across suitable arbitration targets (e.g., test sets, test cases, procedural elements)
shall allow for binding multiple arbitration specifications to the very same corresponding arbitration target in different arbitration configurations
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

9. Simplifying diagram – general view
Arbitration Facility
Test Logs (post-execution)
Config
Input
Arbitration Bindings
Test artifacts (dynamic)
output
COP for test design - web site, forum, group, learning structure, topics project, career
Arbitration Result
Doc#:

10. Simplifying diagram – post-execution
Arbitration Facility
Test Logs (post-execution)
Input
Config
Arbitration Bindings
verdict
Test Case Arbiter
Test Case Log
Arbiter for expected Procedural Element
Actual Procedural Element
verdict
COP for test design - web site, forum, group, learning structure, topics project, career
Arbiter for expected Procedural Element
Actual Procedural Element
verdict
Doc#:
Arbitration Result

11. Simplifying diagram – dynamic
Code generation facility
Test artifacts (dynamic)
Input
Config
Arbitration Directive
Test Case
Test Set
Test Execution Schedule
Procedural Elements
output
COP for test design - web site, forum, group, learning structure, topics project, career
Test Script with Arbitration Information
Verdict calculation process belongs to execution tool
Doc#:

12. Our proposal: Cascading AS bindings
Couple AS with arbitration targets on different levels
Global: AS effectively bound to all arbitration targets of the input
Test Set: AS effectively bound to all arbitration targets of that particular test set of the input
Test Case: AS effectively bound to all arbitration targets of that particular test case of the input
Procedural Element: AS effectively bound to either one particular procedural element or all procedural elements of a specific type (e.g., ExpectResponseAction) of the input
Different scopes for the arbitration binding and evaluation process!
AS bindings on more special scope override AS bindings on more general scope (like in cascading style sheets)
Cascading AS bindings allows for fine-grained override
Reference implementation in Fokus!MBT
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

13. Example: AS Binding Scopes
Test Set AS 1
Test Set 1
Test Set AS Binding 1
Test Case 1
Test Case
AS Binding 1
Test Case AS 1
Test Case 2
Test Set AS 1
COP for test design - web site, forum, group, learning structure, topics project, career
Test Set 1
Test Case 1
Test Case AS 1
Test Case 2
Default Test Case AS
Doc#:

14. Example: AS Binding Scopes (2)
Test Set 1
Test Set AS Binding 1
Test Set AS 1
Test Case 1
Test Case
AS Binding 1
Test Case AS 1
ERA 1
ExpectResponse AS 1
ERA 2
ERA ASB 1
Test Case 2
Test Set AS 1
ERA 3
Test Set 1
Test Case AS 1
Test Case 1
COP for test design - web site, forum, group, learning structure, topics project, career
ERA 1
ExpectResponse AS 1
ERA 2
Default ExpectResponse AS
Test Case 2
Default Test Case AS
ERA 3
Doc#:

15. Example: AS Binding Scopes
Test Set 1
Test Set AS Binding 1
Test Set AS 1
Test Case 1
Test Case
AS Binding 1
Test Case AS 1
ERA 1
ExpectResponse AS 1
ERA 2
ERA ASB 1
Test Case 2
ExpectResponse AS 2
ERA ASB 2
ERA 3
COP for test design - web site, forum, group, learning structure, topics project, career
Test Set 1
Test Set AS 1
Test Case 1
Test Case AS 1
ERA 1
ExpectResponse AS 1
Default Test Case AS
ERA 2
Test Case 2
ExpectResponse AS 2
ERA 3

16. Arbitration Facility Design
Based on TestDirective/TestInstruction-Framework (see UMLTP21-2)
Introduces new stereotypes
ArbitrationDirective (subclassing TestDirective)
ArbitrationSpecificationBinding (subclassing TesttInstruction)
TestSetASBinding
TestCaseASBinding
ProceduralElementASBinding
ArbitrationTarget (superclass of stereotypes TestSet, TestExecutionSchedule, TestCase and ProceduralElement)
Used to harmonize the elements that may produce a verdict
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

17. Abstract Sytax: ArbitrationTarget
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

18. Abstract Sytax: Recap Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

19. Abstract Sytax: ArbitrationDirective
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

20. Abstract Syntax: ArbitrationBindings subclasses
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

21. Abstract Syntax: TestSetASBinding
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

22. Abstract Syntax: TestCaseASBinding
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

23. Abstract Syntax: ProceduralElementASBinding
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#: