1. TestLog.verdict should be extracted into a dedicated ArbitrationResult concept
Issue# UMLTP2-22 Related issues # UMLTP2-24 „Explicit arbitration specification binding does not scale”
# UMLTP2-26 UTP “Test log facility ought to enable logging on procedural element level”

2. Current situation Verdict is a mandatory property of TestLog Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

3. Purpose of a test log
ISTQB: „A chronological record of relevant details about the execution of tests”
Test Logs serves different purposes
Pure informational
Basis for confirmation test (running the very same test with the exact same values again)
Analysis of test results (as part of test evaluation acitivities)
Comparison of different test runs of the very same test case
Post-execution comparison (requires capable test log model – see issue UMLTP2-26)
Post-execution comparison calculates the verdict (and performs all the comparisons) after test execution
Requires test logs for arbitration can be done much later
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

4. Multiple post-execution comparison
Several verdicts can be calculated for the very same test run
Based on different arbitration specifications (e.g., functional and hard-real time; different ArbitrationSpecification for different SIL levels etc.)
Two possible options for verdict arbitration
Dynamic: during test execution; scripted into the test case
Post-Execution: after test execution; based on test logs
Problem: A test log can only store a single verdict!
Consequence: If there are more than two ArbitrationSpecifications bound to a test case, the entire test log must be copied (redundant!)
Mitigation: Decouple attribute ‚verdict‘ from ‚TestLog‘ and introduce a new concept ‚ArbitrationResult‘
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

5. Proposal (for RTF)
Introduce a concept ‚ArbitrationResult‘ as extension of InstanceSpecification
‚ArbitrationResult‘ is an instance (InstanceSpecification) of «ArbitrationSpecification» Behavior
Nature of AR is determined by underlying AS (e.g. if an AR is an instance of an ExpectResponseActionAS, AR is implicitly an ExpectResponseActionAR)
Remove verdict from test log (or at least declare it as optional)
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

6. Proposal (for RTF) Conceptual model (example for test case) Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

7. Proposal (for RTF) Profile (draft) Doc#:
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

8. Proposal (for RTF) Advantageous
Decouples verdicts from test logs, i.e., the very same test log can be subject to multiple AR without introducing any redundancy
Further (proprietary) information can be easily incorporated into the AR by classifying the «AR» InstanceSpecifications by using UML classification
Result of comparison could be stored in AR as well (deviations between expected and actual data)
Further submitted data to the AS implementation such as timestamp, implementation version, entity etc.
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

9. Simpler Proposal (for FTF)
FTF does not allow heavy changes in the spec
Proposal: Declare multiplicity of attribute ‚verdict‘ as optional ([0..1]) in TestLog
Advantage
No need to remove ‚verdict‘ from TestLog, so that AR itself becomes an optional concept
User/vendors are not required to go for the more complex solution
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#:

10. Consensus ArbitrationResult will be integrated into the Profile
No change of the conceptual model required, for the ArbitrationResult is simply a technical solution to avaoid redundancy, not a conceptual change
Fits very well into the UTP 2 context and the UML classification of InstanceSpecifications
ArbitrationResult will contain at least the attribute ‘verdict’
Additional information might be added by means of the pre-defined auxiliary library (later on)
COP for test design - web site, forum, group, learning structure, topics project, career
Doc#: