TestLog.verdict should be extracted into a dedicated ArbitrationResult concept Issue# UMLTP2-22 Related issues # UMLTP2-24 „Explicit arbitration specification binding does not scale” # UMLTP2-26 UTP “Test log facility ought to enable logging on procedural element level”

Current situation Verdict is a mandatory property of TestLog Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Purpose of a test log ISTQB: „A chronological record of relevant details about the execution of tests” Test Logs serves different purposes Pure informational Basis for confirmation test (running the very same test with the exact same values again) Analysis of test results (as part of test evaluation acitivities) Comparison of different test runs of the very same test case Post-execution comparison (requires capable test log model – see issue UMLTP2-26) Post-execution comparison calculates the verdict (and performs all the comparisons) after test execution Requires test logs for arbitration can be done much later COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Multiple post-execution comparison Several verdicts can be calculated for the very same test run Based on different arbitration specifications (e.g., functional and hard-real time; different ArbitrationSpecification for different SIL levels etc.) Two possible options for verdict arbitration Dynamic: during test execution; scripted into the test case Post-Execution: after test execution; based on test logs Problem: A test log can only store a single verdict! Consequence: If there are more than two ArbitrationSpecifications bound to a test case, the entire test log must be copied (redundant!) Mitigation: Decouple attribute ‚verdict‘ from ‚TestLog‘ and introduce a new concept ‚ArbitrationResult‘ COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Proposal (for RTF) Introduce a concept ‚ArbitrationResult‘ as extension of InstanceSpecification ‚ArbitrationResult‘ is an instance (InstanceSpecification) of «ArbitrationSpecification» Behavior Nature of AR is determined by underlying AS (e.g. if an AR is an instance of an ExpectResponseActionAS, AR is implicitly an ExpectResponseActionAR) Remove verdict from test log (or at least declare it as optional) COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Proposal (for RTF) Conceptual model (example for test case) Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Proposal (for RTF) Profile (draft) Doc#: COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Proposal (for RTF) Advantageous Decouples verdicts from test logs, i.e., the very same test log can be subject to multiple AR without introducing any redundancy Further (proprietary) information can be easily incorporated into the AR by classifying the «AR» InstanceSpecifications by using UML classification Result of comparison could be stored in AR as well (deviations between expected and actual data) Further submitted data to the AS implementation such as timestamp, implementation version, entity etc. COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Simpler Proposal (for FTF) FTF does not allow heavy changes in the spec Proposal: Declare multiplicity of attribute ‚verdict‘ as optional ([0..1]) in TestLog Advantage No need to remove ‚verdict‘ from TestLog, so that AR itself becomes an optional concept User/vendors are not required to go for the more complex solution COP for test design - web site, forum, group, learning structure, topics project, career Doc#:

Consensus ArbitrationResult will be integrated into the Profile No change of the conceptual model required, for the ArbitrationResult is simply a technical solution to avaoid redundancy, not a conceptual change Fits very well into the UTP 2 context and the UML classification of InstanceSpecifications ArbitrationResult will contain at least the attribute ‘verdict’ Additional information might be added by means of the pre-defined auxiliary library (later on) COP for test design - web site, forum, group, learning structure, topics project, career Doc#: