1 Discussion Topics for Test Assertion Guideline 30 th Aug, 2007 Youngkon Lee Korea Polytechnic University

2 A table of contents Relationship with Test Metadata For Whom? Prerequisites Assertion Type Assertion Reference Type Assertion Reference Number Definition of Properties Test Criteria Structure of TAG

3 Relationship with Test Metadata Need to Clarify the relationship between TAG and test metadata (most major items are similar) Independence Import items and schema from metadata Compensation Differentiation point Test Guide Test Metadata + Test Assertion Data Items Test Parameter Test Coverage Test Criteria Related Spec.

4 For Whom? Is TAG helpful for spec makers? In a sense TAG provides just a way to reorganize requirements in the form of precondition, event, postconditions, something else. Is TAG helpful for test case developers? If TA gives some information for testing, for example test parameter, it may be helpful to testers. TAG Spec maker Test case maker help

5 Prerequisites Explicit prerequisite: declared in a specification Implicit prerequisite: not declared but used or related in a specification SOAP1.1, 1.2 WS-I ebMS 3.0 (For example) Provide packaging mechanism Guarantee Interoperability for SOAP

6 Assertion Type Assertion type is differentiated according to the easiness of transformation from requirement to test assertion Requirement could be Abstract: not concrete Example) Web Services Security 1.0 [WSS10] or 1.1 [WSS11] can be utilized to secure an ebMS message. Description: concrete but not testable Example) It is REQUIRED that compliant MSH implementations support Detached Signatures as defined by the XML Signature Specification [XMLDSIG]. Testable: concrete and testable Example) Web Services Security X.509 Certificate Token Profile is REQUIRED to encrypt message.

7 Assertion Reference Type Assertion could reference other requirements explicitly or implicitly. Case by case, the referenced requirements should be represented as assertion. Target requirement Requirements in same spec requirements Other specification Other specification Reference explicitly Reference implicitly

8 Assertion Reference Type(Example) Explicit Reference: The Content-Type MIME header of the Message Package MUST contain a type parameter whose value matches the MIME media type of the MIME body part containing the SOAP Envelope document. Implicit Reference: On Receiving side, processing of reliability headers should be possible by MSH ( → This means that reliability header conforms the SOAP header specification.) Reference to specification: Encryption of ebMS Messages is defined in Web Services Security [WSS10] and [WSS11].

9 Assertion Reference Number As the number of specification is increased, the possibility of referencing other specifications is also increased. How about test assertions? Do we need assertion number and reference mechanism? XML DSIG WS-S ebMS 3.0 Security XML ENC … canonica lization encrypt compari ng hashing

10 Definition of Properties Correctness of TA: a requirement is presented as TA without any loss of information. Necessary condition of TA: a TA has a information enough to make test cases. Coverage of TA: requirements area represented by TA Core: requirements of a target specification Sufficient: core + requirements of a specification explicitly related Full: sufficient + requirements of a specification implicitly related Core coverage Sufficient coverage Full coverage Explicit related Implicit related

11 Test Criteria What is the criteria of pass or fail in testing? Do we consider Quality factor? Response time, throughput, security, interoperability, manageability…

12 Structure of TAG Requirement qualifier Reference Pre- condition Trigger- event Effect- event Post- condition TA # Quality Criteria Implicit prerequisite TA: Preference + Main parts + Subsequence Base: explicit/implicit prerequisites + Test condition General Test Condition Explicit prerequisite Test Parameter TA Base for all TAs Preference Main PartsSubsequence Reference to other spec Assertion Type

13 Thank You!! Youngkon Lee: