Quality Assurance in the Presence of Variability Kim Lauenroth, Andreas Metzger, Klaus Pohl Institute for Computer Science and Business Information Systems (ICB), Essen – Germany George Valença Engenharia de Requisitos – /09/2010

Overview  Software Product Line Engineering (SPLE) is a reuse-driven development paradigm that has been applied successfully in information system engineering and other domains.  Quality assurance of the reusable artifacts of the PL is essential for successful SPLE.  As those artifacts are reused in several products, a defect in a reusable artifact can affect several products of the PL.  Since the reusable artifacts contain variability, quality assurance techniques from single-system engineering cannot directly be applied to those artifacts.  This paper describes strategies for quality assurance in the presence of variability and discuss in more detail one of those strategies: comprehensive strategy.

SPLE  SPLE explicitly addresses reuse by differentiating between two kinds of development processes:  Domain engineering: the domain artifacts (requirements, architectural and test models) are realized which implement the commonalities and provide the variability required to derive the set of intended products.  Application engineering: derive products from the domain artifacts and exploits the variability of the domain artifacts by binding (or resolving) variability according to customer and/or market-specific needs.  The central concept for addressing reuse in SPLE is the definition of product line commonality and variability.

Variability Modeling  The central concepts for defining and documenting the variability of a SPL are variation point and variant.  A variation point describes what varies between the products of a SPL.  A variant describes a concrete instance of a variation point.  Orthogonal Variability Model (OVM): variability modeling technique.  Variation points are modeled as triangles and variants as rectangles.  Three types of variability dependencies: mandatory variability dependency (this variant must always be selected when the variation point is considered for the product at hand), optional variability dependency (this variant can be selected but does not need to be selected) and alternative variability dependency (at least two variants which are related to a variation point).  The OVM also allows defining constraint dependencies to document additional dependencies between variation points and variants.

Challenges of Quality Assurance in SPLE  The overall quality of the PL and its derived products strongly depends on the quality of the domain artifacts – similar to the development of single software products.  Domain artifacts are reused in several products derived from the PL – in contrast to the development artifacts that are created in single software products. Thus, a high quality of the domain artifacts is desirable.  Quality assurance techniques from the development of single software products cannot be applied directly to domain artifacts because these artifacts contain variability. A domain requirements specification of a PL contains the: - Variable requirement R, related to the variant v1, - Variable requirement ¬R, related to the variant v2. Using a traditional quality assurance technique and performing a consistency check of this domain requirements specification would result in a contradiction, since R and ¬R cannot be fulfilled together. Hence, without considering the variability model, it is not possible to decide whether this inconsistency can affect any derived product of the PL or not. If the variability model does not allow the combined selection of the variants v1 and v2, then the contradicting requirements can never become part of the same specification and therefore will never cause an inconsistency.

Quality Assurance Strategies and Variability  In order to handle the variability in domain artifacts, quality assurance techniques in domain engineering follow three different strategies:  Commonality strategy: only the common parts shared by all products of the product line are covered by the quality assurance technique.  Sample strategy: the quality assurance technique is applied to sample products that are derived from the product line.  Comprehensive strategy: the quality assurance technique is applied to all possible products of the product line.

Commonality Strategy  Checks only the common parts of a SPL.  The variants are either (1) ignored during the checking of the reusable artifacts or (2) they are replaced by placeholders that abstract from the variants or simulate them.  The benefits of the commonality strategy are that early testing in domain engineering is enabled and that quality assurance activities can be performed even if no variants have been realized so far.  Techniques that follow this strategy must address the following challenges:  The effort for creating placeholders has to be kept at a minimum, since the placeholders are only used for the quality assurance purpose.  An adequate coverage of the domain artifacts has to be guaranteed.

Sample Strategy  Checks a set of sample products of the PL.  The basic steps of this strategy are typically as follows:  Determine one or more sample products (defined in terms of variants that are selected).  For each of the sample products: (a) derive product-specific artifacts by binding the variability in the domain artifacts; (b) apply quality assurance techniques from the single-system development to the derived artifacts.  Existing techniques from single-system development can be used as they are.  The following challenges have to be faced:  Selection of representative sample products is necessary.  Keeping the number of selected sample products manageable.

Comprehensive Strategy  Aims at ensuring the quality of all potential products of the SPL.  The basic steps of this strategy are typically as follows:  Bind the variability in the reusable artifacts for each of the potential products of the SPL.  Apply techniques from the development of single systems to the derived artifacts of each of these products.  Leads to the best coverage of the domain artifacts.  The number of potential products in a SPL of industry-relevant size prevents any ‘brute-force’ approach from being used for realizing the comprehensive strategy in practice.  Challenge: deal with the complexity that is involved in checking all potential products.

Towards a Comprehensive Quality Assurance Strategy in the Presence of Variability  Automated verification approaches are a common way to address quality assurance in SPLE.  In SPLE, model checking of domain artifacts means to verify that every possible product that can be derived from a domain artifact fulfills the specified properties. A single product is verified against expected properties Verifies that a set of products fulfills the properties specified for each product. Model Checking Single-system development SPLE X

Model Checking in the Presence of Variability  Adaptation of a model checking algorithm in order to enable the comprehensive quality assurance of a domain artifact.  The next-time-operator, EX f1 evaluates to true, if there is one path starting at the initial state on which f1 holds on the next state.  The main idea of the approach is to include the variability information specified in the variability model (as Boolean variables) in the model checking algorithms.  During the exploration of the state space, the algorithms consider the variability model to ensure that the current path explored in the state space is valid with respect to the variability model.

Formal Foundations of the Approach [1/3]  The variability model is formalized as follows:  Each variant of the variability model is represented as a Boolean variable vi, which evaluates to true if the corresponding variant is included in the derived product under consideration.  V is a set of all such Boolean variables of an OVM.  The constraints of the variability model are formalized by a Boolean expression OVM(V) over the variables in V.  OVM(v) evaluates to true for each valid product of the SPL (if the variants included in the derived product under consideration satisfy all variability dependencies and all constraint dependencies).

Formal Foundations of the Approach [2/3]  I/O-automata is used for the specification of the behavior of the products of a PL.  For documenting variability in I/O-automata, we define a variability relationship between the transitions Ti of the I/O-automaton and the variants V of the variability model as follows: VRel i/o ⊆ V × W(Ti) [W(Ti) is the power set of Ti].  t ∈ Ti is variable, if t is related to a variant: ∃ (v,T’) ∈ Vrel i/o : t ∈ T’.  t ∈ Ti is common, if t is not related to a variant: ∀ (v,T’) ∈ Vrel i/o : t ∉ T’.  We assume that a transition cannot be related to more than one variant, i.e. (v1, T’1), (v2, T’2). Vrel i/o : (T’1 ∩ T’2 = Ø) V (v1 = v2).

Formal Foundations of the Approach [3/3]  In order to perform the verification of a set of automata, the set of automata is integrated into one automaton by a product construction.  Existing single system algorithms for the product construction do not incorporate the variability of the product line.  Algorithm that incorporates the variability: extended transition relation that captures the combined behavior of the integrated automata.  The extended transition (named as T*) combines a transition with the variant selection information (V’) that is necessary to select a particular transition for a product.

Adaptation of Model Checking for EX f  Adaptation of State Labeling:  The model checking approach Labels each state with the properties that are fulfilled in this state. In variable I/O-automata, the fulfillment of a property may rely on variable transitions. Therefore, the state labeling may include the variant selection which is necessary to fulfill the property.  To incorporate the variability, we extend the labeling procedures as follows: Let f be a computational tree logic (CTL) expression, let z ∈ Z be a state of an I/O- automaton, and let V’ be a selection of variants. The state z is labeled with (f, V’) (i.e. (f, V’) ∈ label(z)), if f1 is fulfilled in state z for the selection V’ of variants.

Adaptation of Model Checking for EX f  Adaption of the Verification Algorithm:  The algorithm has two parameters: first, the property f1 which should be checked and, second, the variant vEX which is related to EX f1. The variant vEX is empty, if f1 is a common property.  For each outgoing transition of each state of a variable I/O-automaton, the algorithm verifies if the reached state z2 is labeled with f1 and the combined selection of variants of the property (the selection variants of the current transition), and the selection of variants associated with f1 in the next state can be fulfilled.  The algorithm checks each outgoing transition of each state and all possible labels. Therefore, every possible witness for EX f1 will be identified.

Adaptation of Model Checking for EX f  Checking the Completeness of Witnesses:  The existing single system algorithms for model checking rely on witnesses to show that a property is fulfilled (or not fulfilled) for a given system.  This approach is not sufficient for product lines. We address this challenge by checking the completeness of witnesses for all possible systems.  The algorithm has three parameters: the property f and the state z for which the completeness check has to be performed, and the variant v which is related to the property f.  The algorithm works as follows: It checks if the orthogonal variability model can fulfill a variant selection in which vp is selected and all possible variant selections related to the witnesses for f are not selected. If this is not possible, it is not possible to derive a product which has no witness for the property f in state z. If such a variant selection exists, this variant selection is an example for a derived product that has no witness for the property f.

Conclusion  We outlined different strategies for assuring the quality of domain artifacts in SPLE.  The variability prevents the direct application of quality assurance techniques from single-system engineering.  The success of the sample strategy depends on the quality of the selected sample products and a complete coverage of the PL is thus hard to guarantee in general.  The comprehensive strategy is based on model checking techniques and considers the variability of the domain artifacts during the verification process, thereby eliminating the need to costly check each product of the PL individually.

Quality Assurance in the Presence of Variability Kim Lauenroth, Andreas Metzger, Klaus Pohl Institute for Computer Science and Business Information Systems (ICB), Essen – Germany George Valença Engenharia de Requisitos – /09/2010