14004 L5 - © D. Deugo, Lecture 5 Combinational Models (Binder chapter 6)

24004 L5 - © D. Deugo, Why Combinational Models? A combinational test model uses a decision table to represent the entity under test. According to Binder, such models are an ideal representation for a test model for several reasons: –straightforward (yet formal and rigorous) representation of responses wrt many related conditions –can be applied at several levels of abstractions (from system scope down to procedure scope) –they support automatic test case generation A warning: – Methods for ‘choosing’ combinations depend on scale of system. » Binder presents those relevant to small and medium-scale systems. –Each variable doubles the size of the table…

34004 L5 - © D. Deugo, Building a Decision Table A decision table is indicated for an entity that has the following characteristics: –One of several distinct responses is to be selected according to distinct cases of input variables –These cases can be modeled by mutually exclusive boolean expressions on these variables –The response to be produced does NOT depend on the order in which the input variables are set or evaluated –The response to be produced does not depend on prior input or output The last 2 are important restrictions! Several tools exist to support decision tables: –a decision table can be presented in several alternate forms but the semantics remain the same.

44004 L5 - © D. Deugo, Alternate Forms Table 6.1 p.125 –a condition expresses a relationship among decision variables that must be resolvable to true or false –Each unique combination of conditions and actions is a variant –Notice the condition section and the action section Table 6.2 p.126 –equivalent but column-wise format Figure 6.2 p. 127 –decision tree Figure 6.3 p.128 –Truth table format Figure 6.3 p.132 –UML Activity Diagram »which is supposed to be a form of eFSM

54004 L5 - © D. Deugo, Working out the Variants A table with n conditions has at most 2 n variants: –we aim for as few variants covering as much as possible –each action must be produced by at least one variant. –If more than one combination of conditions can result in the same action, then explicit variants must be provided for these combinations. –A variant that can be inferred but is not given is an implicit variant: -Implicit variants result from valid abbreviations (don’t care and logically excluded cases) or incorrect modeling (can’t happen, don’t know) -A variable with don’t care decision variables can correspond to several cases: (p.129) -necessary inputs with no effect -e.g., parameters of a procedure not always used -inputs that can be omitted -a type-safe exclusion for non-boolean variable -e.g., age cannot be over and under 25 at the same time

64004 L5 - © D. Deugo, Special Conditions Two special conditions indicate flaws of the specification: -A can’t happen condition reflects an assumption that some inputs are mutually exclusive, or that some inputs cannot be produced by the environment. -A can’t happen condition is not the same as a don’t care or a type-safe exclusion! -One might be tempted to assume that an inconsistent setting of some variables can’t happen and conclude that there is no need to specify or test the corresponding variants. -Ariane 5 demonstrates the danger of this fallacy! -A don’t know condition reflects an incomplete model. -Specifications may have parts to be determined!

74004 L5 - © D. Deugo, Deriving the Logic Function Decision and truth tables are not the same: –A truth table is a special case of a decision table, in that all cells in it must be resolvable to true or false. –Conversely, decision variables and output actions may be arbitrarily complex variables defined from arbitrarily complex types and combined in arbitrarily complex predicate expressions. Figure 6.4 p.134 –a summary of different notations for logic reduction There are inherent limits to minimization techniques: –Table 6.5 p.140 –These techniques are briefly discussed in Binder (pp ): »boolean function p.136: this is what we are looking for! »KV Matrix p.143: from table 6.4 p.135 »Cause-Effect graph p.145: from analysis, not reduction! more complex forms have been proposed (with requires relating variables)

84004 L5 - © D. Deugo, Validation For your logical formula, you may want to use a spreadsheet (see p.149 for boiler function) Binder suggests an extensive checklist p.151: –goes back to the engineering of the variants wrt the conditions and the actions

94004 L5 - © D. Deugo, Faults to Catch in Decision Table Incorrect value assigned to a decision table Incorrect or missing operator in a predicate Incorrect or missing variable in a predicate Incorrect structure in a predicate (dangling else, etc.) Incorrect or missing default case Incorrect or missing actions Extra actions Structural errors in table ’s implementation Missing or incorrect class or method signature when variants are implemented by dynamic binding Generic errors: wrong versions, ambiguous reqs, incorrect or missing specification item

L5 - © D. Deugo, Test Generation Strategies (1) For decision tables: –All-Explicit Variants: »each explicit variant is produced at least once »equivalent to All-True Strategy For truth tables: –All-Variants: every variant is tested once. The number of tests rises exponentially with the number of variables. –All-True: every variant in the table that produces a true outcome is tested. Does not work if important behavior follows from the false actions. –All-False: mirror strategy to All-True –All-Primes: A subset of the All-true strategy relying on a minimal sum-of-products form… (p 136 Binder) –Table 6.15 p.172 shows the size of test suites but these strategies can easily fail due to the test they exclude… –And more….

L5 - © D. Deugo, Test Generation Strategies (2) Each condition/All condition: –The compact test suite (n+1 tests for n variables) is composed of test cases such that each variable is made true once with all other variables being false and one test case where all variables are true (for AND) or false (for OR logic). –This heuristic bets on the independence of condition evaluation and the absence of faults that would mask an error. –Simplistic examples: table 6.8 p.155 and table 6.9 p.156 –For Z = AB~C +AD(pp.156-7) »Need 4 variants for the first term »Need 3 variants for the second term –The number of tests increases linearly with the number of product terms Binary Decision Diagram Determinants: –Requires a truth table… –It’s a tree reduction strategy… –Ignores don’t cares…

L5 - © D. Deugo, Test Generation Strategies (3) Variable Negation: –Requires a specification given as a sum-of-products boolean formula. –It produces a test suite that will contain : »One variant for each product term such that this variant makes the product term true but makes no other product term true (if it exists): unique true points »One variant for the term that results when each literal in each product is negated such that the function evaluates to 0 for the negated term: near false points –Of the variants that meet these criteria, some must be selected… –Example p.162 (refers to table p.135) Choosing when you have a truth function –Table 6.14 p.169: A summary –Figure 6.23 p.171: Inclusion hierarchy Bottom line: why fiddle with techniques that assume a truth function…

L5 - © D. Deugo, Non-Binary Domain Analysis Uses the domain testing strategy (which is a form of boundary value testing): –Notion of subdomain (p.164): equivalence partitioning technique –Minimal domain test strategy: pick one on point and on off point per boundary. –Exhaustive example uses an in-point as well and possibly several off-points per variable. »See tables pp »Some test cases may overlap across these tables

L5 - © D. Deugo, Choosing a Combinatorial Strategy Must trade cost roughly proportional to the number of test cases to obtain increased confidence in the tested implementation Inclusion Hierarchy figure 6.23 shows how strategies can be ordered to the kind of variants they include Number of tests for a decision table is directly related to the number of –Decision variables –Complexity (non-binary variables increase number of test) Table 6:14 and p. 170