COMP 6710 Course NotesSlide 9-0 Auburn University Computer Science and Software Engineering Course Notes Set 9: Functional Testing Computer Science and Software Engineering Auburn University

COMP 6710 Course NotesSlide 9-1 Auburn University Computer Science and Software Engineering Dynamic (running) black-box (blindfolded) testing Also known as behavioral testing Based on specification - if one not available, the software is the spec Exam inputs/outputs: needs test cases Functional Testing

COMP 6710 Course NotesSlide 9-2 Auburn University Computer Science and Software Engineering Test-to-pass - make sure the software minimally works - don’t push it to the limit - apply simplest and/or straightforward cases - not to find bugs, initially - do this test FIRST Purpose of Testing

COMP 6710 Course NotesSlide 9-3 Auburn University Computer Science and Software Engineering Test-to-fail - after test-to-pass - design and run test cases with the purpose to break the software - probe the known and unknown weaknesses - errors forcing Purpose of Testing

COMP 6710 Course NotesSlide 9-4 Auburn University Computer Science and Software Engineering Functional (Black Box) Testing Knowing the specified function (requirements), design test cases to ensure that those requirements are met. –Example : Sort (list); Structural Testing - How well is the code exercised? Functional Testing - How well does Sort perform its intended function? In general, complete functional testing is not feasible –Attempting to test every possible input to the function A randomly selected set of test cases is statistically insignificant –“Not all test cases are created equally” Test case selection –Based on characteristics of input and output sets relative to specified functionality.

COMP 6710 Course NotesSlide 9-5 Auburn University Computer Science and Software Engineering Functional Testing Types of errors looked for during functional testing –Incorrect function or missing function –Interface errors –External database errors –Performance errors (including stress testing) –Initialization/termination errors Tests are designed to answer the following questions –How is functional validity tested? –What classes of input will make good test cases? –Is the system particularly sensitive to certain input values? –How are the boundaries of a data class isolated? –What data rates and data volume can the system tolerate? –What effect will specific combinations of data have on system operations? [Adapted from Software Engineering 4th Ed, by Pressman, McGraw-Hill, 1997]

COMP 6710 Course NotesSlide 9-6 Auburn University Computer Science and Software Engineering Goals and Methods of Functional Testing Goals –Produce test cases that reduce the overall number of test cases –Generate test cases that will tell us something about the presence or absence of errors for an entire class of input Methods/Approaches –Equivalence partitioning –Boundary value analysis –Matrix of functional possibilities –Cause-effect graphing –Decision Tables

COMP 6710 Course NotesSlide 9-7 Auburn University Computer Science and Software Engineering Equivalence Partitioning It is impossible to test all cases Equivalence partitioning provides a systematic means for selecting cases that matter and ignoring those that don’t An equivalence class or equivalence partition is a set of test cases that tests the same aspect or reveals the same bugs e.g., If X >= 15 then do-this else do-that (- 15) 15 (15 )

COMP 6710 Course NotesSlide 9-8 Auburn University Computer Science and Software Engineering Equivalence Partitioning Equivalence partitions – groups for similar inputs, outputs, and/or operation of the software e.g., file-name, characters - valid characters - invalid characters - valid length - invalid length

COMP 6710 Course NotesSlide 9-9 Auburn University Computer Science and Software Engineering Equivalence Partitioning e.g., copy operation - copy menu - c or C - Ctrl-c or Ctrl-Shift-c Fully tested in the first effort, equivalence partitioning (1 case each) test for new versions Goal: to reduce the set of possible test cases Too few partitions => may not reveal all catchable bugs

COMP 6710 Course NotesSlide 9-10 Auburn University Computer Science and Software Engineering Equivalence Partitioning Equivalence partitioning divides the input domain of a program into classes of data from which test cases can be derived –Ideally, each test case could uncover classes of errors, thereby reducing the total number of test cases that must be developed Input condition - some kind of condition placed on the input –Typically a specific value, a range of values, a set of related values, or a Boolean condition Equivalence Class - a set of valid or invalid states for input conditions –Range - 1 valid and 2 invalid equivalence classes are defined –Specific Value - 1 valid and 2 invalid equivalence classes are defined –Set - 1 valid and 1 invalid equivalence class are defined –Boolean - 1 valid and 1 invalid equivalence class are defined [Adapted from Software Engineering 4th Ed, by Pressman, McGraw-Hill, 1997]

COMP 6710 Course NotesSlide 9-11 Auburn University Computer Science and Software Engineering Example Ϭ¹¹¹¹¹¹¹¹¹ Þßàfunction in_list (input1:name_type; ϧÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏÏinput_list : list_names) ϧÏreturn boolean is Ϫ˹¹¹¹¹¹¹¹ ÏϧÏíÏp : list_names; Ïϧbegin ÏϨ¹¹Ïp := input_list; ÏϨ¹¹±while not(p = null) loop ÏϧÏÏ·¹³´if Ada.Strings.Fixed.Index ÏϧÏϯϵ§(Source => String(p.name), ÏϧÏϯϵ§Pattern => String(input1)) /= 0 then ÏÂÄÏϯϵ¾¹¹Ïreturn true; ÏϧÏϯ϶´else ÏϧÏϯϸ¾¹¹Ïp := p.next_name; ÏϧÏϯÏÈÏend if; ÏϧÏÏ°end loop; ÏÂĹ¹Ïreturn false; Equivalence Classes: (1) Inputs where input1 is in the list (2) Inputs where input1 is not in the list Specific Input Partitions: Listinput1 Empty? One elementIn the list One elementNot in the list >One elementFirst element >One elementLast element >One elementMiddle element >One elementNot in list Test Cases Listinput1Output ?false birdbirdtrue birdfishfalse bird, cat, owlbirdtrue dog, pig, chickenchickentrue...

COMP 6710 Course NotesSlide 9-12 Auburn University Computer Science and Software Engineering Boundary Value Analysis Range : a..b –Example : Test cases : 99, 100, 101, 199, 200, 201 Number of values –Test cases that exercise minimums and maximums Apply the above to the output conditions –Try to drive output to invalid range Internal data structures with boundaries –Example : A(1..100) with test cases A(0), A(1), A(2), A(99), A(100), A(101) –A(0) and A(101) should generate exceptions ab ()) ((

COMP 6710 Course NotesSlide 9-13 Auburn University Computer Science and Software Engineering Boundary Condition Test Cases If software can operate on the edge of its capabilities, it will almost certainly operate well under normal conditions For I = 1 to 10 data (I) = -1; end; 10 elements, data(0), data(1), data(2) data(9), data(10), data(11)

COMP 6710 Course NotesSlide 9-14 Auburn University Computer Science and Software Engineering Boundary Condition Test Cases Boundary conditions for a legitimate triangle Boundary conditions for side classification Boundary conditions for angle classification Valid input/extremes

COMP 6710 Course NotesSlide 9-15 Auburn University Computer Science and Software Engineering Boundary Condition Test Cases Types of Boundary conditions numericcharacterpositionquantity speedlocationsize Also, extremes first/lastmin/maxstart/finishover/under empty/fullShortest/longest slowest/fastestlargest/smallest…

COMP 6710 Course NotesSlide 9-16 Auburn University Computer Science and Software Engineering Boundary Condition Test Cases Partitions - boundary - one or two valid points inside the boundary - one or two invalid points outside the boundary e.g., First – 1 / Last + 1 Smallest –1 / Largest + 1

COMP 6710 Course NotesSlide 9-17 Auburn University Computer Science and Software Engineering Sub-boundary Conditions Also known as Internal boundaries Bit, nibble, byte, word, K, M, G, T Why? E.g., 256 commands, 15 are frequently used. Needs only a nibble.  0XXXX nibble, 1XXXXXXXX byte

COMP 6710 Course NotesSlide 9-18 Auburn University Computer Science and Software Engineering Sub-boundary Conditions ASCCI table – boundaries not obvious Default, empty, blank, null, zero, none (may be of a separate equivalence partition and treated individually) Invalid, wrong, incorrect, garbage data (test to fail)

COMP 6710 Course NotesSlide 9-19 Auburn University Computer Science and Software Engineering Matrix of Functional Possibilities Input/Output Conditions –If the number of combinations of input/output is manageable, then consider using a matrix of functional possibilities –Especially useful if the input/output combinations are enumerated in the requirements specification Example : Input (or output) will be a combination of {A,B} and {x,y,z}

COMP 6710 Course NotesSlide 9-20 Auburn University Computer Science and Software Engineering Example : The Triangle Problem Input –3 floating point numbers Processing –Determine if the 3 numbers form a triangle If not, print message “Not a Triangle” If it is a triangle –Classify according to side : equilateral, isosceles, scalene –Classify according to largest angle : acute, right, obtuse Output –List the 3 numbers –List the classification or “Not a triangle”

COMP 6710 Course NotesSlide 9-21 Auburn University Computer Science and Software Engineering MFP for the Triangle Problem Additional Functional Test Cases (if any):

COMP 6710 Course NotesSlide 9-22 Auburn University Computer Science and Software Engineering Cause Effect Graphing Causes (input conditions) and effects (actions) are listed for a module, and an identifier is assigned to each A cause-effect graph is developed –Looking for causes without effects –Looking for effects without causes The graph is converted to a decision table (if a decision table has been used as a design tool, developing the graph and table is not necessary) Decision table rules are converted to test cases

COMP 6710 Course NotesSlide 9-23 Auburn University Computer Science and Software Engineering Cause-Effect Graph Symbology Identity “Not” “And” “Or” SymbologyConstraints c1 c2 c1 c2 e1 c a a a aa b b b b b E I O R M Exclusive Inclusive Only One RequiresMasks

COMP 6710 Course NotesSlide 9-24 Auburn University Computer Science and Software Engineering Cause-Effect Graphing Example The CHANGE subcommand - used to modify a character string in the “current line” of the file being edited –Inputs Syntax : C /string1/string2 String1 represents the character string you wish to replace –1-30 characters –Any character except ‘/’ String2 represents the character string that is to replace string1 –0-30 characters –Any character except ‘/’ At least one blank must follow the command name “C”

COMP 6710 Course NotesSlide 9-25 Auburn University Computer Science and Software Engineering Cause-Effect Graphing Example –Outputs Changed line is printed to the terminal if the command is successful “NOT FOUND” is printed if string1 cannot be found “INVALID SYNTAX” is printed if the command syntax is incorrect –System Transformations If the syntax is valid and string1 can be found in the current line, then string1 is removed and string2 replaces it If the syntax is invalid or string1 cannot be found, the line is not changed

COMP 6710 Course NotesSlide 9-26 Auburn University Computer Science and Software Engineering Cause-Effect Graphing Example Cause 1: The first nonblank character following the “C” and one or more blanks is a ‘/’ Cause 2: The command contains exactly two ‘/’ characters Cause 3: String1 has length 1 Cause 4: String1 has length 30 Cause 5: String1 has length 2-29 Cause 6: String2 has length 0 Cause 7: String2 has length 30 Cause 8: String2 has length 1-29 Cause 9: The current line contains an occurrence of string1 Effect 1: The changed line is typed Effect 2: The first occurrence of string1 in the current line is replaced by string2 Effect 3: NOT FOUND is printed Effect 4: INVALID SYNTAX is printed

COMP 6710 Course NotesSlide 9-27 Auburn University Computer Science and Software Engineering Complete Cause-Effect Graph c1 c2 c3 c4 c5 c6 c8 c7 c9 i1 i2 i3 e1 e2 e3 e4

COMP 6710 Course NotesSlide 9-28 Auburn University Computer Science and Software Engineering Converting to a Decision Table