1. PVK-HT051 Contents Introduction Requirements Engineering Project Management Software Design Detailed Design and Coding Quality Assurance Maintenance

2. PVK-HT052 Quality Assurance Introduction Testing oTesting Phases and Approaches oBlack-box Testing oWhite-box Testing oSystem Testing

3. PVK-HT053 What is Quality Assurance? Constructive vs analytic approaches to QA Qualitative vs quantitative quality standards Measurement oDerive qualitative factors from measurable quantitative factors Software Metrics QA is the combination of planned and unplanned activities to ensure the fulfillment of predefined quality standards.

4. PVK-HT054 Approaches to QA Constructive Approaches oSyntax-directed editors oType systems oTransformational programming oCoding guidelines o... Analytic approaches oInspections oStatic analysis tools (e.g. lint) oTesting o... Usage of methods, languages, and tools that ensure the fulfillment of some quality factors. Usage of methods, languages, and tools to observe the current quality level.

5. PVK-HT055 Fault vs Failure ?! human errorfaultfailure can lead to Different types of faults Different identification techniques Different testing techniques Fault prevention and -detection strategies should be based on expected fault profile

6. PVK-HT056 Specification/ requirements Environment/ support Documen- tation OtherDesignCode Fault origin: WHERE? Missing Unclear Wrong Changed Better way Fault mode: WHY? Fault type: WHAT? Requirements or specifications Functionality HW interface SW interface User interface Functional description Test HW Test SW Integration SW Development tools Logic Computation Data handling Module interface/ implementation Standards (Inter-)Process communications Data definition Module design Logic description Error checking Standards HP´s Fault Classification

7. PVK-HT057 Fault Profile of a HP Division See [Pfleeger 98].

8. PVK-HT058 Testing Phases Unit test Unit test Unit test Integration test Function test Performance test Acceptance ( ,  ) test Installation test Component code...... Tested component Integrated modules Functioning system Verified software Accepted, validated system SYSTEM IN USE! Design specifications System functional requirements Other software requirements Customer requirements specification User environment System test Pre-implementation test

9. PVK-HT059 Pre-Implementation Testing Inspections oevaluation of documents and code prior to technical review or testing Walkthrough oIn teams oExamine source code/detailed design Reviews oMore informal oOften done by document owners Advantages oEffective oHigh learning effect oDistributing system knowledge Disadvantages o Expensive

10. PVK-HT0510 Integration Testing How to integrate & test the system Top-down Bottom-up Critical units first Functionality-oriented (threads) Implications of build order Top-down => stubs; more thorough top-level Bottom-up => drivers; more thorough bottom- level Critical => stubs & drivers. A DCB GFE Test E Test F Test G Test B,E,F Test C Test D,G Test all

11. PVK-HT0511 Further phases of testing System testing: Test the functionality, performance, reliability, security of the entire system. Acceptance testing: Operate system in user environment, with standard user input scenarios. Regression testing: Test of modified versions of previously validated system.

12. PVK-HT0512 Testing vs “Proofing” Correctness Verification oCheck the design/code against the requirements Are we building the product right? Validation oCheck the product against the expectations of the customer Are we building the right product? Testing Testing can neither proof that a program is error free, nor that it is correct [Myers 76] Testing can only show the presence of errors, never their absence. [Dijkstra 69] Testing is the process in which a (probably unfinished) program is executed with the goal to find errors.

13. PVK-HT0513 Testing Principles Construction of test suites oPlan tests under the assumption to find errors oTry typical and untypical inputs oBuild classes of inputs and choose representatives of each class Carrying out tests oTesters  implementers oDefine the expected results before running a test oCheck for superfluous computation oCheck test results thoroughly oDocument test thoroughly Simplify test oDivide programs in separately testable units oDevelop programs test friendly Each test must be reproducible

14. PVK-HT0514 Test Methods Structural testing (white-box, glass-box) oUses code/detailed design to develop test cases oTypically used in unit testing oApproaches: Coverage-based testing Symbolic execution Data flow analysis... Functional testing (black-box) oUses function specifications to develop test cases oTypically used in system testing oApproaches: Equivalence partitioning Border case analysis... time develop black-box test cases develop white-box test cases perform white-box testing perform black-box testing

15. PVK-HT0515 Test Preparation Exhaustive testing is prohibited, because of the combinatorial explosion of test cases Choose representative test data for i := 1 to 100 do if a = b then X else Y; ipaths to test#tests 1X, Y2 2XX, XY, YX, YY4 3XXX, XXY,...8... 1002 100 2  2 100 - 2 > 2,5  10 30 With 10 6 tests/sec this would take 8  10 16 years Choose test data (test cases)

16. PVK-HT0516 How to Choose Test Data Example 1 Both paths must be tested! Example 2 How can I know there is a “path”? if ((x + y + z)/3 = x) then writeln( “x, y, z are equal”) else writeln( “x, y, z are unequal”); Test case 1:x=1, y=2, z=3 Test case 2:x=y=z=2 if (d = 0) then writeln( “division by zero”) else x = y/n; (*-----------------------------*) x = y/n;

17. PVK-HT0517 Test Case Development Problems: oSystematic way to develop test cases oFind a satisfying set of test cases Test case  test data Test data: Inputs devised to test the system Test case: oSituation to test oInputs to test this situation oExpected outputs è Test are reproducible è Equivalence partitioning è Coverage-based testing

18. PVK-HT0518 Equivalence Partitioning System Input data Output data Inputs causing anomalous behaviour Outputs which reveal the presence of faults Input- and output data can be grouped into classes where all members in the class behave in a comparable way. Define the classes Choose representatives u Typical element u Borderline cases x  [25.. 100] class 1:x < 25 class 2: x >= 25 and x <= 100 class 3: x > 100

19. PVK-HT0519 Equivalence Partitioning Examples oModule keeps a running total of amount of money received oInput: AmountofContribution, specified as being a value from $0.01 to $99,999.99 oModule prints an appropriate response letter oInput: MemberStatus, specified as having a value of {Regular, Student, Retiree, StudioClub} o 3 equivalence classes: Values from $0.01 to $99,999.99 (valid) Values less than $0.01 (invalid) Values greater than $99,999.99 (invalid) o 2 equivalence classes: Values of {Regular, Student, Retiree, StudioClub} (valid) All other input (invalid)

20. PVK-HT0520 Statement Coverage 1 3 2 4 5 6 7 8 2 test cases: 12467; 13567 Every statement is at least executed once in some test

21. PVK-HT0521 Branch Coverage 1 3 2 4 5 6 7 8 2 test cases: 12467; 1358 For every decision point in the graph, each branch is at least chosen once

22. PVK-HT0522 Condition Coverage Test all combinations of conditions in boolean expressions at least once if (X or not (Y and Z) and... then b; c := (d + e * f - g) div op( h, i, j);

23. PVK-HT0523 Path Coverage Assure that every distinct paths in the control-flow graph is executed at least once in some test

24. PVK-HT0524 Path Coverage 1 3 2 4 5 6 7 8 Assure that every distinct paths in the control-flow graph is executed at least once in some test

25. PVK-HT0525 Path Coverage 1 3 2 4 5 6 7 8 4 test cases: 12467; 1358; 1248; 13567 Assure that every distinct paths in the control-flow graph is executed at least once in some test

26. PVK-HT0526 Test Coverage–Example Statement Coverage: 5/10 = 50% Branch Coverage: 2/6 = 33% Path Coverage: 1/4 = 25%

27. PVK-HT0527 Data-flow testing Def-use analysis: match variable definitions (assignments) and uses. Example: x = 5; … if (x > 0)... Does assignment get to the use?

28. PVK-HT0528 Data Flow Coverage All-uses coverage x :=2 x :=1 z := 2*r x :=3 z := 2*x-y z :=x+y y :=2 r :=4 Red path covers the defs y :=2; r :=4; x :=1 Blue path covers y :=2; x :=3. Does not cover x :=2

29. PVK-HT0529 Coverage-based Testing Advantages oSystematic way to develop test cases oMeasurable results (the coverage) oExtensive tool support Flow graph generators Test data generators Bookkeeping Documentation support Disadvantages oCode must be available oDoes not (yet) work well for data-driven programs

30. PVK-HT0530 Testing Tools / Support Test data generators oInput: Program + testing strategy oOutput: Sets of input data Profilers oInstrument code to collect run-time data Time spent in operations Number of calls to operations Number of loop iterations... è Find bottle-necks è Indicate dead code Simulators oCommon in hard-/software systems and/or real- time systems oEmulate critical parts by software

31. PVK-HT0531 Testing Tools / Support Debuggers oManual code instrumentation oInspect/trace variables o... File comparators oE.g. for regression testing Test-stub/-driver generators oSimulate client or server components, which are not yet available ClientServer