1. Figure 1. Basic Software Testing Words ErrorAn error is a mistake of commission or omission that a person makes. An error causes a defect. In software development one error may cause one or more defects in requirements, designs, programs, or tests. DefectAlso called a fault or a bug, a defect is an incorrect part of code that is caused by an error. An error of commission causes a defect of wrong or extra code. An error of omission results in a defect of missing code. A defect may cause one or more failures. Error of omission: missing design components OR design omission FailureA failure is a deviation from expected behaviour exhibited by software and observed as a set of symptoms by a tester or user. A failure is caused by one or more defects. The Causal TrailA person makes an error that causes a defect that causes a failure. Root Cause Analysisneeds traceability mechanism and pref. A TOOL Sources of errors:FIXES REQTS DESIGN CODE CHANGES Based on V&V FAULT MODEL

2. Process Management SEI Capability Maturity Model Level 1 2 3 4 Initial SE Repeatable Defined SE Managed SE Process Process Process Chaotic, Measurements Metrics over budget (compilers, TOOLS Testing late word processors, time (real) Tools & Powerpoint) (elapsed) Traceability # of people Tool LOCC Excel, Micro Planner 5 Optimized SE Process Continuous Improvement (modify process to achieve quality or productivity gains) REQTS & DESIGN capturing, checking, analyzing TOOLS

3. Figure 3. Sample STL Sentence ActionA01 is allowed in statenormal receives eventitemwarning_interrupt; transmits eventsafety_action1; uses dataitemwater_temp_reading, water_temp_base; produces dataitemsafety_action_report; has duration time maximum3 ; has duration time unit“second”. Tool can analyze REQTS. in Formal notation for consistency (pairwise)  ij [if Ri is satisfied, Rj can also be satisfied] Criticism: Too low level (“states”) e.g. SDL too low level in TAU? For REQTS., try use cases or MSCs.

4. Verification & Validation  SQE SQE contains management strategies –milestones –measurements techniques REQTS-based (Black Box) DESIGN-based (Grey Box) Code-based (White Box) Behaviour-based (Reliability & Robustness) Fix-based (Regression) Tools Progress quality

5. Benefit - MSC’s have - tool support - precise syntax - some checking for consistency and ambiguity - customer friendly - graphical (with abstraction) Weakness - graphical (?) - finite # of MSC’s (same as reqts. List - completeness issue) - tend to leave original reqts. behind RTM MSCs REQTS M1 M2 M3 M4 M5 M6 R1 R2 ·

6. SQE ideally with tools, REQTS gathering, representation, validation REQTS. Engineering tools (RTM) Arrival of New REQT. How does this affect integrity of R1 - R29? [not many tools for checking consistency or completeness of requirements] need for check that set of requirements is (worked phrases requirements) “must” “must not” RTM Test REQTS Cases T1 T2 T3 T4 R1 C 0 R11 0 C R12 C R2 ; R30 unambiguous consistent complete

7. Problems with RTM: Updating RTM to code and to test cases needed for functional tests · regression tests · need for new or revised tests Ideally hyper-links REQTS. DESIGN TESTS CODE Allows change documentation to flow from REQTS. A RTM tool can be very useful in keeping REQTS, DES., CODE, TESTS unambiguous, consistent  2  1  1 ?  1  2 not recommended

8. Requirements-based Testing Mapping traceability (REQTS, Tests) –test plans –scenarios or MSCs generate reqts.-based tests –test oracle [whether a test result is a PASS or FAIL or INCONCLUSIVE] –avoid obsolete tests when reqts. change, but –incur additional testing cost automate test execution measure test quality (coverage) optimize testing cost-benefit

9. Specification Languages ASN.1 Neufeld, G. and S. Vuong, “An Overview of ASN.1,” Comput. Networks ISDN Syst., Vol. 23, 1992, pp. 319-415. Estelle “A Formal Description Technique Based on an Extended State Transition Model,” Intl. Org. Standard, IS 9074, 1988. Larch Guttag, J., J. Horning, and J. Wing, “The Larch Family of Specification Languages,” IEEE Software, Vol. 2, No. 5, 1985, pp. 24-36. LOTOS “A Formal Description Technique Based on Temporal Ordering of Observed Behavior,” Intl. Org. Standard, IS 8807, 1988. SDL Saracco, R. and P. Tilanus, “CCITT SDL: Overview of the (Specification description) Language and Its Applications,” Comput. Networks ISDN Syst., Vol. 13, No. 1, 1987, pp. 65-74. STL “The Semantic Transfer Language,” in IEEE Standard 1175- 1994, Standard Reference Model for Computing System Tool Interconnections. IEEE Press, New York, pp. 37-117, 1994. Z Wordsworth, J., Software Development with Z: A Practical Approach to Formal Methods in Software Engineering, Addison-Wesley, Workingham, England, 1992. VDM Jones, C.B., Systematic Software Development Using VDM, Prentice-Hall, Englewood Cliffs, NJ, 1986.