1. Testing Terminology Project Group Org. Review Terminology OO Concepts Development Products Team Meeting CEN 5076 Class 2 – 09/12

2. CEN 5076 Class 2 - 09/122 What is software? Software is the instruction codes and data necessary to accomplish some task on a computer. Includes all representations and artifacts of the code and data i.e., –program source code and data files –models created during analysis and design activities.

3. CEN 5076 Class 2 - 09/123 Software Process S/w Specification – requirements elicitation (func. & non-func.) and analysis. S/w Development – systems design, detailed design (OO design), implementation. S/w Validation – validating system against requirements (testing). S/w Evolution – meet changing customer needs and error correction (maintenance).

4. CEN 5076 Class 2 - 09/124 Unified Software Development Process Analysis model Use case model Design model Deployment model Implementation model Test model specified by realized by distributed by verified by implemented by Process is use case driven! Requirements captured as a set of use cases. All models are related through traceability dependencies. System Development

5. CEN 5076 Class 2 - 09/125 View of UDSP from a Testing Perspective Analysis model Use case model Design model Deployment model Implementation model Test model specified by realized by distributed by Verified and/or validate implemented by Process is use case driven! Requirements captured as a set of use cases. All models are related through traceability dependencies. System Development

6. CEN 5076 Class 2 - 09/126 What is software testing? Software testing is the process of operating software under specified conditions, observing or recording the results and making an evaluation of some aspect of the software. (IEEE/ANSI std 610.12-1990)

7. CEN 5076 Class 2 - 09/127 What is software testing? Software testing is the process of uncovering evidence of defects in software systems. A defect can be introduced during any phase of development or maintenance and results from one or more “bugs” – mistakes, misunderstandings, omissions or even misguided intent on the part of the developers. (McGregor & Sykes 2001)

8. CEN 5076 Class 2 - 09/128 Testing Terminology The following defns are taken from Binder 2000 and McGregor & Sykes 2001. Failure – is the manifested inability of a system or component to perform a required function within specified limits e.g. abnormal termination, or unmet time and space constraints of the software. A fault is missing or incorrect code. An error is a human action that produces a fault.

9. CEN 5076 Class 2 - 09/129 Testing Terminology cont A bug is an error or a fault. Debugging is the process of tracking down the source of failures (errors and/or faults) and making repairs. A test case specifies the pretest state of the component under test (CUT) and its environment, the test inputs and conditions, and the expected results. A test point is a specific value for test case input state variables.

10. CEN 5076 Class 2 - 09/1210 Testing Terminology cont A test suite is a collection of test cases, typically related to some testing goal or criterion. A test driver is a class or utility program that applies test cases to a CUT. A stub is a partial, temporary implementation of a component. A test harness is a system of test drivers and other tools to support test execution.

11. CEN 5076 Class 2 - 09/1211 Testing Approach (McGregor & Sykes) Kinds of testing –Model testing –Class testing, replaces unit testing –Interaction testing, replaces integration testing –System (and subsystem) testing –Acceptance testing –Deployment/self testing Motto: Test early, Test often, Test enough

12. CEN 5076 Class 2 - 09/1212 The Testing Perspective Testing perspective - looking at any development product and questioning its validity. Software testing accomplished by a combination of inspections, reviews, and test executions. Inspection: –Defn: A formal small group procedure for detailed scrutiny of a technical work product. –Cost effective and efficient for defect prevention. –Process has a formal entry-exit criteria, defined checklist, captures process and product metrics, has a follow up session.

13. CEN 5076 Class 2 - 09/1213 The Testing Perspective cont Review: –Defn: A meeting at which a work product or set of work products is presented to project personnel, managers, users, customers, or other interested parties for commitment and approval. [IEE 610] –Types of review: client, project, peer (inspections and walkthroughs), status. –Examines the software with the purpose of finding errors and faults even before the software is executed. –Deeper interest in s/w than do inspections.

14. CEN 5076 Class 2 - 09/1214 The Testing Perspective cont Test execution: –Defn: Testing the s/w in the context of a running program. –Challenges: identifying suitable input (test data selection criteria and test adequacy criteria), determining correct outputs (oracle problem), determining how to observe the outputs Walkthrough (informal review) –A group meeting to conduct an informal peer review of a technical work product.

15. CEN 5076 Class 2 - 09/1215 OO Concepts Before testing software it is important to know the paradigm used to develop the software. Some OO concepts are: –Object –Message –Interface –Class –Inheritance –Polymorphism

16. CEN 5076 Class 2 - 09/1216 OO Concepts - Definitional and Operational Semantics Dynamic Operational Definitional Static Bound at runtime Bound at compile time Object Bound at Analysis time Interface/Class Definitions

17. CEN 5076 Class 2 - 09/1217 OO Concepts cont Object - an operational entity that encapsulates both specific data values an the code that manipulates those values. –Basic computational entity. –Community of objects collaborate to solve some problem. –During program execution objects are created, modified, accessed, and/or destroyed.

18. CEN 5076 Class 2 - 09/1218 OO Concepts cont Object (testing perspective)  Encapsulation - makes the complete defn of an object “easy” to identify, easy to pass around the system, and easy to manipulate.  Information hiding – sometimes makes observation of changes to object difficult i.e., checking the state of the object.  State – can become inconsistent and become source of errors.  Lifetime – must check state during life time.

19. CEN 5076 Class 2 - 09/1219 OO Concepts cont Message (member function call - C++, method invocation - Java) – is a request (from sender) that an operation be performed by some object (receiver). –Pass data via parameters (value and/or reference). –Receiver can return value to sender. –Collaboration achieved by sending messages between objects.

20. CEN 5076 Class 2 - 09/1220 OO Concepts cont Message (testing perspective) –Sender - may send the message at the incorrect time. –Receiver - may not be ready for a specific message and take corrective action. –Actual parameters – used to update the state of the receiver; must be in correct state before and after message is processed. –Sender – must use correct interface for intended message.

21. CEN 5076 Class 2 - 09/1221 OO Concepts cont Interface – is an aggregation of behavioral declarations, related by a single concept. –Building blocks for specifications. –Specification defines the total set of public behaviors for a class. –Java has interface, C++ has abstract base class with all public pure virtual member functions.

22. CEN 5076 Class 2 - 09/1222 OO Concepts cont Interface (testing perspective) –Encapsulates operation specifications. –Used to incrementally build the spec. of larger grouping e.g., classes, packages. –Closely related to the design model. –Relationships with other interfaces and classes identifies most dependencies between classes.

23. CEN 5076 Class 2 - 09/1223 OO Concepts cont Class – set of objects that share a common conceptual basis. –Template for creating objects. –Type (user-defined or built-in) = set + operations define over the set –Used to define OO programs –Object instance of a class. –Class specification: the what –Class implementation: the how

24. CEN 5076 Class 2 - 09/1224 OO Concepts – Class Spec. Class Specification – describes what the class represents and what an instance of the class can do. –Includes a specification for each operation of an instance. –Operation is an action that can be applied to an object to obtain a certain effect. –Operations are categorized as follows:

25. CEN 5076 Class 2 - 09/1225 OO Concepts – Class Spec. cont  Accessor (or inspector) – provides info about an object  Modifier (or mutator) – changes the state of an object.  Constructor (class operation) - creates a new object and initializes the new instance (data and methods).  Destructor – performs processing on object prior to the end of the object’s lifetime.

26. CEN 5076 Class 2 - 09/1226 OO Concepts – Class Spec. cont Operation semantics are specified using precondition, postconditions, and invariants. Preconditions – prescribe conditions that must hold before the operation can be performed. Stated in terms of: 1.attributes of receiver object, and/or 2.actual parameters of the message requesting the operation.

27. CEN 5076 Class 2 - 09/1227 OO Concepts – Class Spec. cont Specifying operation semantics: Postconditions – prescribe conditions that must hold after the operation is performed. Stated in terms of: 1.attributes of the receiver object, 2.actual parameters in the sender’s message, 3.value of any return value, and/or 4.exceptions that might be raised.

28. CEN 5076 Class 2 - 09/1228 OO Concepts – Class Spec. cont Specifying operation semantics: Invariants – prescribe conditions that must always hold during the lifetime of an object. A class invariant describes a set of operating boundaries. -Treated as an implied postcondition for each operation. -Must hold whenever an operation completes. -May violate invariants during execution of operation -Can define interface invariants.

29. CEN 5076 Class 2 - 09/1229 OO Concepts – Class Spec. cont In this course we will be using OCL to specify operation semantics. See handout given in class. When you submit the final project all classes and operations should be specified by invariants, preconditions and postconditions. We will use the OCL statement to generate unit test cases. Start writing OCL statements to specify classes now!!!

30. CEN 5076 Class 2 - 09/1230 OO Concepts – Class Spec. cont Contracts are constraints on a class that enable a sender (caller) and a receiver (callee) to share the same assumptions about the class (Meyer ’97). A contract specifies constraints that the caller must meet before using the class as well as constraints that are ensured by the callee when used. Operation semantics are specified using precondition, postconditions, and invariants.

31. CEN 5076 Class 2 - 09/1231 OO Concepts – Class Spec. cont Two basic approaches to write specs for operations between the receiver and sender: 1.Contract – emphasizes preconditions with simple postconditions. Example P. 26 2.Defensive programming – emphasizes the postconditions with simple preconditions. Example P. 28. The decision on which approach(es) is used to write specs is made at the design phase.

32. CEN 5076 Class 2 - 09/1232 OO Concepts – Class Spec. cont Design by contract – views the relationship between a class and its clients as a formal agreement, expressing each party’s rights and obligations (Meyer ’97). –The precondition is an obligation for the client (sender) and a benefit for the supplier (receiver). –The postcondition is a benefit for the client and an obligation for the supplier.

33. CEN 5076 Class 2 - 09/1233 OO Concepts – Class Spec. cont Defensive design – intended to ensure the continuing function of a piece of software in spite of unforeseeable usage of said software (Wikipedia). –Results in defensive programming. –Supports security e.g., buffer overflow –Operation typically returns some indication concerning the status of the result of the request – success or failure. –Uses exception handling mechanism to support above

34. CEN 5076 Class 2 - 09/1234 OO Concepts – Class Spec. cont ContractDefensive Prog. Mutual responsibility between sender and receiver. Eliminate need of receiver to verify preconditions. How to enforce contract? Simplifies class testing. Complicates interaction between classes. Lack of trust between sender and receiver. Increase size of code and execution time. “Garbage in eliminate garbage out.” Complicates class testing. Complicates interaction testing.

35. CEN 5076 Class 2 - 09/1235 OO Concepts – Class Spec. cont Contract –Complicated interaction testing: must ensure that any sender meets preconditions. Defensive Programming –Complicated class testing: test cases must address all possible outcomes –Complicated interaction testing: must ensure all possible outcomes are produced and that they are properly handled by a sender.

36. CEN 5076 Class 2 - 09/1236 OO Concepts – Class Impl. Class implementation - describes how an object represents attributes and carries out operations. Components: –Data members (instance variables or attributes): store the state of an instance. –Methods (member functions or routines): code used to implement some algorithm that accomplishes one operation. –The data members and methods exhibit properties that can affect the testing process.

37. CEN 5076 Class 2 - 09/1237 OO Concepts – Class Impl. Properties of data members and methods (features): –Types (built-in, user-defined) –Accessibility (private, protected, private) –Shared class features (static) –Polymorphism –Dynamic binding –Deferred features –Exception handling –Concurrency

38. CEN 5076 Class 2 - 09/1238 OO Concepts – Testing perspective Potential causes of failure within design and impl.: Initialization of the attributes in a new instance. Implementation of operations. Does implementation “satisfy” the specification? Implementation may not support all required operations. Constraints need to be written in a “testable” manner. This is why we will use OCL.

39. CEN 5076 Class 2 - 09/1239 OO Concepts – Inheritance Inheritance is a relationship between classes that allows for the definition of a new class based on the definition of an existing class. –Allows the reuse of both specification and implementation. –New class referred to as subclass or derived class (descendents). –Existing class is referred to as superclass or base class (ancestors). –The inheritance relation is represented as an inheritance hierarchy.

40. CEN 5076 Class 2 - 09/1240 OO Concepts – Inheritance –Some languages allow multiple inheritance. –Abstract classes? –Concrete subclasses? –Deferred features? –Public, protected and private inheritance? –Virtual inheritance? Focus of inheritance in object design is to reduce redundancy and enhance extensibility.

41. CEN 5076 Class 2 - 09/1241 OO Concepts – Inheritance Factoring all redundant behavior into a single superclass reduces the risk of inconsistencies when changes are made. Use of abstract classes and interfaces allows for extensibility i.e., new specialized behavior can be written in new subclasses. Inheritance is a powerful mechanism and should be carefully considered before use.

42. CEN 5076 Class 2 - 09/1242 OO Concepts – Inheritance Note there is usually strong coupling between classes in the inheritance hierarchy. There are two types of inheritance: 1.Implementation inheritance – sole purpose of reusing code. 2.Specification inheritance (or interface inheritance) – classification of concepts into type hierarchies.

43. CEN 5076 Class 2 - 09/1243 OO Concepts – Inheritance Liskov Substitution Principle: If an object of type D can be substituted in all places where an object of type C is expected then D is a subtype of C ( strict inheritance ). –Models the is a (or is a kind of) relationship. –Promotes specification inheritance. –Allowed behavior for a new subclass D derived from C: Preconditions in D must be the same or weaker than in C.

44. CEN 5076 Class 2 - 09/1244 OO Concepts – Inheritance Postconditions in D must be the same or stronger than in C. Class invariant in D must be the same or stronger than in C. Viewed from perspective of observable states: -All observable states and transitions must be preserved between D and C. -D may add transitions between these states. -D may add observable states as long as each is either concurrent or a sub-state of an existing state.

45. CEN 5076 Class 2 - 09/1245 OO Concepts – Inheritance Testing perspective: Bugs can be propagated. May reuse test cases, because derived classes inherits part of its spec. and impl. from its superclass. Models is a kind of relationship i.e., specification inheritance. Proper use of inheritance in design leads to benefits in execution and testing of classes.

46. CEN 5076 Class 2 - 09/1246 OO Concepts – Polymorphism Polymorphism is the ability to treat an object as belonging to more than one type. There are different types of polymorphism: inclusion and parametric. 1.Inclusion polymorphism – the occurrence of different forms of the same class. Gives the programmer the ability to substitute an object of the same type in a request for an operation. –Polymorphism arises from inheritance. –C++ inheritance of classes –Java inheritance of classes and interfaces

47. CEN 5076 Class 2 - 09/1247 OO Concepts – Polymorphism Testing perspective for inclusion inheritance: –Allows systems to be extended incrementally by adding classes; unanticipated behavior may occur. –Allows any operation to have one or more parameters of a polymorphic reference; increases number of possible kinds of actual parameters to be tested. –Allows the operation to specify replies that are polymorphic references; referent could be incorrect or unanticipated by the sender.

48. CEN 5076 Class 2 - 09/1248 OO Concepts – Polymorphism 2.Parametric polymorphism – is the capability to define a type in terms of one or more parameters. -C++ templates: instantiation of a “new” class at compile-time. Testing perspective: -Supports a different type of relationship from inheritance. -Must make sure operations work for all possible instances of the “new” class. Should be done during static analysis e.g., walkthrough.

49. CEN 5076 Class 2 - 09/1249 Development Products - UML Diagrams A diagram is a graphical representation of a set of elements, most often rendered as a connected graph of vertices (things) and arcs (relationships). Diagrams in UML – structural diagrams (static)  Class – shows a set of classes, interfaces, and collaborations and their relationships.  Object – class diagram that includes only instances.  Component – class diagrams that focus on system’s components e.g., executables, libraries, tables, files …  Deployment – shows the configuration of run-time processing nodes and the components that live on them

50. CEN 5076 Class 2 - 09/1250 Diagrams in UML cont Diagrams in UML – behavioral diagrams (dynamic)  Use case - represents the actors and uses of the system and relationships between the uses.  Sequence - interaction diagram that focuses on the time ordering of messages.  Collaboration - interaction diagram that focuses on the structural organization of objects that send and receive messages  Statechart – shows a state machine that shows different states (data-attribute values) of the object driven by events (messages).  Activity – shows flow of control from activity to activity, emphasizes flow of control among objects.

51. CEN 5076 Class 2 - 09/1251 Development Products - Analysis model Good documentation is critical for successful development and successful testing. Analysis model Purpose to define the problem to be solved and determine the requirements for a solution to the problem. Two levels of analysis: –Domain analysis focuses on understanding the problem domain. –Application analysis focuses on a specific problem and the requirements for a solution.

52. CEN 5076 Class 2 - 09/1252 Development Products - Analysis model Analysis model represents a system from the perspective of what it is suppose to do. Analysis model provides an understanding for the various stakeholders. The diagrams in the analysis model should be checked for: –Accuracy –Consistency –Completeness

53. CEN 5076 Class 2 - 09/1253 Development Products - Analysis model Models in the analysis model:  Functional model – represented by use cases and scenarios.  Analysis object model – represented by class and object diagrams.  Dynamic model – represented by statechart and sequence diagrams. Entity, boundary and control objects are identified. Generalization and specification activities performed.

54. CEN 5076 Class 2 - 09/1254 Development Products – Design Model Design model Represents how the software meets the requirements. Refines and extends the analysis model. Can reuse and extend test cases. System design:  Identification of design goals Performance – response time, throughput, memory. Dependability – robustness, reliability, availability, fault tolerance, security, safety. Maintenance – extensibility, modifiability, adaptability, portability, readability, traceability of reqs.

55. CEN 5076 Class 2 - 09/1255 Development Products – Design Model  Software architecture describing subsystem decomposition: responsibilities, dependencies, mapping to h/w, policy decisions – control flow, access control and data storage.  Boundary use cases identified. Object design  Reuse - COTS identified, class libraries and other components selected, design patterns identified.  Interface specification – operations, arguments, type signatures, exceptions (Here we use OCL).

56. CEN 5076 Class 2 - 09/1256 Development Products – Design Model  Restructuring – manipulate system model to meet reuse and other design goals.  Optimization – changing algorithms, optimize data associations. Many of the diagrams used in the analysis model are used, but with an emphasis on the solution and not the problem.

57. CEN 5076 Class 2 - 09/1257 Development Products – Impl. Model Implementation model Involves mapping design models to source code i.e., a number of transformations are performed on the model. Transformations include:  Optimization of the system model to address performance issues.  Realizing associations – map associations to source code constructs such as references and collections of references.

58. CEN 5076 Class 2 - 09/1258 Development Products – Impl. Model  Mapping contracts to exceptions – what happens when the contract is broken i.e., raising exceptions, throwing exceptions.  Mapping class models to a storage schema – select persistent data storage e.g. DBMS Mapping concepts  Model transformations – e.g. attribute address to a class address.  Refactorings – improve a single aspect of the source code

59. CEN 5076 Class 2 - 09/1259 Development Products – Impl. Model  Forward engineering – production of source code templates.  Reverse engineering – produces a model form the code.

60. CEN 5076 Class 2 - 09/1260 Laws of Testing Handbook of SSE Online debugging is more efficient than offline debugging (Sackman’s). Testing can show the presence but not the absence of errors (Dijkstra). A developer is unsuited to test his or her code (Weinberg’s). Approximately 80 percent of defects come from 20 percent of modules (Paret-Zipf-type) Performance testing benefits significantly from system-level benchmarks (Gray-Serlin)

61. CEN 5076 Class 2 - 09/1261 Laws of Testing Handbook of SSE Usability is quantifiable (Nielsen-Norman law). Partition testing is more effective than random testing (Gutjahr’s hypothesis). The adequacy of a coverage criterion can only be intuitively defined (Weyuker’s hypothesis). The test suite needed to verify an arithmetic path expression can be determined (Endres- Glatthaar hypothesis). Suspicion-based testing can be more effective than most other approaches (Hamlet’s hypothesis).