CPIS 357 Software Quality & Testing I.Rehab Bahaaddin Ashary Faculty of Computing and Information Technology Information Systems Department Fall 2010

Lecture Objectives Chapter 1 :Basic Concepts and Preliminaries Central Issue in Testing Testing Activities Test Levels Sources of Information for Test Case Selection White- Box and Black – Box Testing Test Planning and Design Monitoring and Measuring Test Execution Test Tools and Automation Test Team Organization and Management

Central Issue in Testing Figure 1: A subset of the input domain exercising a subset of the program behavior Divide the input domain D into D1 and D2 Select a subset D1 of D to test program P It is possible that D1 exercise only a part P1 of P Selection of the subset of the input domain must be done in a systematic and careful manner so that the deduction is as accurate and complete as possible. For example. The idea of coverage is considered while selecting the test case.

Figure 2: Different activities in process testing Testing Activities Figure 2: Different activities in process testing Identify the objective to be tested. (Purpose designing one or more test case) Select inputs.(Requirements specification, source code) Compute the expected outcome.(High Level understanding both objective and specification) Set up the execution environment of the program. (Local/Remote) Execute the program.(Different location at different time) Analyze the test results.(Pass/Fail/Inconclusive)

Figure 3: Development and testing phases in the V model Test Levels Unit testing Individual program units, such as procedure, methods, Function and Classes in isolation Integration testing Modules are assembled to construct larger subsystem and tested by software developer and test engineer System testing Includes wide spectrum of testing such as functionality, Security, Stability, Performance, reliability and load Acceptance testing Customer’s expectations from the system Two types of acceptance testing UAT: System satisfies the contractual acceptance criteria BAT: System will eventually Measure the quality of the product rather than searching for the defect. Figure 3: Development and testing phases in the V model

Figure 4: Regression testing at different software testing levels Test Levels Figure 4: Regression testing at different software testing levels New test cases are not designed Test are selected, prioritized and executed To ensure that nothing is broken in the new version of the software

Source of Information for Test Case Selection Requirement and Functional Specifications Capturing user needs. Informal or formal manner Examples: plaintext, flowchart, figures, equations, use case, entity relationship diagram or class diagram Source Code Internal behavior of the system. By software engineer (for the design details) and programmer add more details Example: Sort Function Input and Output Domain Example: factorial (0) =1; factorial (1) =1 ; factorial (n) =n* factorial (n-1) Operational Profile Is a quantitative characterization of how the system will be used and guide test engineers in selecting test case input. Fault Model Error Guessing Fault Seeding (Fault Injection) Mutation Analysis (Fault Simulation)

White-box and Black-box Testing White Box Testing ( Structural Test) Black Box Testing (Functional Test) Examines source code with focus on: Control Flow Data Flow Control flow refers to flow of control from one instruction to another. Data flow refers to propagation of values from one variable or constant to another variable. It is applied to individual units of a program. Software developers perform structural testing on the individual program units they write Examines the program that is accessible from outside Applies the input to a program and observe the externally visible outcome. It is applied to both an entire program as well as to individual program units. It is performed at the external interface level of a system. It is conducted by a separate software quality assurance group.

Test Planning and Design The purpose is to get ready and organized for test execution. A test plan provides a: Framework A set of ideas, facts or circumstances within which the tests will be conducted. Scope The test activities and the outline of the domain. Details of resource needed Effort required Schedule of activities Budget During the test design phase the system requirements are critically studied, system features to be tested and the objective and the detailed behavior of test cases are defined. Test objectives are identified from different sources. Each test case is designed as a combination of modular test components called test steps. Test steps are combined together to create more complex tests.

Monitoring and Measuring Test Execution Metrics for monitoring test execution (concern on test). Metrics for monitoring defects (concern on the result from the test). Needed to be tracked and analyzed in a periodic basic (daily or weekly). Metrics is meaningful if enable management to make decision making which reduce delay, improve quality and lower the cost. The evaluation done through the following: Evaluate the effectiveness, Evaluate productivity, Evaluate the quality, Evaluate the product. Test case effectiveness metrics Measure the “defect revealing ability” of the test suite Use the metric to improve the test design process Test-effort effectiveness metrics Number of defects found by the customers that were not found by the test engineers After product Developed.

Test Tools and Automation strengths Prerequisites List Increased productivity of the testers. Better coverage of regression testing. Reduced durations of the testing phases. Reduced cost of software maintenance. Increased effectiveness of test cases. The test cases to be automated are well defined. Test tools and an infrastructure are in place. The test automation professionals have prior successful experience in automation. Adequate budget have been allocation for the procurement of software tools. Weaknesses Long learning Curves. High time and cost for test preparation. Test tool limitation.

Figure 5: Structure of test groups Test Team Organization and Management Figure 5: Structure of test groups Unit Testing  Programmer System Integration  Test engineer System test group  must be separated from development group and they put the plan for the UAT, BAT Acceptance  Client special group (SW quality assurance engineer, business associate, customer support engineer) Hiring and retaining test engineers is a challenging task. Interview is the primary mechanism for evaluating applicants.