Chapter 9 Testing the System Shari L. Pfleeger Joann M. Atlee

Slides:



Advertisements
Similar presentations
Test process essentials Riitta Viitamäki,
Advertisements

Database Planning, Design, and Administration
Ch-11 Project Execution and Termination. System Testing This involves two different phases with two different outputs First phase is system test planning.
Chapter 10 Delivering the System Shari L. Pfleeger Joann M. Atlee 4 th Edition.
Chapter 9 Testing the System, part 2. Testing  Unit testing White (glass) box Code walkthroughs and inspections  Integration testing Bottom-up Top-down.
Lecturer: Dr. AJ Bieszczad Chapter 99-1 Causes of faults during development.
MSIS 110: Introduction to Computers; Instructor: S. Mathiyalakan1 Systems Design, Implementation, Maintenance, and Review Chapter 13.
SE 555 – Software Requirements & Specifications Introduction
Swami NatarajanJuly 14, 2015 RIT Software Engineering Reliability: Introduction.
Testing - an Overview September 10, What is it, Why do it? Testing is a set of activities aimed at validating that an attribute or capability.
Galin, SQA from theory to implementation © Pearson Education Limited Chapter 13 CASE Tools and their Effect on Software Quality.
Chapter 11: Testing The dynamic verification of the behavior of a program on a finite set of test cases, suitable selected from the usually infinite execution.
Senior Design – Acceptance Test Plan Review The goal is to: define the criteria for approving the application. Tightly coupled to the Requirements document.
CSC230 Software Design (Engineering)
LAN/WAN Optimization Techniques. Agenda Current Traffic Current Traffic Equipment Inventory and Forecasted Growth Equipment Inventory and Forecasted Growth.
Issues on Software Testing for Safety-Critical Real-Time Automation Systems Shahdat Hossain Troy Mockenhaupt.
Pfleeger and Atlee, Software Engineering: Theory and Practice CS499 Chapter 9 Testing the System Shari L. Pfleeger Joann M. Atlee 4 th Edition.
Functional Testing Test cases derived from requirements specification document – Black box testing – Independent testers – Test both valid and invalid.
Chapter 9 Testing the System Shari L. Pfleeger Joann M. Atlee
Software Testing Verification and validation planning Software inspections Software Inspection vs. Testing Automated static analysis Cleanroom software.
System Testing There are several steps in testing the system: –Function testing –Performance testing –Acceptance testing –Installation testing.
Database System Development Lifecycle © Pearson Education Limited 1995, 2005.
ISBN Prentice-Hall, 2006 Chapter 10 Delivering the System Copyright 2006 Pearson/Prentice Hall. All rights reserved.
CH09: Testing the System to ensure the system does what the customer wants it to do: * Principles of System Testing * Function Testing * Performance Testing.
Information Systems Security Computer System Life Cycle Security.
Chapter 9 Testing the System. Chapter 9  Function testing  Performance testing  Acceptance testing  Installation testing  Test documentation  Testing.
Module CC3002 Post Implementation Issues Lecture for Week 6 AY 2013 Spring.
RUP Implementation and Testing
Unit 8 Syllabus Quality Management : Quality concepts, Software quality assurance, Software Reviews, Formal technical reviews, Statistical Software quality.
Software Metrics - Data Collection What is good data? Are they correct? Are they accurate? Are they appropriately precise? Are they consist? Are they associated.
Software testing basic. Main contents  Why is testing necessary?  What is testing?  Test Design techniques  Test level  Test type  How to write.
Chapter 4 Requirements engineering Chapter 4 – Requirements Engineering Lecture 1 1.
Software Testing Testing principles. Testing Testing involves operation of a system or application under controlled conditions & evaluating the results.
ISBN Prentice-Hall, 2006 Copyright 2006 Pearson/Prentice Hall. All rights reserved. Unit 8 Testing the System.
 Chapter 6 Architecture 1. What is Architecture?  Overall Structure of system  First Stage in Design process 2.
Principles of Information Systems, Sixth Edition Systems Design, Implementation, Maintenance, and Review Chapter 13.
Configuration Management (CM)
Testing Basics of Testing Presented by: Vijay.C.G – Glister Tech.
1 The Software Development Process  Systems analysis  Systems design  Implementation  Testing  Documentation  Evaluation  Maintenance.
Dr. Tom WayCSC Testing and Test-Driven Development CSC 4700 Software Engineering Based on Sommerville slides.
Principles of Information Systems, Sixth Edition Systems Design, Implementation, Maintenance, and Review Chapter 13.
Unit 7 Chapter 8 Testing the Programs. Unit 7 Requirements Read Chapters 8 and 9 Respond to the Unit 7 Discussion Board (25 points) Attend seminar/Take.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 22 Slide 1 Software Verification, Validation and Testing.
Testing the System 中国科学技术大学软件学院 孟宁 2010年12月.
Capturing the requirements  Requirement: a feature of the system or a description of something the system is capable of doing in order to fulfill the.
Principles of Computer Security: CompTIA Security + ® and Beyond, Third Edition © 2012 Principles of Computer Security: CompTIA Security+ ® and Beyond,
Principles of Information Systems, Sixth Edition 1 Systems Design, Implementation, Maintenance, and Review Chapter 13.
PRESENTER PRIYANKA GUPTA.  Testing the complete system with respect to requirements.  In System testing, the functionalities of the system are tested.
Chapter 1: Fundamental of Testing Systems Testing & Evaluation (MNN1063)
1 Software Testing Strategies: Approaches, Issues, Testing Tools.
1 The Software Development Process ► Systems analysis ► Systems design ► Implementation ► Testing ► Documentation ► Evaluation ► Maintenance.
HNDIT23082 Lecture 09:Software Testing. Validations and Verification Validation and verification ( V & V ) is the name given to the checking and analysis.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 23 Slide 1 Software testing.
System Testing 12/09. Hierarchy of Testing Testing Program Testing Top Down Bottom Up Integration TestingUnit Testing System Testing Big Bang Sandwich.
Lecturer: Eng. Mohamed Adam Isak PH.D Researcher in CS M.Sc. and B.Sc. of Information Technology Engineering, Lecturer in University of Somalia and Mogadishu.
CASE Tools and their Effect on Software Quality
Chapter 9 Testing the System 9.1 Principles of System Testing Focus A: The objective of unit and integration ensure the code implemented the design.
Software Testing Strategies for building test group
SOFTWARE TESTING Date: 29-Dec-2016 By: Ram Karthick.
PREPARED BY G.VIJAYA KUMAR ASST.PROFESSOR
Testing the System.
Systems Analysis and Design
Frequently asked questions about software engineering
Chapter 5 Designing the Architecture Shari L. Pfleeger Joanne M. Atlee
Lecture 09:Software Testing
Testing and Test-Driven Development CSC 4700 Software Engineering
Progression of Test Categories
Course: Module: Lesson # & Name Instructional Material 1 of 32 Lesson Delivery Mode: Lesson Duration: Document Name: 1. Professional Diploma in ERP Systems.
Chapter 9 Testing the System Shari L. Pfleeger Joann M. Atlee
Chapter 9 Testing the System Shari L. Pfleeger Joann M. Atlee
Presentation transcript:

Chapter 9 Testing the System Shari L. Pfleeger Joann M. Atlee 4th Edition

Contents 9.1 Principles of system testing 9.2 Function testing 9.3 Performance testing 9.4 Reliability, availability, and maintainability 9.5 Acceptance testing 9.6 Installation testing 9.7 Automated system testing 9.8 Test documentation 9.9 Testing safety-critical systems 9.10 Information systems example 9.11 Real-time example 9.12 What this chapter means for you

Chapter 9 Objectives Function testing Performance testing Acceptance testing Software reliability, availability, and maintainability Installation testing Test documentation Testing safety-critical systems

9.1 Principles of System Testing Source of Software Faults During Development

9.1 Principles of System Testing System Testing Process Function testing: does the integrated system perform as specified by the requirements? Performance testing: are the non-functional requirements met? Acceptance testing: is the system what the customer expects? Installation testing: does the system run at the customer site(s)?

9.1 Principles of System Testing System Testing Process (continued) Pictorial representation of steps in testing process

9.1 Principles of System Testing Techniques Used in System Testing Build or integration plan Regression testing - deltas, separate files and conditional compilation Configuration management versions and releases production system vs. development system change control

9.1 Principles of System Testing Build or Integration Plan Define the subsystems (spins) to be tested Describe how, where, when, and by whom the tests will be conducted Test lowest levels first (spin 0) and move upward(e.g. spin 0,1,2…n)

9.1 Principles of System Testing Example of Build Plan for Telecommunication System Spin Functions Test Start Test End O Exchange 1 September 15 September 1 Area code 30 September 15 October 2 State/province/district 25 October 5 November 3 Country 10 November 20 November 4 International 1 December 15 December

9.1 Principles of System Testing Example Number of Spins for Star Network Spin 0: test the central computer’s general functions Spin 1: test the central computer’s message-translation function Spin 2: test the central computer’s message-assimilation function Spin 3: test each outlying computer in the stand alone mode Spin 4: test the outlying computer’s message-sending function Spin 5: test the central computer’s message-receiving function

9.1 Principles of System Testing Configuration Management Versions and releases Production system vs. development system Change control

9.1 Principles of System Testing Regression Testing Identifies new faults that may have been introduced as current one are being corrected Verifies a new version or release still performs the same functions in the same manner as an older version or release Three primary ways to control versions and releases: Deltas, separate files and conditional compilation

9.1 Principles of System Testing Regression Testing Steps Inserting the new code Testing functions known to be affected by the new code Testing essential function of m to verify that they still work properly Continuing function testing m + 1

9. 1 Principles of System Testing Sidebar 9 9.1 Principles of System Testing Sidebar 9.1 The Consequences of Not Doing Regression Testing A fault in software upgrade to the DMS-100 telecom switch 167,000 customers improperly billed $667,000 Local calls billed as long distance due to wrong area code being passed to billing interface

9.1 Principles of System Testing Test Team Professional testers: organize and run the tests Analysts: who were involved in requirements definition System designers: understand the proposed solution Configuration management specialists: to help control changes Users: to evaluate appropriateness of audience, ease of use and other human factors

9.2 Function Testing Purpose and Roles Compares the system’s actual performance with its requirements Develops test cases based on the requirements document

9.3 Performance Tests Types of Performance Tests Stress tests Volume tests Configuration tests Compatibility tests Regression tests Security tests Timing tests Environmental tests Quality tests Recovery tests Maintenance tests Documentation tests Human factors (usability) tests

Load, Stress, Performance Tests A load test is usually conducted to understand the behavior of the system under a specific expected load concurrent number of users on the application performing a specific number of transactions within the set duration Stress testing is normally used to understand the upper limits of capacity within the system to determine the system's robustness in terms of extreme load

9.4 Reliability, Availability, and Maintainability Definition Software reliability: operating without failure under given condition for a given time interval Software availability: operating successfully according to specification at a given point in time Software maintainability: for a given condition of use, a maintenance activity can be carried out within stated time interval, procedures and resources

9.4 Reliability, Availability, and Maintainability Different Level of Failure Severity Catastrophic: causes death or system loss Critical: causes severe injury or major system damage Marginal: causes minor injury or minor system damage Minor: causes no injury or system damage

Mean time to failure (MTTF) Mean time to repair (MTTR) 9.4 Reliability, Availability, and Maintainability Measuring Reliability, Availability, and Maintainability Mean time to failure (MTTF) Mean time to repair (MTTR) Mean time between failures (MTBF) MTBF = MTTF + MTTR Reliability R = MTTF/(1+MTTF) Availability A = MTBF (1+MTBF) Maintainability M = 1/(1+MTTR)

9.5 Acceptance Tests Purpose and Roles Enable the customers and users to determine if the built system meets their needs and expectations Written, conducted and evaluated by the customers

9.6 Installation Testing If acceptance test conducted at the customer site, installation testing my not be needed Before the testing Configure the system to the user environment Attach proper number and kind of devices Establish communication with other systems Allocate files and assign access The testing: work with customer to determine what tests to be conducted Regression tests: to verify that the system has been installed properly and works

9.8 Test Documentation Test plan: describes system and plan for exercising all functions and characteristics Test specification and evaluation: details each test and defines criteria for evaluating each feature Test description: test data and procedures for each test Test analysis report: results of each test

9.8 Test Documentation Documents Produced During Testing

9.8 Test Documentation Parts of a Test Plan

9.8 Test Documentation Test Description Example INPUT DATA: Input data are to be provided by the LIST program. The program generates randomly a list of N words of alphanumeric characters; each word is of length M. The program is invoked by calling RUN LIST(N,M) in your test driver. The output is placed in global data area LISTBUF. The test datasets to be used for this test are as follows: Case 1: Use LIST with N=5, M=5 Case 2: Use LIST with N=10, M=5 Case 3: Use LIST with N=15, M=5 Case 4: Use LIST with N=50, M=10 Case 5: Use LIST with N=100, M=10 Case 6: Use LIST with N=150, M=10 INPUT COMMANDS: The SORT routine is invoked by using the command RUN SORT (INBUF,OUTBUF) or RUN SORT (INBUF) OUTPUT DATA: If two parameters are used, the sorted list is placed in OUTBUF. Otherwise, it is placed in INBUF. SYSTEM MESSAGES: During the sorting process, the following message is displayed: “Sorting ... please wait ...” Upon completion, SORT displays the following message on the screen: “Sorting completed” To halt or terminate the test before the completion message is displayed, press CONTROL-C on the keyboard.

9.10 Information Systems Example Things to Consider in Selecting a Test Tool Capability Reliability Capacity Learnability Operability Performance Compatibility Nonintrusiveness

9.12 What This Chapter Means for You Should anticipate testing from the very beginning of the system life cycle Should think about system functions during requirement analysis Should use fault-tree analysis, failure modes and effect analysis during design Should build safety case during design and code reviews Should consider all possible test cases during testing