Requirements-Based Testing

Slides:



Advertisements
Similar presentations
Testing and Test Case Development A “primitive” method of testing, with NO test preparation, may include the following steps : – Initiate the system –
Advertisements

CMSC 345, Version 11/07 SD Vick from S. Mitchell Software Testing.
Black box testing  Black box tests focus on the input/output behavior of the component  Black-box tests do not deal with the internal aspects of the.
Unit Testing CSSE 376, Software Quality Assurance Rose-Hulman Institute of Technology March 27, 2007.
Testing an individual module
Understanding the Mainline Logical Flow Through a Program (continued)
Recall The Team Skills Analyzing the Problem
Fundamentals of Python: From First Programs Through Data Structures
Software Systems Verification and Validation Laboratory Assignment 3
System/Software Testing
Fundamentals of Python: First Programs
CMSC 345 Fall 2000 Unit Testing. The testing process.
CS4311 Spring 2011 Unit Testing Dr. Guoqiang Hu Department of Computer Science UTEP.
Overview of Software Testing 07/12/2013 WISTPC 2013 Peter Clarke.
Unit Testing -Ranjit Shewale Contents  Scope  Owner  Approach  Conventional approach  Object oriented approach  Tips for the.
Software Testing. 2 CMSC 345, Version 4/12 Topics The testing process  unit testing  integration and system testing  acceptance testing Test case planning.
Testing Testing Techniques to Design Tests. Testing:Example Problem: Find a mode and its frequency given an ordered list (array) of with one or more integer.
Test Coverage CS-300 Fall 2005 Supreeth Venkataraman.
Black-box Testing.
Software Testing Reference: Software Engineering, Ian Sommerville, 6 th edition, Chapter 20.
Testing Data Structures Tao Xie Visiting Professor, Peking University Associate Professor, North Carolina State University
1 Phase Testing. Janice Regan, For each group of units Overview of Implementation phase Create Class Skeletons Define Implementation Plan (+ determine.
October 21, 2010COMS W41561 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser
Software Testing Reference: Software Engineering, Ian Sommerville, 6 th edition, Chapter 20.
Verification vs. Validation Verification: "Are we building the product right?" The software should conform to its specification.The software should conform.
Laurea Triennale in Informatica – Corso di Ingegneria del Software I – A.A. 2006/2007 Andrea Polini XVIII. Software Testing.
Testing Data Structures
Functional testing, Equivalence class testing
Software Testing.
Software Testing.
Control Flow Testing Handouts
Handouts Software Testing and Quality Assurance Theory and Practice Chapter 4 Control Flow Testing
The Joy of Breaking Code Testing Logic and Case Selection
Black Box Testing PPT Sources: Code Complete, 2nd Ed., Steve McConnell
Software Testing Techniques
Topics Introduction to Repetition Structures
Testing UW CSE 160 Winter 2017.
Recall The Team Skills Analyzing the Problem
Specification-Based Testing and the Category-Partition Method
Outline of the Chapter Basic Idea Outline of Control Flow Testing
Chapter 13 & 14 Software Testing Strategies and Techniques
Structural testing, Path Testing
Types of Testing Visit to more Learning Resources.
Topics Introduction to Repetition Structures
Testing Approaches.
White-Box Testing.
Unit Test: Functions, Procedures, Classes, and Methods as Units
Testing UW CSE 160 Spring 2018.
Formatted and Unformatted Input/Output Functions
UNIT-4 BLACKBOX AND WHITEBOX TESTING
Software Testing (Lecture 11-a)
Lecture 09:Software Testing
Testing UW CSE 160 Winter 2016.
White-Box Testing.
Static Testing Static testing refers to testing that takes place without Execution - examining and reviewing it. Dynamic Testing Dynamic testing is what.
Software testing.
Coding Concepts (Basics)
Informatics 43 – April 28, 2016.
Chapter 10 – Software Testing
Laboratory of Advanced Research on Software Technology
Topics Introduction to Value-returning Functions: Generating Random Numbers Writing Your Own Value-Returning Functions The math Module Storing Functions.
Topics Introduction to Repetition Structures
More Loops Topics Relational Operators Logical Operators for Loops.
CSE 1020:Software Development
TYPES OF TESTING.
What is a Function? Takes one or more arguments, or perhaps none at all Performs an operation Returns one or more values, or perhaps none at all Similar.
Learning Intention I will learn about the standard algorithm for input validation.
UNIT-4 BLACKBOX AND WHITEBOX TESTING
Software Testing.
Chapter 13 & 14 Software Testing Strategies and Techniques 1 Software Engineering: A Practitioner’s Approach, 6th edition by Roger S. Pressman.
Presentation transcript:

Requirements-Based Testing

Assignments Read Ostrand and Balcer paper on the “category partition method”, available on the Course Handouts page, for Monday, October 23.

Review: Test Adequacy Criteria Adequacy criterion: A rule that lets us judge the sufficiency of a set of test data for a piece of software. Adequacy criteria lead to systematic, rigorous methodologies for testing software.

Review: Some Adequacy Criteria Requirements coverage Statement coverage Branch coverage Condition coverage Path coverage Boundary value coverage Dataflow coverage

Review: Some Adequacy Criteria Requirements coverage Statement coverage Branch coverage Condition coverage Path coverage Boundary value coverage Dataflow coverage

Requirements Coverage Definition Requirements coverage: Given system S with requirements R(S), create a test suite T such that for each requirement r in R(S), T contains at least one test case that verifies that S behaves correctly relative to r.

Simple Requirements Coverage: Requirements for SQRT Startup: User types “sqrt” to invoke the program The prompt “Enter number:” is displayed and the system waits for input If user types “sqrt [arg]” the prompt is not displayed, the system proceeds at 2. Action on input: If input >= 0 and input < MAXINT, square root of input is displayed and program terminates If input < 0 and input > MININT system outputs “number must be positive” If input is a non-number, system says “enter a non-negative integer” If input < MININT or input > MAXINT output is unpredictable

Covering Requirements: Domain Partitioning When conditions are ranges, this partitions the space into one valid and two invalid sub-ranges (e.g. between 0 and MAXINT) When conditions are boolean, this partitions the space into one vallid and one invalid class (e.g. input is a number or a non-number) When conditions are sets, this partitions the space into one valid (in the set) and one invalid (not in the set), provided you expect each set member to be handled the same.

Covering Requirements: Boundary-Value Testing After partitioning the input domain into classes, test the program using input values not only “inside” the classes, but also at their boundaries. E.g., for sqrt, where requirement 2.b states “input >= 0 and input < MAXINT”, what boundaries could be considered?

Covering Requirements: Loop Testing Also consider requirements related to “loops” or iteration: addressing iteration implies considering “boundaries” related to numbers of repetitions. E.g., given the requirement: “for each record on the CallStack, free any registers associated with that record”, what boundaries could be considered, at least theoretically?

Requirements Coverage Process System Requirements Testing Requirements Test Cases

Testing Requirements Testing requirements are a list of program behaviors that we require be tested, derived from a program specification They make the different conditions that test cases must exercise explicit

Simple Requirements Coverage: Requirements for SQRT Startup: User types “sqrt” to invoke the program The prompt “Enter number:” is displayed and the system waits for input If user types “sqrt [arg]” the prompt is not displayed, the system proceeds at 2. Action on input: If input >= 0 and input < MAXINT, square root of input is displayed and program terminates If input < 0 and input > MININT system outputs “number must be positive” If input is a non-number, system says “enter a non-negative integer” If input < MININT or input > MAXINT output is unpredictable

Testing Requirements for SQRT Invoke SQRT with no argument Invoke SQRT with an argument Use a non-numeric argument Use an argument < 0 and > MININT Use an argument = 0 Use an argument > 0 and < MAXINT Use an argument > MAXINT Use an argument < MININT

Testing Requirements for SQRT with Traceability Invoke SQRT with no argument 1a, 1b Invoke SQRT with an argument 1c Use a non-numeric argument 2c Use an argument < 0 and > MININT 2b Use an argument = 0 2a, 2b Use an argument > 0 and < MAXINT 2a Use an argument > MAXINT 2d Use an argument < MININT 2d

Test Case for SQRT Test 1. PURPOSE: SETUP: Test the invocation of sqrt without arguments, and the operation of sqrt on an integer in the expected range. SETUP: None needed INPUTS and EXPECTED OUTPUTS: inputs expected output ------------------------------------------- “sqrt” “Enter number:” “4” “2” followed by command-line prompt TESTING REQUIREMENTS COVERED: 1 and 4

In the Absence of Requirements? From the Aristotle user manual: To invoke the Aristotle menu interface, type “aristotle source-file”, where source-file is an optional argument that names a C-language source file you wish to work with. The Aristotle top menu is displayed. You can exit the Aristotle menu interface from any menu by typing “x” and pressing the return key.

Requirements Coverage Ad hoc Documentation System Requirements Testing Requirements Test Cases

System Requirements Derived from Prior User Documentation 1 Invoking the system 1.a “aristotle [source-file] displays top menu 1.a.i source-file must be a valid C file 1.b “aristotle <CR>” displays top menu 2 Exiting the system 2.a “x <CR>” exits system from any menu From these, you can derive testing requirements.

Testing Requirements Derived from Prior System Requirements 1 Invoking the system 1.a Type “aristotle <filename>” for existing accessible C file 1.b Type “aristotle <filename>” for nonexisting file 1.c Type “aristotle <filename>” for existing but protected C file 1.d Type “aristotle <filename>” for a fortran file named .c 1.e Type “aristotle <CR>” 2 Exiting the system 2.a From each menu, try “x CR” 2.b From each menu, try “X CR” 2.c From each menu, try “C CR” for C not an option From these, you can construct test cases.

Test Plans A test plan is a “specification” for the testing process for a given system. A test plan contains: Information on the test process being used (black box, white box, etc.) Testing requirements Test cases that meet these requirements, including their inputs, outputs, and traceability information

Test Plans Test plans add rigour to testing, supporting reuse, assessment of success, replication, and documentation -- especially when coupled with meaningful testing methodologies or test adequacy criteria But at what cost? Test plans may be partially implemented via test automation.

Requirements Coverage Ad hoc Documentation System Requirements Test Plan Testing Requirements Test Cases