Software Testing Written by Zvika Gutterman Adam Carmi

Question? Who has the highest Salary in Israel public service in 2001? The test pilot of IAI ! Why?

Agenda Goal Introduction Vocabulary White Box Black Box Static vs. Dynamic Example - TVRS

Goal The process of executing a program (or part of it) with the intention of finding errors. [Myers] Any activity aimed at evaluating attribute of a program and determining that it meets its requires results. [Hetzel] –Myers: “The art of software testing” [1979] –Hetzel: “The Complete Guide to software testing” [1988]

Introduction 50-90% of software life cycle TVRS vs. Air Craft Industry Usually not done by professionals Not an exact science (can be impossible) Complete coverage is impossible ! –Number of options is exponential – just RAM O(2 ^N ) Fixes introduce new bugs

Introduction (cont.) OS & Hardware dependent Programming Language Dependent! –C++ vs. Java ! Never Ends (client as a tester..) –There is always another Bug Marketing vs. Testing –1980’s – PC –1990’s – Internet hype

Introduction (cont.) Requires writing additional code (which can include bugs) Automatic tools for testing exists –Mainly for knows fields Web servers (see ….) –Parasoft –Mercury Interactive

Vocabulary Bug, Fault, Error, Defect, Errata,.., Feature Fix, Patch, Service Pack, Update, Upgrade Versions: Alpha, Beta, Release Candidate, 1.1.x vs. 1.2.x Regression/Functional/White Box/Black Box/Unit/System/Integration Testing

White Box – Unit Testing Testing the Implementation Logic Usually done by the programmer – (good/bad?) Tests the source code Example: –Implementing a Sorting Method Merge Sort vs. Bubble sort

Black Box – Functional Testing Use Cases and Requirements based Interface Testing Input-Output Testing Use Valid and Invalid inputs Use “real” samples

Black Box – Sorting Method Unaware of the used algorithm Sorting: –Use complete enumeration for small examples 10 elements (10!) Should we test again for 11 elements? –Check for equal values (2,2,2,5,3,4,7) –Which input ranges are valid? (use extremes) Random Testing (why?)

Testing affects results Test Programs Another Software is using the same resources It is not the same code –Debug vs. Release For example: counters initialized to zero (gdb) assert(x=3);

Acceptance Testing testing used to simulate client activities within the software house used on a small group of clients –Advanced users? –Brave clients..

Static Testing Static –Compiler C++: -Wall, -pedantic, -ansi –Static Checker – e.g. – Lint ( –Human code review –Formal verification

Dynamic Testing Runtime Monitors –Memory management Debugger Unique Tools – Purify/Bounds Checker –Timing Quantify Microsoft –Coverage Pure coverage

Testing Metrics Number of bugs –Day/developer/team/product Define severity levels –No Go! –.. –Feature Regression Tools (re-approving the product) Acceptance Testing Poor man solution: –Time / Budget

Black Box - Algorithm 1.For each Use Case a)Identify different cases (named equivalent classes) b)Identify endpoints 2.Create tests i.Prefer automatic tests ii.Iterate on all possible Environments iii.Prioritize tests i.Among Use Cases ii.Among different tests

Example – TRVS – Black Box Use case: 3 – find Policeman –Equivalent classes Policeman with searched ID exists/not Policeman with searched Name exists/not Few policemen with searched Name exists Policeman with searched ID just deleted.. –Endpoints Does system allow to enter ID’s with varying number of digits?..

Example – TRVS – Black Box Test Index1 Use Case3 - Find Policeman PreconditionUser : {“Moshe”, “Cohen”, } exists in system. Action1.Select: Find by Name 2.Insert name: “Moshe” 3.Select – find button. Expected ResultsUser found with above details. Window appears on screen. Test Result Pass/Fail Reasoning

Example – TRVS – Black Box Test Index2 Use Case3 - Find Policeman PreconditionPoliceman records file “policemen.db” loaded. Action1.Select: Find by Name 2.Insert name: “Moshe” 3.Select – find button. Expected ResultsUser not found. Window appears on screen. Test Result Pass/Fail Reasoning