Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s test plan? You have one, right? :-)

Some TLAs (Three Letter Acronyms) SQA –Software Quality Assurance many different activities (including good communication, good design, etc) V&V –Verification and Validation Part of SQA Verification: are we building the product right? Validation: are we building the right product? Fault detection –Reviews: Walkthroughs and Inspections –Testing Debugging –what you do once you’ve established that there is a bug

Types of testing Unit testing Integration testing Functional testing Acceptance testing Pilot testing Performance testing Installation testing Usability testing

Who does the testing? Developers Testers Customers

Unit tests Written by developers Tests individual components In TDD (another TLA) unit tests are written before code Automated test frameworks –JUnit –CPPUnit –etc.

Integration testing Testing of components when brought together Different strategies: –big bang: test all components together at once –bottom-up: test layer by layer, from bottom up –top-down: test layer by layer, from top (UI) down Unintegrated components are simulated by stubs during integration testing

Functional testing A whole-system test Also called “requirements testing” Uncovers differences between use-case model and observed system behavior

Acceptance testing Benchmark testing –testing against a specific set of test cases Competitor testing –evaluation wrt competitor product Shadow testing –evaluation wrt legacy system

Pilot testing Field testing –alpha test: system is tested by users in development environment –beta test: system is tested by users in the deployment environment

Performance testing Stress testing – simultaneous requests Volume – large amounts of data Security testing – is it secure? Timing tests – are timing constraints violated? Recover tests – can system recover from errors?

Installation testing Testing of system in deployment environment (after beta test)

Usability testing Is user interface acceptable to users?

Random testing Series of studies done by B. Miller and colleagues First study: Unix utilities (50-85 utilities, depending on OS version, 6 versions) Second study: Unix utilities and X- Windows applications Third study: Windows NT applications

Results How many utilities/applications crashed or hung on random input?

Results How many utilities/applications crashed or hung on random input? first study 25%-33% second study –utilities 40% –applications 25%

Window NT study Similar study for Windows NT, using software/utilities from different vendors (Microsoft, Adobe, Command Software Systems, Qualcomm, Aladdin, Ghostgum, Free Software Foundation, Sun, Netscape, Jasc, Van Dyke, MIT Kerberos Group, Nullsoft, Ipswitch, Metrowerks) 23 applications

Results How many utilities/applications crashed or hung on random input?

Results How many utilities/applications crashed or hung on random input? 21% of applications crashed (valid keyboard/mouse events) Additional 24% hung (valid keyboard/mouse events) 100% crashed or hung with random input streams

Blackbox vs. whitebox testing Blackbox testing considers only input- output behavior of a component. Blackbox testing tests functionality. Whitebox testing considers the internal structure of components. Whitebox testing tests “structural and dynamic aspects of the component” [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 448]

Equivalence testing Blackbox technique Partition inputs into equivalence classes. A representative from each equivalence class is used for each test. Assumption: all members of equivalence class will behave the same. [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 453]

Example Calendar Month equivalence classes –months with 30 days –months with 31 days –months with 28 or 29 days –Invalid inputs: month 12 Year equivalence classes –non-leap years –leap years –Invalid inputs: year < 0 If method depends on both month and year, we have 2*3=6 equivalence classes. [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 454]

Example Calendar Month equivalence classes –months with 30 days –months with 31 days –months with 28 or 29 days –Invalid inputs: month 12 Year equivalence classes –non-leap years –leap years –Invalid inputs: year < 0 If method depends on both month and year, we have 2*3=6 equivalence classes (My question: why not 4*3=12?) [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 454]

Boundary testing Test boundaries of equivalence classes In example, February yields many boundary cases. Leap years are years divisible by 4, unless divisible by 100, unless divisible by 400. –leap years divisible by 400 (e.g. Feb 2, 2000) –non-leap years divisible by 100 (e.g. Feb 2, 1900) [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 454]

Path testing Whitebox testing Ensure that all paths through the code are exercised Start with a flow graph (a graph whose nodes are executable blocks of code and whose edges represent flow of control) Test cases must cover each edge in graph Minimum number of test cases required is the number of independent paths in graph, which is # edges - # nodes + 2 Can work off of activity diagram [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 456]

State-based testing Whitebox testing Ensures that state transitions of system work as expected Works off of statechart diagram [Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 459]

Unit Testing Example In this exercise we will develop a simple class (RationalNumber) in a TDD fashion.

Properties A rational number is of the form n/d, where n is an integer, and d is an integer except zero. Rational numbers can be added, multiplied, tested for equality. Each rational number has an additive inverse (its negative) and a multiplicative inverse (its reciprocal).

Rational class Will provide public constructor: public Rational(int numerator, int denominator); Will provide public methods: public Rational add(Rational r); public Rational negative(); public Rational multiply(Rational r); public Rational reciprocal(); public boolean equals(Object obj);

Exercise structure Break into 6 groups Each group identifies tests to perform on their operation (construction, addition, negative, multiplication, reciprocal, equals) I will write the tests, then we will write the code