Cairo University Faculty of Computers and Information CS251 – Software Engineering Lecture 20: SW Testing Presented by: Mohammad El-Ramly, PhD

2 Outline Definition of Software Testing Testing as a Quality Assurance Technique Testing Terminology Levels and Types of Testing

3

4 What is a computer program?

5 What is a house?

6 Any better definition? The focus is on Stakeholders – (for people) Purpose – (to live in) Stakeholder: Any person affected by: s uccess or failure of a project, or actions or inactions of a product, or e ffects of a service.

7 Apply this definition to prog.

8 A computer program is a communication among several humans and computers who are distributed over space and time, that contains instructions that can be executed by a computer.

9 Software Quality

10 Software Error (Bug)

11 Software Error (Bug) An attribute of a software product that reduces its value to a favored stakeholder or increases its value to a disfavored stakeholder without a sufficiently large countervailing benefit. An error may or may not be a coding error, or a functional error. Design errors are bugs too.

12 Software Testing is an empirical technical investigation conducted to provide stakeholders with information about the quality of the product or service under test

13 Software Testing

14 Software Testing is Search for Information 1.Find important bugs 2.Assess the quality of the product 3.Help managers assess the progress of the project 4.Help managers make release decisions 5.Block premature product releases 6.Help predict and control product support costs 7.Check interoperability with other products

15 Software Testing is Search for Information 8.Assess conformance to specifications 9.Certify the product meets a particular standard 10.Ensure the testing process meets accountability standards 11.Minimize the risk of safety-related lawsuits 12.Help clients improve their processes 13.Evaluate the product for a third party

16 Fact of testing Testing does not guarantee the absence of defects it gurantees their presence

17

18 Assume we want to test Font Increase functionality in Word Calibri Font size from 7 to 13.5

19 Assume we want to test Font Increase functionality in Word Arial Font size from 7 to 13.5

20 Assume we want to test Font Increase functionality in Word Times New Roman Font size from 7 to 13.5

21 Assume we want to test Font Increase functionality in Word Courier New Font size from 7 to 13.5

22

23 So, what should we test?

24 So, what should we test? What to cover? Every font size (to a tenth of a point). Every character in every font. Every method of changing font size. Every user interface element associated with font size functions. Interactions between font sizes and other contents of a document. Interactions between font sizes and every other feature in Word. Interactions between font sizes and graphics cards & modes. Print vs. screen display.

A program can fail in many ways System under test Program state Intended inputs System state Configuration and system resources From other cooperating processes, clients or servers Monitored outputs Program state, including uninspected outputs System state Impacts on connected devices / system resources To other cooperating processes, clients or servers

Unit testing with JUnit26

27

28 Why? Why testing? Improve software quality Increase customer satisfaction and reduce cost Finish faster and on budget Catch errors and bugs In short, to Produce Better Code

29 What should be tested ? Test for boundary conditions Test for both success and failure Test for general functionality Test for configuration and resources Test for interoperability Etc..

30 When to start testing Software quality and is a life-cycle process Testing can start as early as the project starts

Software Quality Landscape Planning Requirements Design Development Testing Guidelines, Standards and Templates QA Planning and Budgeting Testing Debugging Reviews Refactoring Requirements V & V Design Review Collaborative Development / Reviews Metrics Change and Configuration Management Bug Tracking Surveys Deployment & Maintenance M a n a g e m e n t Contract Review

Software Quality Landscape Planning Requirements Design Development Testing Guidelines, Standards and Templates QA Planning and Budgeting Testing Debugging Reviews Refactoring Requirements V & V Design Review Collaborative Development / Reviews Metrics Change and Configuration Management Bug Tracking Surveys Deployment & Maintenance M a n a g e m e n t Contract Review

Software Quality Landscape Planning Requirements Design Development Testing Guidelines, Standards and Templates QA Planning and Budgeting Testing Debugging Reviews Refactoring Requirements V & V Design Review Collaborative Development / Reviews Metrics Change and Configuration Management Bug Tracking Surveys Deployment & Maintenance M a n a g e m e n t Contract Review

1/9/2016 CPSC , CPSC , Lecture Testing Organization (V-Model) ProduceValidate Requirements specification System Specification System Design Module Design Coding Unit Testing Integration Testing System Testing Acceptance Testing User Specification System Specification Software Specification Module Specification ProduceValidate ProduceValidate Produce Validate

Testing Terminology

What is test case 36 A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly 1

37 Good test case design An good test case satisfies the following criteria: Reasonable probability of catching an error Does interesting things Doesn’t do unnecessary things Neither too simple nor too complex Not redundant with other tests Makes failures obvious Mutually Exclusive, Collectively Exhaustive

Unit testing with JUnit38 Test case design technique Test case design techniques can be broadly split into two main categories Black box (functional) White box (structural) 2 3

39 Black Box tests Targeted at the apparent simplicity of the software Makes no assumptions about implementation Good for testing component interactions Tests the interfaces and behavior InputOutput 2

40 White Box tests Targeted at the underlying complexity of the software Intimate knowledge of implementation Good for testing individual functions Tests the implementation and design InputOutput 3

Black box testing: Test against the specifications of the component Tests the interface of the component. White box testing: Test design and implementation of the component. Test cases exercise specific sets of condition, loops, etc. Black box vs. White box

CPSC , CPSC , Lecture 11 Test to specifications: Also known as black-box, data-driven, functional, or input/output driven testing. Ignore the code — use the specifications to select test cases. Test to code: Also known as glass-box, logic-driven, structured, or path- oriented testing. Ignore the specifications — use the code to select test cases. Neither exhaustive testing to specifications nor exhaustive testing to code is feasible. Another way to classify test cases

43

Unit testing with JUnit44 Test case writing example Suppose we have two parameters we want to cover in a set of tests. Parameters are as follows.. Operating system Win98 Win2k Winxp Printers HP 4100 HP 4200 How We should write test case for this ??

Unit testing with JUnit45 Types (Levels) of Tests Unit Testing Individual classes or types Component Testing Group of related classes or types Integration Testing Interaction between classes 4 5 6

46 Types (Levels) of Tests System Testing Testing to confirm that all code modules work as specified, and that the system as a whole performs adequately on the platform on which it will be deployed. Acceptance Testing Final testing usually involving users to ensure their requirements are met. 7 8

Testing that Involves Users Alpha testing: In-house testing. By a test team or end users. Beta testing: By users or selected subset of actual customers in a normal work environment. Product is very close to completion. Open beta release: Let public carry out the beta testing. Acceptance testing: By users to check that system satisfies requirements to decide whether to accept the system based on the test result. 9 10

48

49 What is a testing framework? A test framework provides reusable test functionality which: Is easier to use (e.g. don’t have to write the same code for each class) Is standardized and reusable Provides a base for regression tests 11

Regression testing New code and changes to old code can affect the rest of the code base ‘Affect’ sometimes means ‘break’ We need to run tests on the old code, to verify it works – these are regression tests Regression testing is required for a stable, maintainable code base 12

51 Why use a testing framework? Each class must be tested when it is developed Each class needs a regression test Regression tests need to have standard interfaces Thus, we can build the regression test when building the class and have a better, more stable product for less work

52 Example: Old way vs. new way int max(int a, int b) { if (a > b) { return a; } else { return b; } } void testMax() { int x = max(3, 7); if (x != 7) { System.out.println("max(3, 7) gives " + x); } x = max(3, -7); if (x != 3) { System.out.println("max(3, -7) gives " + x); } } public static void main(String[] args) { new MyClass().testMax(); void testMax() { assertEquals(7, max(3, 7)); assertEquals(3, max(3, -7)); }

53 JUnit JUnit is a framework for writing unit tests A unit test is a test of a single class A test case is a single test of a single method A test suite is a collection of test cases Unit testing is particularly important when software requirements change frequently Code often has to be refactored to incorporate the changes Unit testing helps ensure that the refactored code continues to work 13 14

54 JUnit.. JUnit helps the programmer: Define and execute tests and test suites Formalize requirements and clarify architecture Write and debug code Integrate code and always be ready to release a working version

55 Test suites Obviously you have to test your code to get it working in the first place You can do ad hoc testing (testing when something happens), or You can build a test suite (set of tests that can be run at any time) Disadvantages of writing a test suite It’s a lot of extra programming True—but use of a good test framework can help quite a bit You don’t have time to do all that extra work False—Experiments repeatedly show that test suites reduce debugging time more than the amount spent building the test suite Advantages of having a test suite Your program will have many fewer bugs It will be a lot easier to maintain and modify your program