Teaching slides Chapter 9

Chapter 9: software testing (verification & validation) Contents Software engineering methodology considerations Software testing, verification and validation Unit testing Integration testing System testing User acceptance testing Production environment testing Software maintenance testing Test lifecycle

Chapter 9: software testing (verification & validation) Software testing, verification and validation Software testing on Waterfall used to be done after software construction. At this level system and user acceptance testing can be done. In contrast on Agile projects, software testing is done at many levels. When developers write each new class, they test the business logic contained in the class. This type of testing is known as unit testing. Each class is then integrated with main build of the software product. Integration testing is done to ensure that integration of the class with the main build happens without any problems. When the software product features are implemented in a Sprint then system testing is performed to ensure that the developed software product features work as per user stories. The customer also performs user acceptance testing to see if developed software product features are working fine before the end of the Sprint.

Chapter 9: software testing (verification & validation) Software testing, verification and validation If a defect found at any stage during software development is removed immediately then it will not propagate to the next stage. So if a defect is found in the user story itself then removing it there will result in no defect entering into software design. This fact has resulted in testing all artifacts during the entire software development process. Testing a piece of source code is done by executing and running this source code. But artifacts like user stories, software designs etc. can not be executed and run. They are tested by verifying the artifacts by going through them or having formal reviews. So we can have user story reviews, software design reviews, code reviews etc. All these types of testing grouped together are known as verification. When a piece of source code or machine code is executed and run to find out defects so that they can be removed is known as validation. Unit testing, integration testing, system testing and user acceptance testing come under validation.

Chapter 9: software testing (verification & validation) Software testing, verification and validation

Chapter 9: software testing (verification & validation) Unit testing A business logic is designed and implemented by creating a class and code is written in the class. This class is tested to know if the business logic is implemented correctly. This kind of testing is known as unit testing. Generally a user provides some input to the class and the class will do some computation and come up with some output. When testing is done, based on a user input, some expected output value will be computed. If the actual value is different from this expected value then it means business logic is not implemented correctly. In this case, the test case which was run will fail. Test cases which result in test failures are known as negative test cases. On the other hand, if actual and expected values match then it means, the business logic is implemented correctly. In this case, the test case which was run will pass. Test cases which result in test failures are known as positive test cases. Both positive and negative testing must be performed to ensure that a business logic is implemented correctly.

Chapter 9: software testing (verification & validation) Unit testing Let us take an example to understand unit testing better. Suppose there is a warehouse for a retailer. In the warehouse goods keep coming and are stored in an organized way. When sales orders arrive then these goods are picked and shipped. Suppose we need to develop the picking business logic. The total weight to be picked should not exceed 1000 kg as the picking truck has only this much capacity. Please note that items are stored in this warehouse in terms of grams and thus we will also need to convert item weights into kgs. Public class pick_weight { Public Boolean pick_logic(integer weight1, integer weight2, integer weight3){ boolean pick; Integer item1_wt = weight1/1000; Integer item2_wt = weight2/1000; Integer_item3_wt = weight3/1000; If item1_wt + item2_wt + item3_wt <= 1000 then pick = true; Return pick; }

Chapter 9: software testing (verification & validation) Unit testing To do unit test of this business logic we can create a test class. Public class test { Public void main () { Pick_weight logic_test = new pick_weight(); Boolean pick_test = logic_test.pick_logic(250000, 430000, 170000); Print (“The weight can be picked = “ + pick_test); } When you run the test, it will print a message if the picked weight is correct or not (true or false). In our case, the weight can be picked as total weight is coming to 850 kgs which is less than maximum weight which can be picked (1000 kgs).

Chapter 9: software testing (verification & validation) Unit testing In many situations, we need to test a business logic where there could be a large number of possible entries. For example, suppose there is a web page to create shipping address of a user. The user may need to provide user information like name, address, email and phone number to register. For each of the fields on the web page there can be many possible entries. For the phone number field alone, here are the possibilities: Using only alphabetic values Using a mix of alphabetic, special characters and numeric values Using only numeric values with length less than 8 digits Using only numeric values with length between 8 to 10 digits Using only numeric values with length more than 10 digits Leaving the field blank Out of all these possibilities, only a few of them will be a valid entry. So validation of user input is important here. Otherwise user can enter values in the fields which have no meaning. A phone number is generally 8 to 10 digits long. So any entry which has 8 to 10 digits (numerical input) will be valid. All other entries will be invalid. Since we have categorized the possible input range, we can do the testing with less number of test cases. Just a few test cases from each range will be enough to test the phone number field. This kind of partitioning range of possible inputs is known as equivalence partitioning.

Chapter 9: software testing (verification & validation) Unit testing In the previous slide we found possible input ranges for one field out of many fields on a web page. In the phone number field, the possible entries for 8 to 10 digit range could be 8 digits, 9 digits and 10 digits. Generally when a range of values could be possible entries then the possibility of an invalid entry lies at the boundaries. In case of the phone number, an entry with 8 digits or 10 digits could be an invalid entry. So these boundaries should be tested. This kind analysis for testing is known as boundary value analysis.

Chapter 9: software testing (verification & validation) Unit testing In the previous slide we found possible input ranges for just one field out of many fields on a web page. Suppose validation rules have been implemented to ensure that any user should not be able to save meaningless information on this web page. Even when a valid entry is made for a field; an invalid entry in another field will make the entry for the web page invalid (the validation rules will ensure no invalid is saved). We can observe that for each valid entry at the web page level; there will be a large number of invalid entries. This means a lot of testing will be required for this web page. When we need to test this web page then we need to figure out how much testing will be good enough to minimize amount of testing. We can create a decision table to test such a class. Amount of testing required in such cases is 2n . Since we have 4 decision points (name, address, email and phone number), there will be 64 combinations of testing to be done for each entry made in these fields. Out of 64 tests, only one value be valid and 63 invalid. A decision table as depicted in next slide is used to test all possible scenarios.

Chapter 9: software testing (verification & validation) Unit testing

Chapter 9: software testing (verification & validation) Unit testing Many classes may be involved in doing permanent data manipulation as part of business logic implementation. Permanent data reside in databases and a software product needs to be connected with the database to manipulate permanent data. How unit testing can be done in such cases? There are some challenges when unit testing needs to be done when database connection is involved. Database is a separate entity and connected to the business logic through a database connection. This connection may or may not be available at a given time. If connection is not available then database testing can’t be done. Data created or changed in the database is permanent. If some data is changed in the database then the old data is lost. Any test case trying to access overwritten data will result in a failure. During testing, testers create lots of test data. This data may corrupt the production data. Due to these challenges, some tools have been developed which simulate a database so that unit testing can be performed. But drawback is that actual database connection can not be tested in such cases. One alternative is that a test database can be prepared for testing purposes.

Chapter 9: software testing (verification & validation) Integration testing A software product may be developed as separate parts and later they are integrated. To ensure that the parts integrate well, integration testing is performed. The open interfaces through which integration may happen is tested. On Agile projects, continuous integration is done. After a new class is developed, it is integrated with the main software build. This is better approach. Since only one class is integrated at a time with the main software build, finding out integration problem if it happens is easier as number of integration points in such cases is not much. When a developer creates a new class, he/she integrates it with the main build. The next 2 slides depict the scenario of integration at sub-system level and at class level.

Chapter 9: software testing (verification & validation) Integration testing

Chapter 9: software testing (verification & validation) Integration testing

Chapter 9: software testing (verification & validation) Integration testing Let us take an example to understand integration testing better. Suppose we have a class which computes grades of students based on marks in their examination. Public class grade_marks { Public String compute (integer physics, integer chemistry, integer math) { Character grade; If (physics + chemistry + math)% =>75 Grade = “A”; Else if (physics + chemistry + math)% =>50 Grade = “B”; Else grade = “C”; Return grade; } Suppose this class “grade_marks” is submitted in the software build. Then another class was created to assign and print grades to students. Public class assign { Public void grade () { Grade_marks grading = new grade_marks; Character grade = grading.compute(80, 45, “B”); Print grade;

Chapter 9: software testing (verification & validation) System testing After a software product is developed incrementally or in a big bang, it needs to be tested again. This time however, the source code is not needed. This type of testing is done with the machine code. This type of testing is known as system testing. System testing is needed because: Business logic functionality must be tested at system level to ensure that it works. Even though business logic testing is already done at unit level; it has not been tested in the complete system context when user interface, business logic and database have been put together. Due to changes or new development, some existing functionality may have changed and thus testing is done to ensure that software product features still work fine. Non functional testing like performance or security can not be done at a prior level (unit or integration). They must be performed at system level. Boundaries of unit, integration and system testing is depicted in next slide.

Chapter 9: software testing (verification & validation) System testing

Chapter 9: software testing (verification & validation) System testing Under system testing, many types of testing are performed. Again these testing can be broadly categorized as regression testing, functional testing and non functional testing. Regression testing is done in Agile projects. Since software is developed incrementally; software may need to be redesigned several times using refactoring. After refactoring some existing functionality of the software product may get changed due to changes made in the source code. Regression testing is done for all software product features which were developed prior to refactoring. Functional testing is done to find out if developed software product features work as per user stories. Functional testing can be done either using formal testing or exploratory testing. In formal testing, test cases are prepared before testing is done. In exploratory testing, no test cases are prepared and testing is done just observing the user stories. Exploratory testing is fast as no test case writing is involved. Localization testing is also done when a software product is introduced in a new market where language or business practices are different. Non functional testing involves testing for factors which influence the environment under which a software product is run. These factors include performance (speed of computations), security, usability etc.

Chapter 9: software testing (verification & validation) System testing

Chapter 9: software testing (verification & validation) User acceptance testing User acceptance testing is done by users of a software product once it is deployed at customer site. Even after all the testing done at unit, integration and system level, some defects might escape to production environment. So users test the software product to ensure that it has no critical defects. Once user acceptance testing is found to be satisfactory, user may start using the system. User acceptance testing is similar to system testing except this time it is being done by users and not the project team.

Chapter 9: software testing (verification & validation) Production environment testing Once a software product is deployed, users start using it. An instance of a deployed software product is known as a production instance. During its use, users may discover some defects or problems due to which they may not be able to use it. These problems are reported to the support team. The support team then can fix those problems. Mission critical software products are important for doing business by a company. If the software product is down for any reason then business will get impacted. To ensure that a production instance of a software product is always available, the support team conducts some tests regularly. Using those tests, they ensure that critical functions are working properly and transactions can be performed. This kind of testing is known as sanity testing. If any problems or defects are found after conducting sanity tests, they are fixed immediately by the support team to ensure that the production instance can be used by users without any problems. Software testing is also required after some maintenance project is taken to fix software defects. Software testing after a maintenance project is known as maintenance testing.

Chapter 9: software testing (verification & validation) Software testing lifecycle When software testing is done, it follows a lifecycle. First of all, test cases are written. These test cases are executed to find defects. If the test case passes (the actual result is same as the expected result) then it means there is no defect in the software product for that test case. So next test case will be taken for execution. If no defects are found then testing moves to the next test case and so on. During testing if a test case fails (the actual result is not the same as the expected result) then it means a defect has been found. This defect is reported so that it can be fixed. A software developer may fix it and report it back for fix verification. The software tester who had reported this defect will verify if the defect has indeed been fixed. If it is fixed then the defect will be closed. If not then the defect will be fixed. This software testing lifecycle is same for almost all type of tests. The exception is exploratory testing. In exploratory testing, test cases are not written and testing is done referring to user stories. If a test fails only then a test case is written for the sole purpose of defect reporting and fixing.

Chapter 9: software testing (verification & validation) Software testing lifecycle

Chapter 9: software testing (verification & validation) Software testing lifecycle for exploratory testing