SYSTEM TESTING

SYSTEM TESTING The testing conducted on the complete integrated products and solutions to evaluate system compliance with specified requirements on functional and non functional aspects is called system testing. A system is defined as a set of hardware, software and other parts that together provide product features and solutions. System testing is the only phase of testing which tests the both functional and non functional aspects of the product.

Functional side- Testing focuses on real time customer usage. Non functional side- Focuses on quality factors 1. Performance / Load testing : To evaluate the time taken or response time of the system to perform its required functions in comparison with different versions of same products (s) or a different component (s) is called performance testing,

Scalability testing: A testing that requires enormous amount of resource to find out the maximum capability of the system parameter is called scalability testing.

Reliability testing: To evaluate the ability of the system or any independent component of the system to perform its required functions repeatedly for a specified period of time is called reliability testing.

Stress testing: Evaluating a system beyond the limits of the specified requirements or system resources ( disk space, memory, processor utilization ) to ensure that the system does not break down unexpectedly is called stress testing

Interoperability testing: this testing is done to ensure that two or more products can exchange information, use the information and work closely. Localization testing: Testing conducted to verify that the localized products works in different languages is called localization testing.

System testing is performed on the basis of written test cases according to information collected from detailed architecture/design documents, module specifications and system requirements specifications. System test cases can also be developed bases on user stories, customer discussions, and points made by observing typical customer usage.

System testing may not include many negative scenario verification such as testing for incorrect and negative values. Since such negative testing already performed by component and integration testing. System testing started once unit, component and integration testing are completed.

Why system testing? A independent team normally does system testing. This independent team is different from the team that does the component and integration testing. The behavior of the complete product is verified during system testing.

System testing is done for the following reasons Provide independent perspective in testing Bring in customer perspective in testing Provide a “ fresh pair of eyes” to discover defects not found earlier by testing. Test product behavior in a holistic, complete and realistic environment. Test both functional and non functional aspects of the product. Build confidence in the product. Analyze and reduce the risk of releasing the product. Ensure all requirements are met and ready the product for acceptance testing.

Apart from verifying the pass or fail status, non functional tests results are also determined by the amount of effort involved in executing them and any problems faced during execution. Ex: Test met pass/fail after 10 th iterations, the experience is bad and the result cannot be taken as pass.

Types of system tests Types are as follows: • Functional testing • Performance testing • Stress testing • Configuration testing • Security testing Recovery testing Reliability testing Usability testing

Not all software systems need to undergo all the types of system testing. Test planners need to decide on the type of tests applicable to a particular software system. Decisions depend on the characteristics of the system and the available test resources. For example, if multiple device configurations are not a requirement for your system, then the need for configuration test is not significant.

During system test the testers can repeat these tests and design additional tests for the system as a whole. The repeated tests can in some cases be considered regression test. Properly planned and executed system tests are excellent preparation for acceptance test.

An important tool for implementing system tests is a load generator. A load generator is essential for testing quality requirements such as performance and stress. A load is a series of inputs that simulates a group of transactions.

A Transaction consists of a set of operations that may be performed by a Person, software system, or a device that is outside the system. A use case can be used to describe a transaction. EX- system testing of telecommunication System needs a load that simulated a series of phone calls (transactions) of particular types and lengths arriving from different locations. A load can be a real load, that is, we can put the system under test to real usage by having actual telephone users connected to it. Loads can also be produced by tools called load generators

1. Functional vs non functional testing Functional – testing a product’ functionality and features. Non functional – Testing the product’s quality factors. Functional testing – Checking whether it met or not. Testing result normally depends on the product, not on the environment. It requires in-depth customer and product knowledge .

Non functional testing: Checking the quality factors Non functional testing: Checking the quality factors. Testing requires the expected results to be documented in qualitative and quantifiable terms. It requires large amount of resources and results are different for different configurations and resources. It is very complex since it needs large amount of data.

P e r f o r m a n c e T e s t i n g The goal of system performance tests is to see if the software meets the performance requirements. Testers also learn from performance test whether there are any hardware or software factors that impact on the system’s performance. Performance testing allows testers to tune the system; that is, to optimize the allocation of system resources.

Performance objectives must be articulated clearly by the users/clients. In the requirements documents, and be stated clearly in the system test Plan. The objectives must be quantified. For example, a requirement that the system return a response to a query in “a reasonable amount of time” Is not an acceptable requirement; the time requirement must be specified In quantitative way.

Results of performance tests are quantifiable. At the end of the tests the tester will know, for example, the number of CPU cycles used, the actual response time in seconds (minutes, etc.), the actual number of transactions processed per time period. These can be evaluated with respect to requirements objectives.

Resources for performance testing must be allocated in the system test plan. A source of transactions to drive the experiments. For example if you were performance testing an operating system you need a stream of data that represents typical user interactions. Typically the source of transaction for many systems is a load An experimental test bed that includes hardware and software the system-under-test interacts with. The test bed requirements sometimes include special laboratory equipment and space that must be reserved for the tests.

Instrumentation or probes that help to collect the performance data Instrumentation or probes that help to collect the performance data. Probes may be hardware or software in nature. A set of tools to collect, store, process, and interpret the data. Very often, large volumes of data are collected, and without tools the testers may have difficulty in processing and analyzing the data in order to evaluate true performance levels.

Testing the app with maximum number of user and input is defined as load testing. While testing the app with more than maximum number of user and input is defined as stress testing. In load testing we measure the system performance based on a volume of users. While in stress testing we measure the breakpoint of a system. Example: If an app is build for 500 users, then for load testing we check up to 500 users and for stress testing we check greater than 500. A banking application can take a maximum user load of 20000 concurrent users. Increase the load to 21000 and do some transaction like deposit or withdraw. As soon as you did the transaction, banking application server database will sync with ATM database server. Now check with the user load of 21000 does this sync happened successfully. Now repeat the same test with 22000 thousand concurrent users and so on.

Factors governing performance testing

Sources for deriving performance requirements

Types of requirements – Performance testing Generic requirements : common across all products in the product domain area. All products in that area are expected to meet those performance expectations. EX: time taken to load a page, initial response when a mouse is clicked, time taken to navigate between screens. Specific Requirements : It depends on implementation of a particular product and differ form one product to another in a given domain. Ex: Time taken to withdraw amount from ATM

WRITING TEST CASES

Automating performance test cases

Executing performance test cases

Analyzing the performance of test results

Configuration Testing Configuration testing is the process of testing a system with each of the supported software and hardware configurations. During this testing Tester will test whether the s/w build is supporting different hardware technologies or not Ex: printer, scanners and topologies etc., Testing the software against the hardware or testing the software against the software is called configuration testing 

Configuration testing is the process of testing a system Under development on machines which have various Combinations of hardware and software. In many situations the number of possible configurations is Far too large to test. For example, suppose you are a Member of a test team which working on some desktop user Application. The number of combinations of operating system Versions, memory sizes, hard drive types, cpu’s alone could Be enormous. If you target only 10 different operating System versions, 8 different memory sizes, 6 different hard Drives, and 7 different cpu’s, there are already 10 * 8 * 6 * 7 = 3,360 different hardware configurations.

Security testing Security testing is a type of software testing that intends to uncover vulnerabilities of the system and determine that its data and resources are protected from possible intruders.

Websites are not meant only for publicity or marketing but these have been evolved into the stronger tools to cater complete business needs. Web based payroll systems, shopping malls, banking, stock trade application are not only being used by organizations but are also being sold as products today. This means that online applications have gained the trust of customers and users regarding their vital feature named as security. No doubt, the security factor is of primary value for desktop applications too. However, when we talk about web, importance of security increases exponentially. If an online system cannot protect the transaction data, no one will ever think of using it.

Examples of security flaws in an application: 1) A Student Management System is insecure if ‘Admission’ branch can edit the data of ‘Exam’ branch 2) An online Shopping Mall has no security if customer’s Credit Card Detail is not encrypted 4) A custom software possess inadequate security if an SQL query retrieves actual passwords of its users

Typical security requirements may include specific elements of confidentiality, integrity, authentication, availability, and authorization. Actual security requirements tested depend on the security requirements implemented by the system. Security testing as a term has a number of different meanings and can be completed in a number of different ways.

Confidentiality A security measure which protects against the disclosure of information to parties other than the intended recipient. Integrity A measure intended to allow the receiver to determine that the information provided by a system is correct. Authentication is any process by which a system verifies the identity of a User who wishes to access it.

Authorization The process of determining that a requester is allowed to receive a service or perform an operation. Availability Assuring information and communications services will be ready for use when expected. Information must be kept available to authorized persons when they need it.

RECOVERY TESTING In software testing, recovery testing is the activity of testing how well an application is able to recover from crashes, hardware failures and other similar problems. Recovery testing is the forced failure of the software in a variety of ways to verify that recovery is properly performed. Recovery testing should not be confused with reliability testing, which tries to discover the specific point at which failure occurs. Recovery testing is basically done in order to check how fast and better the application can recover against any type of crash or hardware failure etc. Type or extent of recovery is specified in the requirement specifications. It is basically testing how well a system recovers from crashes, hardware failures, or other catastrophic problems

Examples of recovery testing While an application is running, suddenly restart the computer, and afterwards check the validness of the application's data integrity. While an application is receiving data from a network, unplug the connecting cable. After some time, plug the cable back in and analyze the application's ability to continue receiving data from the point at which the network connection disappeared. Restart the system while a browser has a definite number of sessions. Afterwards, check that the browser is able to recover all of them.