Testing Principles And Concepts CBOK Knowledge Domain 5

Testing Principles And Concepts - Outline Testing Techniques Testing Methods Independent Testing

Testing Techniques Various approaches used in testing –Static or “human” (desk checking) –Dynamic (testing that executes system code)

Dynamic Testing Techniques used are determined by the type of testing that must be conducted –Functional –Structural

Functional Testing Addresses the overall behavior of the program by testing the following: –Transaction flows –Input validation –Functional completeness

Structural Testing Includes the following: –Path testing –Code coverage testing –Analysis testing –Logic testing –Nested loop testing –Performance testing –Stress testing –Others...

White Box Testing Consists of testing paths, branch by branch, to produce predictable results

White Box Testing Techniques Statement coverage Decision coverage Condition coverage Decision/Condition coverage Multiple Condition coverage

Statement Coverage Execute all statements at least once

Decision Coverage Execute each decision direction at least once

Condition Coverage Execute each decision with all possible outcomes at least once

Decision/Condition Coverage Execute all possible combinations of condition outcomes in each decision Treat all iterations as two-way conditions exercising the loop zero times and once

Multiple Condition Coverage Invokes each point of entry at least once

White Box Testing Techniques continued... Choose a combination of techniques appropriate for the application Too exhaustive a combination of these techniques leads to an unmanageable number of test cases

Black Box Testing Focuses on testing the function of the program or application against its specifications Determines whether combinations of inputs and operations produce expected results

Black Box Testing continued... When creating black box test cases, the input data used is critical Three successful techniques for managing the amount of input data required: –Equivalence Partitioning –Boundary Analysis –Error Guessing

Equivalence Partitioning A subset of data that is representative of a larger class A technique for testing equivalence classes rather than undertaking exhaustive testing of each value of the larger class

Equivalence Partitioning continued... For example, a program which edits credit limits within a given range ($10,000 - $15,000 would have 3 equivalence classes: –Less than $10,000 (invalid) –Between $10,000 and $15,000 (valid) –Greater than $15,000 (invalid)

Boundary Analysis A technique that consists of developing test cases and data that focus on the input and output boundaries of a given function

Boundary Analysis continued... In the same credit limit example, boundary analysis would test: –Low boundary plus or minus one ($9,999 and $10,001) –On the boundary ($10,000 and $15,000) –Upper boundary plus or minus one ($14,999 and $15,001)

Error Guessing Based on the theory that test cases can be developed based upon the intuition and experience of the Test Engineer For example, in an example where one of the inputs is the date, a test engineer might try February 29,2000 or 9/9/99

Incremental Testing A disciplined method of testing the interfaces between unit-tested programs as well as between system components Involves adding unit-tested programs to a given module or component one by one, and testing each resultant combination

Incremental Testing Types Top-down Bottom-up

Top-Down Begins testing from the top of the module hierarchy and works down to the bottom using interim stubs to simulate lower interfacing modules or programs Modules are added in descending hierarchical order

Bottom-Up Begins testing from the bottom of the hierarchy and works up to the top Modules are added in ascending hierarchical order Bottom-up testing requires the development of driver modules which provide the test input, call the module or program being testing, and display test output

Thread Testing A technique, often used during early integration testing Demonstrates key functional capabilities by testing a string of units that accomplish a specific function in the application

Thread & Incremental Combined Thread and Incremental testing usually utilized together –Units can undergo incremental testing until enough units are integrated and a single business function can be performed, threading through the integrated components

Effective Strategies For a simple two-tier client/server application: 1. Unit and bottom-up incremental testing of the application server components 2. Unit and incremental testing of the GUI or client components 3. Testing of the network 4. Thread testing of a valid business transaction through the integrated client, server, and network

Illustrates various techniques used throughout the test stages

Establishing A Test Policy The establishment of a testing policy is an IS management responsibility Three methods can be used to establish a testing policy: –Management directive –Information technology consensus –User’s meeting

Management Directive One or more senior IT managers write the policy They determine what they want from testing, document that into a policy, and issue it to the department This is an economical and effective method for writing a testing policy; the potential disadvantage is that it is not an organizational policy, but rather the policy of IS management

Information Technology Consensus IT management has a group of more senior and respected individuals in the department to develop a policy Senior management must have the responsibility for accepting and issuing the policy Development of the policy is representative of the thinking of all the IT department It is advantageous, as it involves key members of the IS department, therefore, staff is encouraged to follow the policy The disadvantage is that it is an IT policy and not and organizational policy

User’s Meeting Key members of user management, representing all areas of the organization, work with IS to jointly develop a testing policy IT management has final responsibility for the policy The advantage is that it is a true organizational policy and involves all areas with an interest in testing Disadvantages: –The time required to follow this approach –A policy might be developed that IT is obligated to accept because it is a consensus policy and yet is not the type of policy that IT itself would have written

Test Policy continued... Testing is an organizational responsibility. It is the recommendation of the author that a user committee be convened to develop a testing policy. This meeting serves the following purposes: –It permits all involved parties to participate in the development of a testing policy –It is an educational process where users understand the options and costs associated with testing –It clearly establishes for all involved departments that testing is an organizational responsibility and not just an IS responsibility

Test Policy continued... Processes, procedures, and standards are designed and implemented to support the fulfillment of organizational policies. If there is no organizational testing policy established or supported by executive management, it will be difficult to gain support for the implementation of test processes. Test processes that aren’t tied to a policy are considered guidelines and do not have to be followed.

Independent Testing A fundamental principle of effective testing is that a program or application should not be tested by the developer or development team that created it. –The developer’s goal is to make the application work –The application tester’s goal is to make the application fail

Independent Testing continued... By nature, it is very difficult for an individual to adopt both mindsets. Based on this principle, it is recommended that most testing be performed by independent testers or test teams.

Independent Testing continued... The primary responsibility of individuals accountable for testing activities is to ensure that quality is measured accurately Roles and reporting structure of test resources differ across and within organizations –Business or Systems Analysts –Test Engineers Ideally, the test resources will have a reporting structure independent from the group designing or developing the application, this helps assure the quality of the application is given as much consideration as the project budget and timeline

Independent Testing continued... Misconceptions regarding the skill set required to perform testing: –Testing is easy –Anyone can perform testing –No training or prior experience is necessary

Independent Testing continued... In truth, to test effectively, an individual must: –Thoroughly understand the system –Thoroughly understand the technology the system is being deployed upon (e.g. client/server or Internet technology introduce their own challenges) –Possess creativity, insight, and business knowledge –Understand the development methodology being used and the resulting artifacts

Independent Testing continued... An independent test team is usually responsible for system testing, the overseeing of acceptance testing, and providing an unbiased assessment of the quality of an application Team may also support or participate in other phases of testing as well as executing special test types, such as performance and load testing

Independent Testing continued... Usual Test Team Is Comprised Of: –Test Manager or Team Lead –Test Engineers –Additional Testers

Independent Testing continued... Roles Of Players On The Test Team: –Test Manager should join the team no later than the start of the requirements definition stage –Key Test Engineers may also join the team at this stage on large projects to assist with test planning activities –Other Test Engineers can join later to assist with the creation of test cases and scripts –Additional Testers, including users who will participate in testing, usually join the test team right before system testing is scheduled to begin

Independent Testing continued... The Test Manager ensures that testing is performed, that it is documented, and that testing techniques are established and developed. They are responsible for ensuring that tests are designed and executed in a timely and productive manner, as well as: –Test planning and estimation –Designing the test strategy –Reviewing analysis and design artifacts –Chairing the test readiness review –Managing the test effort –Overseeing acceptance tests

Independent Testing continued... Test Engineers are usually responsible for: –Developing test cases and procedures –Test data planning, capture, and conditioning –Reviewing analysis and design artifacts –Test execution –Utilizing automated test tools for regression testing –Preparing test documentation –Defect tracking and reporting

Independent Testing continued... Other Testers: –Primarily focus on test execution –Defect reporting –Regression testing These Testers may be junior members of the test team, users, marketing or product representatives, etc...

Independent Testing continued... The test team should be represented in all key requirements and design meetings, including: JAD or requirements definition sessions, risk analysis sessions, prototype review sessions, etc… They should also participate in all inspections or walk-throughs for requirements and design artifacts

Testing Principles And Concepts - Summary Establish a Test Policy Develop a test strategy that is relevant to the situation –Include the types of testing to be performed Ensure testing is performed by an independent test team

Bibliography CSTE Study Guide, QAI, 2000 Perry, William E. Effective Methods for Software Testing, Wiley Computer Publishing, 1995