1. Wastage in software testing Team Manager (Linde AG)
Indrajit Ghatak
Team Manager (Linde AG)
Arnab Patra, Test Specialist – Automation Tools (IBM)
Arghyadip Ghosh, Test Specialist – Automation Tools (IBM)

2. Abstract
The measurement and metrics for software testing are primarily around test execution efficiency, cost, coverage, defects etc. However, we seldom measure the effort and cost for tests that get wasted in software testing. Every time there are changes in the requirement, we go ahead and change the scripts. If the extent of change is significant, the old scripts are retired and get replaced by new scripts. At times, there are test scripts that never get executed and require complete re-scripting. The effort and cost that goes into such scripts are completely wasted. If this waste is acknowledged, measured and tracked we will start to realize when “not to create tests”, thus preventing the effort and costs for tests that get continually wasted. In the process, it will also make software testing a more valuable activity and not something that can be wasted at will.

3. Waste is not evil
The most important thing is to acknowledge the fact that every software testing project has some “waste”. It is not an “evil”, but software testing reality.
We shall henceforth call it “testing waste”. It is anything that gets unused during software testing lifecycle.
Here are a few examples:
We onboard consultants but they need to wait for 2 weeks.
We procure a testing tool and we create only PoCs (Proof of Concepts) and never deploy the tool for any commercial project.
We design 500 test cases and by the time we need to execute these test cases, 200 of these are obsolete due to major changes in the software.

4. Composite set of activities
Software testing is not a single activity but a composite set of activities. There are several dependencies and each of these have potential waste components. Here are a few examples:
The software itself can have several waste. It can change, it can upgrade, it can migrate or the platform on which it has been built can itself change.
Environments can change, can merge or completely new environments can be created.
Data might change.
There might be changes in organizational strategy.
Statutory norms, compliance, taxation laws etc.
As you can see, none of these are specific to testing, but all of these can result in testing waste.

5. Waste value
In the estimation model, testing waste needs to be included. Each estimation element should have a waste component, which needs to be estimated and waste value ($) needs to be calculated.
For each waste value, set thresholds, as per the financial appetite of the project. For example, a project with a strict budge should not have an idle time of X (person hours) resulting in Y ($) of waste.
Estimate the waste value of the testing project in one or more quarters to check if such waste is worthwhile. For example, if the objective of the project it to give you a positive ROI (Return on Investment) in two quarters and the estimated waste value outweighs that, you need to re-think why you would still like to go ahead with the project.

6. Additional lever
Estimating testing waste is not good enough. It is important to closely track it and see where it is going. There is a possibility that waste has been over or under estimated. In either case, the reason for more or less waste should not “known” and well understood.
If the waste is “too much” and coming out of something that is not “too important”, curtail it. For example, you have invested your resources for an application that is going to be decommissioned. Most of these resources are idle. Why waste? Take them out and deploy for something that has more value and less of waste!
Testing waste gives you an “additional lever” to weigh out cost in terms of value by tracking elements that have unutilized cost and might even have unknown values.

7. Testing waste metrics
Testing waste metrics can result in interesting reports. Here are a few examples:
What was the utilization of all the servers which were used in testing? What was the idle time and how much it costed?
How many user licenses were used, how many unutilized and how much it costed?
How many tests were designed, what % of tests saw the light till execution, how many were unused, how many reworked, how many got added because of the discarded ones and how much all these cost?
How many recruitment drives happened, how many positions got confirmed and were deployed and what was the loss for positions which never materialized?
How much testing effort was wasted because of testing tool migration?

8. Prevent testing waste
It makes sense to carry out a retrospective analysis of testing waste by identifying the areas that resulted in maximum waste.
There might be areas with lot of waste and these were very important to business. If the waste was worthwhile since there was no better option, we might need to “live with” such waste but look for better preventive options in future.
There might be staggering $ being wasted on areas that might have hardly mattered. This is where we need to focus most and make it a point to reduce it upto the maximum extent.
To summarize, acknowledge testing waste, estimate/track it closely, optimize the ones we need to live with and kill the ones that wastes most with no obvious value.

9. Indrajit Ghatak has been in software testing industry for more than 13 years. He started my career as a software developer and had the opportunity to setup the first independent software testing team for Web Spiders India Private Limited at Kolkata, India. He worked with IBM for 7+ years and his experience includes leading the test automation service area of IBM India for 2 years (as SAL - Service Area Leader). Key focus areas included: business growth, accelerate ROI and niche skills, improve the quality of test automation deliverable, deepen capability, strengthen positioning, improve client value, align capacity/capability with market demand, built the right pyramid mix, define new training model as per market, improve technical vitality and skill development, maintain accuracy of skills, optimize staffing and fulfillment of demands, forecast and approve all hiring decisions for testing, band-mix management, improve diversity ratio, improve fill rates, reduce cost of idle resources, continuous delivery improvement, harvest assets and accelerators, improve practice maturity, align offering with industry and improve test automation productivity.
Currently, he is working in the independent application testing team of Linde at Kolkata as a Team Manager.
In January 2016 he founded a voluntary software testing group named Kolkata Testers. The vision of this group is philanthropic and transformational with an objective to provide a deep platform of direction and learning for software testers based out of Kolkata.
Web:

10. Thank You!!!