1. SFO Technologies Pvt Ltd
Test Estimation By Quantitative Project Management – A Probabilistic Approach
Balamurali L – Director Quality & Testing Group
Shajeena Shahul Hameed – Senior Lead Engineer QA
SFO Technologies Pvt Ltd
A NeST Group Company
STC 2016

2. The product is delivered to the testing team
The product is delivered to the testing team. Can plan the customer release in a month’s time
Oh! No.. Lot of bugs… mmm… Atleast another month for external delivery!
A month later…
STC 2016

3. Have estimated the size of project using IFPUG Guidelines
Have estimated the size of project using IFPUG Guidelines.. Productivity taken from organization data base.. Still!!!
What went wrong???
STC 2016

4. The estimates were based only on size and organizational baseline values. Quality of work product and process followed in the project not considered!!
Should have done that…
STC 2016

5. The hard nut
Estimation
STC 2016

6. The Industry Practice Percentage of effort Work Break down structure
Function Points
Test Point Estimates
Test Case Point Estimates
Use case based estimates
STC 2016

7. The Lacunae Quality of work product of each phase not considered
Expertise assumed to be constant across the teams
Test estimates to be refined based on the quality of product in each phase
STC 2016

8. Process Performance Model
Defining the relation between the dependent parameter and the various independent parameters
Identify the parameters which influence the dependent parameter (Testing Effort)
From the organizational data, ensure the co-relation
Conduct Regression Analysis to get the relation between the dependent and independent parameters
STC 2016

9. The PPM Approach
Identify the parameters influencing Testing effort estimate
Size, Expertise of team, Requirement changes, Code Complexity etc.
Establish the relationship
Use Historical data
E.g.. Effort for Testing= * Size of the product – 4.01* Expertise Index –9.4 * Requirement Stability Index *Code Complexity
STC 2016

10. Estimate and Refine the Test Estimate
Fine-tune the data
At defined frequencies
As and when parameters change
STC 2016

11. Estimate - Conventional Method
Testing Productivity = Size/Testing Effort
The estimated effort & actual effort differ a lot due to
poor estimates,
change in requirements
poor quality work products
STC 2016

12. Closed Loop Estimation Model- Initial Phase
PPM based estimate using
Initial Size estimate
Planned Expertise of the developer
Estimated Requirements Stability Index (RSI)
Code Complexity (CC).
STC 2016

13. Closed Loop Estimation Model- Project Planning Phase
Resources are freeze
Initial estimate refined by replacing the actual expertise index instead of estimated expertise index
Expertise index - effort & vice versa
Expertise index
Effort
STC 2016

14. Closed Loop Estimation Model- Along with Requirement Changes
The volatility of the requirements measured
a minor change in requirement  low volatile
a major change in testing phase  highly volatile.
Requirement Stability Index is the complement of volatility
RSI - effort & vice versa
RSI
Effort
STC 2016

15. Closed Loop Estimation Model- After Coding Phase
Cyclomatic complexity of the code measured using the tools
The actual value of complexity replaced for the estimated value there by fine tuning the total estimate
Increase in Code complexity  higher the effort for testing and bug fix & vice versa.
Effort for testing and bug fix
Code complexity
STC 2016

16. Advantage
Better estimate which is fine tuned with the real measures of the factors influencing the total testing effort
The increase (or decrease) of total testing effort is brought to the notice of management well in advance
Better planning is done so that the Test manager has a better control over the chaotic phases
It provides a relation with the upstream parameters of software development to the total testing effort.
Strategic planning can be done for controlling these parameters using statistical process control techniques.
STC 2016

17. Case Study – Conventional Method
Size
(15KLOC)
Total Testing Effort for Project (100PD)
Testing Productivity (150LOC/PD)
STC 2016

18. Closed Loop Estimation Model- Initial Phase
Size
(15KLOC)
Effort (106PD)
Expertise Index
(15)
Estimated RSI
(0.95)
Using the Process Performance Model
Estimated Code Complexity
(20)
STC 2016

19. Fine Tuning of Estimate
Effort (126 PD)
After Planning put the actual Expertise index
(10)
Effort (106 PD)
Effort (127 PD)
After Requirement Changes put the actual RSI
(0.85)
After Coding put the actual code complexity (28)
Effort (152 PD)
STC 2016

20. Variance
52% increase in Effort
STC 2016

21. Scope for future work Prediction Intervals can be introduced
Expertise of teams in each phase can be introduced
PPMs for schedule variation, productivity, defect density.....
STC 2016

22. 22
Questions
STC 2016

23. References & Appendix
Bug Prediction and Tracking - A QPM Approach by Balamurali L and Annu George, NeST, QAI’s SEPG Conference 2003
Test Effort Estimation Using Use Case Points By Suresh Nageswaran, Cognizant Technology Solutions, Pune, Quality Week 2001, San Francisco, California, USA, June 2001
Test Case Point Analysis White Paper By Nirav Patel, Muthukrishnan Govindrajan, Susmita Maharana and Shoba Ramdas From Cognizant Technology Solutions
STC 2016

24. Author Biography
Balamurali L is an M.Tech from IIT Kanpur and has an industrial experience of more than 15 years. He is working as Director- Quality and Testing Group of SFO Technologies Software Division. He has core competency in the areas of process consulting, quality assurance and control. He has played a lead role in implementing the models like CMMI Dev V1.3, and ISO9001:2008, Automotive SPICE, ISO 27001, IEC61508 and ISO He has worked as Assessment Team Member for CMMI V 1.3. A CSQA & CQSPE from QAI, he is also a Certified Tester (advanced level- Test Management) by ISTQB and plays an active role in the core group of the Trivandrum chapter of Software Process Improvement Network (SPIN). He is also a certified Test Manager by Department of IT, Govt. of India. He is a certified lead auditor for ISO 9001:2015 He has conducted many public workshops and corporate trainings on Software Engineering, Quality and Testing. He has also marked his presence in various International Conferences like QAI, STC, and ISQT by presenting papers in concepts of CMMI High Maturity, Software Testing and Automation, Quality Assurance etc.
Shajeena Shahul Hameed is working as Senior Engineer-QA at SFO Technologies. She did her B-Tech from Kerala University and has more than 8 years of experience in IT industry. She is an expert in software models like CMMI and TMMI. She has also played a major role in the implementation of models like CMMI Dev V1.3, ISO 9001, ISO ISO 27001, ISO etc in the organization. She is an active member of the metrics team in organization. She is certified in ISTQB Foundation Level and Six Sigma Green Belt. She has competency in the areas of process consulting, quality assurance and control. She is a certified lead auditor for ISO 9001:2015. She has also marked her presence in various International Conferences like QAI, STC, ISQT by presenting papers in concepts of CMMI High Maturity, Software Testing, Quality Assurance etc
STC 2016

25. Shajeena Shahul Hameed
Thank You!!!
Balamurali L
Shajeena Shahul Hameed
STC 2016