Balamurali L Senior SQA Manager Diana Ambrose Senior Lead SQA Arun Kumar V Senior Engineer QA Geethu K Nair Engineer QA 19/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 24 September Software Systems – A Part of Daily Life Banking applications Home appliances Educational applications Cars and other vehicles Communicati- on Media Software is now omnipresent

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Reduction of errors Increasing confidence of end users Ensuring safe operations Contributes to quality Reduces cost of defect fixing 3 Role of Testing 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 24 September Testing: Perspective shift… Establish confidence that a program does what it is supposed to do (Hetzel, 1973)

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 24 September Testing: Perspective shift… The process of executing a program or system with the intent of finding errors (Myers, 1979)

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 24 September Testing: Perspective shift… Verifying that a system satisfies its specified requirements and identifying the differences between expected and actual results

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 7 The Lacuna 9/24/2015

SQT’s STEP-AUTO 2011: Conference on ERP Testing 9/24/20158 Where we stand now… You can manage, what you can measure You can measure, what you can define You can define, what you understand.

ISQT’s STEP-AUTO 2011: Conference on ERP Testing bug prediction and tracking 99/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Introduction Uses baseline data of the organization and size of the current project Predicts the expected number of bugs and the testing schedule for the current project Cumulative bug trend curve resembles the curve of a first order response system The actual cumulative bug trend is tracked against the estimated Any notable deviation from the bug prediction curve, analysis will be triggered and corrective actions taken accordingly. 109/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 11 Cumulative Bug Trend Curve A curve with: –The days of testing marked on X-Axis –The cumulative number of bugs caught on Y-Axis The slope of the curve gives the rate of bug capture 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Monitoring – Bug Prediction Curve – Weekly Status Report – % Schedule Completed – Resource Utilization Process In Testing - Monitoring Schedule ODB Project Team Project Team No. Testing Days No. Testing Days Defect Density Defect Density Size Defect Prevention measures Expected Defects & Schedule Expected Defects & Schedule 12 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 13 Bug Prediction Curve Number of days Cumulative number of bugs Helps in tracking the test execution 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 14 Bug Prediction Curve 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 15 Possible Outcomes The actual curve - within expected trend The actual curve - outside expected trend – The number of bugs in the initial quarters is much less than estimated – The number of bugs in the initial quarters is much more than estimated 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 16 The Scenario Diagram Situations Bugs in initial stage : Less than what is estimated Bugs in initial stage : More than what is estimated Stable Product Inadequate Testing Unstable Product 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 17 Inadequate Testing 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing 18 Unstable Product 9/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Challenges in ERP Testing Wide range of stakeholders that use the system Impact of changes across various modules and higher risk of issues to be found at integration Inherent slippages that is accumulated over various phases of the development lifecycle Tracking and Managing multiple test iterations in parallel High pressure to ensure delivery dates as well as the quality of delivery 199/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing In the ERP context Bug trend curve can be calculated for different iterations Specification limits can be defined for each iterations Actual bug trend curve can be plotted and monitored for each iterations 209/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Bug Trend Curve for multiple iterations 219/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Test Status Indicator 229/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Test Status Indicator Root mean square of the variation between the estimated and actual bug trend curve The variation between the planned and actual value of the bugs can be calculated on a daily basis. The overall TSI values of the total projects in an organization can be plotted in a control chart for monitoring. The specification limits can be defined and from the control chart, the points which fall outside the limits can be analyzed by the top management. 239/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing SPC for TSI 249/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing SPC for TSI in different iterations 259/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Advantages 269/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Case Study ProjectLanguagePlat formStart Date Cost of a Bug (Hrs/Bug) % Testing Effort Before implementation of the method Project 1VCWin NT13-Mar % Project 2VCWin NT21-Dec % Project 3C#Win NT13-Mar % After implementation of the method Project 4VCWin NT16-May % Project 5VCWin NT11-Jun % Project 6C#.net29-Apr % Reduction in % effort for testing Implementation of Bug Prediction and Tracking method ensures quantitative monitoring and control Critical phases tracked in an efficiently and effectively 279/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Questions 289/24/2015

ISQT’s STEP-AUTO 2011: Conference on ERP Testing Thank you 299/24/2015