Software Reliability Engineering Tool Nederlandse Testdag 27 november 2008 Alessandro di Bucchianico, Lime – TU/e Ed Brandt, Refis VOF

2 2008© Refis/Lime-TU/e Definition of “Reliability” "The probability that an item will perform a required function without failure under stated conditions for a stated period of time”

3 2008© Refis/Lime-TU/e Reliability growth curve 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0, interval reliability period = 5

4 2008© Refis/Lime-TU/e

5 2008© Refis/Lime-TU/e

6 2008© Refis/Lime-TU/e Step 1: Data collection document data sources record data collection procedures clean-up data

7 2008© Refis/Lime-TU/e Step 1: Data collection

8 2008© Refis/Lime-TU/e Step 1: Data collection

9 2008© Refis/Lime-TU/e

© Refis/Lime-TU/e Step 2: Determine Growth formal tests (Laplace, MIL-HB, LRT,…) time cumulative number of defects

© Refis/Lime-TU/e

© Refis/Lime-TU/e Step 3: Initial Model Selection

© Refis/Lime-TU/e

© Refis/Lime-TU/e Step 4: Estimate Models parameters cumulative number of defects likelihood MLLS time

© Refis/Lime-TU/e

© Refis/Lime-TU/e Step 5: Validate Model goodness-of-fit plots (TTT-plot, u-plot, QQ-plot, residual plots) goodness-of-fit tests (Kolmogorov, Zhao-Wang) time cumulative number of defects maximal absolute deviation

© Refis/Lime-TU/e

© Refis/Lime-TU/e Step 6: Interpret Model time cumulative number of defects use confidence bounds to learn about precision

© Refis/Lime-TU/e

© Refis/Lime-TU/e SREtool: goals provide support for software reliability analyses enable to apply best practices from statistics make available state-of-the-art statistical algorithms

© Refis/Lime-TU/e SREtool: features available for Windows, Unix and Linux import and export functions data editor wizard for initial model selection consistency check on data

© Refis/Lime-TU/e SREtool: implementation statistical procedures written in R R is open source, free statistics software R is maintained by a core group of world- class experts in computational statistics interface written in Java combination of R and Java using tool kit from University of Augsburg (Prof. Unwin)

© Refis/Lime-TU/e Sample data cumulatief aantal defects testinterval (dagen) aantal defects basis gegevens t/m 8/11 basis gegevens t/m 16/11 geschat aantal defects Yamada S-shaped

© Refis/Lime-TU/e Conclusions statistics can be a useful additional method to evaluate software reliability a new tool will soon be available for software reliability analyses

© Refis/Lime-TU/e Some of Refis clients and services Reliability analysis Quality metrics dashboard Test and quality management Process improvement

© Refis/Lime-TU/e Merellaan AK Bilthoven T F M W E Postbus MB Eindhoven T F W E