1. Software Reliability Engineering Tool Nederlandse Testdag 27 november 2008 Alessandro di Bucchianico, Lime – TU/e Ed Brandt, Refis VOF

2. 2 2008© Refis/Lime-TU/e Definition of “Reliability” "The probability that an item will perform a required function without failure under stated conditions for a stated period of time”

3. 3 2008© Refis/Lime-TU/e Reliability growth curve 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 1234567 interval reliability period = 5

4. 4 2008© Refis/Lime-TU/e

5. 5 2008© Refis/Lime-TU/e

6. 6 2008© Refis/Lime-TU/e Step 1: Data collection document data sources record data collection procedures clean-up data

7. 7 2008© Refis/Lime-TU/e Step 1: Data collection

8. 8 2008© Refis/Lime-TU/e Step 1: Data collection

9. 9 2008© Refis/Lime-TU/e

10. 10 2008© Refis/Lime-TU/e Step 2: Determine Growth formal tests (Laplace, MIL-HB, LRT,…) time cumulative number of defects

11. 11 2008© Refis/Lime-TU/e

12. 12 2008© Refis/Lime-TU/e Step 3: Initial Model Selection

13. 13 2008© Refis/Lime-TU/e

14. 14 2008© Refis/Lime-TU/e Step 4: Estimate Models parameters cumulative number of defects likelihood MLLS time

15. 15 2008© Refis/Lime-TU/e

16. 16 2008© Refis/Lime-TU/e Step 5: Validate Model goodness-of-fit plots (TTT-plot, u-plot, QQ-plot, residual plots) goodness-of-fit tests (Kolmogorov, Zhao-Wang) time cumulative number of defects maximal absolute deviation

17. 17 2008© Refis/Lime-TU/e

18. 18 2008© Refis/Lime-TU/e Step 6: Interpret Model time cumulative number of defects use confidence bounds to learn about precision

19. 19 2008© Refis/Lime-TU/e

20. 20 2008© Refis/Lime-TU/e SREtool: goals provide support for software reliability analyses enable to apply best practices from statistics make available state-of-the-art statistical algorithms

21. 21 2008© Refis/Lime-TU/e SREtool: features available for Windows, Unix and Linux import and export functions data editor wizard for initial model selection consistency check on data

22. 22 2008© Refis/Lime-TU/e SREtool: implementation statistical procedures written in R R is open source, free statistics software R is maintained by a core group of world- class experts in computational statistics interface written in Java combination of R and Java using tool kit from University of Augsburg (Prof. Unwin)

23. 23 2008© Refis/Lime-TU/e Sample data cumulatief aantal defects 0 20 40 60 80 100 120 140 160 180 200 13579 1113151719212325272931333537 testinterval (dagen) aantal defects basis gegevens t/m 8/11 basis gegevens t/m 16/11 geschat aantal defects Yamada S-shaped

24. 24 2008© Refis/Lime-TU/e Conclusions statistics can be a useful additional method to evaluate software reliability a new tool will soon be available for software reliability analyses

25. 25 2008© Refis/Lime-TU/e Some of Refis clients and services Reliability analysis Quality metrics dashboard Test and quality management Process improvement

26. 26 2008© Refis/Lime-TU/e Merellaan 5 3722 AK Bilthoven T +31 30 225 36 37 F +31 30 225 36 49 M +31 6 543 12 528 W www.refis.nlwww.refis.nl E info@refis.nlinfo@refis.nl Postbus 513 5600 MB Eindhoven T +31 40 247 48 75 F +31 40 244 24 89 W www.lime.tue.nlwww.lime.tue.nl E lime@tue.nllime@tue.nl