1. 1 07 September 2004 Parastoo Mohagheghi The Impact of Software Reuse and Incremental Development on the Quality of Large Systems Parastoo Mohagheghi, Dept. Computer and Information Science (IDI), University of Science and Technology (NTNU), Trondheim parastoo@idi.ntnu.no

2. 2 07 September 2004 Parastoo Mohagheghi What is this presentation about? It presents results of a series of empirical studies at Ericsson in Grimstad in 2001-2004.It presents results of a series of empirical studies at Ericsson in Grimstad in 2001-2004. The studies are performed during a doctoral work in the INCO project and supervised by Prof. Reidar Conradi.The studies are performed during a doctoral work in the INCO project and supervised by Prof. Reidar Conradi. 11 students were involved in different studies, an example of university-industry cooperation.11 students were involved in different studies, an example of university-industry cooperation. For more details, see the PhD thesis at: http://www.idi.ntnu.no/grupper/su/publ/phd/moha gheghi-thesis-10jul04-final.pdfFor more details, see the PhD thesis at: http://www.idi.ntnu.no/grupper/su/publ/phd/moha gheghi-thesis-10jul04-final.pdf http://www.idi.ntnu.no/grupper/su/publ/phd/moha gheghi-thesis-10jul04-final.pdf http://www.idi.ntnu.no/grupper/su/publ/phd/moha gheghi-thesis-10jul04-final.pdf

3. 3 07 September 2004 Parastoo Mohagheghi Motivation Systematic software reuse, product family development and incremental development seem be potent technologies to achieve benefits in productivity, quality and maintainability, and to reduce risks of changes.Systematic software reuse, product family development and incremental development seem be potent technologies to achieve benefits in productivity, quality and maintainability, and to reduce risks of changes. Quantifiable benefits, but few empirical studies in industry that can verify these benefits or possible other impacts.

4. 4 07 September 2004 Parastoo Mohagheghi Company background Ericsson has developed two large-scale telecom systems (470 KSLOC or 1000 KSLOC in equivalent C each) that shareEricsson has developed two large-scale telecom systems (470 KSLOC or 1000 KSLOC in equivalent C each) that share –Software architecture, –Components and a component model, –Development environment, –Software process. Systems are developed incrementally, and lots of data is gathered during 5 years on defects, changes, characteristics etc.Systems are developed incrementally, and lots of data is gathered during 5 years on defects, changes, characteristics etc.

5. 5 07 September 2004 Parastoo Mohagheghi Overview of the Ericsson packet- switched core network- GPRS

6. 6 07 September 2004 Parastoo Mohagheghi Evolution of the GSN software architecture and the software process model

7. 7 07 September 2004 Parastoo Mohagheghi The GSN RUP

8. 8 07 September 2004 Parastoo Mohagheghi Research questions RQ1. Why a reuse program is initiated and how is it implemented?RQ1. Why a reuse program is initiated and how is it implemented? RQ2. What is the impact of software reuse, Component-Based Development (CBD) and incremental development on the quality? The impact of development approaches on product quality metrics and on project attributes such as schedule or effort are sought.RQ2. What is the impact of software reuse, Component-Based Development (CBD) and incremental development on the quality? The impact of development approaches on product quality metrics and on project attributes such as schedule or effort are sought. RQ3. How to improve the practice of incremental development of product families in some aspects?RQ3. How to improve the practice of incremental development of product families in some aspects?

9. 9 07 September 2004 Parastoo Mohagheghi Research methods Quantitative studies by exploring (‘data mining’) company databases and assessing hypotheses.Quantitative studies by exploring (‘data mining’) company databases and assessing hypotheses. Quantitative results are discussed with experienced personnel, and combined with qualitative feedbacks and studies of the process and practice to assess validity of the results.Quantitative results are discussed with experienced personnel, and combined with qualitative feedbacks and studies of the process and practice to assess validity of the results. Several case studies, a small survey and an experiment.Several case studies, a small survey and an experiment. Combining results of several studies in the interpretation phase:Combining results of several studies in the interpretation phase: –The impact of introducing reuse or incremental development is widespread. –Taking benefit of all available data, –Confirming results by other studies/data.

10. 10 07 September 2004 Parastoo Mohagheghi Qualitative study of RUP in the context of reuse The approach to initiating a product family was an extractive one.The approach to initiating a product family was an extractive one. Software architecture has evolved to support reuse, while GSN RUP has not been adapted for reuse to the same degree.Software architecture has evolved to support reuse, while GSN RUP has not been adapted for reuse to the same degree. Examples of proposals:Examples of proposals: –“Domain analysis” and “Make vs. Reuse vs. Buy” decisions in the inception phase. –“Update reuse documentation” for reusable components. –“Record reuse experience” in the conclusion phase.

11. 11 07 September 2004 Parastoo Mohagheghi Quantitative study of Trouble Reports (TRs) For defects detected during integration testing, system testing, or later in maintenance.For defects detected during integration testing, system testing, or later in maintenance. For defects detected in previous releases.For defects detected in previous releases. For all types of defects (software, hardware, toolbox, and documentation).For all types of defects (software, hardware, toolbox, and documentation). We have ‘data mined’ databases:We have ‘data mined’ databases: –13000 TRs for several systems and releases were parsed, data was extracted, and inserted in a SQL database by a C# program. –Inconsistencies were resolved -> Data was not analyzed. The company provided us excel sheets on software size (software module level). The company provided us excel sheets on software size (software module level).

12. 12 07 September 2004 Parastoo Mohagheghi Hypotheses: Software Reuse and Quality (Defect-density and Stability) H01 Reused components have the same defect-density as non-reused ones. Rejected Rejected H02 There is no relation between number of defects and component size for all/reused/non-reused components. Not-rejectedNot-rejectedRejected H03 There is no relation between defect- density and component size for all/reused/non-reused components. Not-rejectedNot-rejectedNot-rejected H04 Reused and non-reused components are equally modified. Rejected

13. 13 07 September 2004 Parastoo Mohagheghi H1: Reuse and defect-Density, Release 3 and blocks Defect-density of blocksMeanMedianVariance #TRs/KLOC, Reused1.320.761.70 #TRs/KLOC, Non-Reused3.012.444.39 #TRs/MKLOC, Reused3.501.7821.26 #TRs/MKLOC, Non-Reused5.693.7321.76 Reused components have lower defect density.Reused components have lower defect density. Reused components have proportionally more defects with Severity A (highest severity).Reused components have proportionally more defects with Severity A (highest severity). Reused components have proportionally less defectsReused components have proportionally less defects after delivery.

14. 14 07 September 2004 Parastoo Mohagheghi H2: Size vs. the number of Defects

15. 15 07 September 2004 Parastoo Mohagheghi H3: Size vs. Defect-Density

16. 16 07 September 2004 Parastoo Mohagheghi Discussion Reuse benefits: Reused components are less defect-prone and more stable.Reuse benefits: Reused components are less defect-prone and more stable. Why? Reused components are designed more thoroughly, and changed with care.Why? Reused components are designed more thoroughly, and changed with care. –Confounding factors (conclusion validity): Programming language. Not applicable.Programming language. Not applicable. Type of functionality. Not applicable for stability, but may be for defect-density.Type of functionality. Not applicable for stability, but may be for defect-density. Developers’ experience: Not applicable.Developers’ experience: Not applicable. Internal validity: Missing data, but not systematic.Internal validity: Missing data, but not systematic. Construct validity: Are defect-density and stability indicators of quality?Construct validity: Are defect-density and stability indicators of quality? External validity: At least inside the company for similar systems, or in the domain.External validity: At least inside the company for similar systems, or in the domain.

17. 17 07 September 2004 Parastoo Mohagheghi Contributions Research:Research: –Empirical verification of reuse benefits. No other studies on large industrial systems. –Combined with other studies in the company to assess development approaches and identifying metrics. Company:Company: –Evaluating the trouble reporting system. –Evaluating the measurement program: Granularity of component definition and data for incremental development. –Identifying defect-prone and unstable components.

18. 18 07 September 2004 Parastoo Mohagheghi Software evolution and incremental development With incremental development, software changes:With incremental development, software changes: –during each release (development) –between successive releases (evolution) –and after delivery (maintenance). Some changes are pre-planned (incremental development) and some are not (iterative improvement, adaptive changes etc.).Some changes are pre-planned (incremental development) and some are not (iterative improvement, adaptive changes etc.). The IEEE Standard 1219 on software maintenance defines software maintenance as “The modification of a software product after delivery to correct faults (i.e. corrective), to improve performance or other attributes (i.e. perfective/preventive), or to adapt the product to a modified environment (adaptive)”.The IEEE Standard 1219 on software maintenance defines software maintenance as “The modification of a software product after delivery to correct faults (i.e. corrective), to improve performance or other attributes (i.e. perfective/preventive), or to adapt the product to a modified environment (adaptive)”.

19. 19 07 September 2004 Parastoo Mohagheghi Quantitative study of Change Requests (CRs) Ericsson defines requirements for each release in an ARS (Application Requirement Specification).Ericsson defines requirements for each release in an ARS (Application Requirement Specification). Changes to ARS or previous deliveries (releases) are handled by issuing Change Requests (CRs).Changes to ARS or previous deliveries (releases) are handled by issuing Change Requests (CRs). –CRs handle coarse-grained changes to add/delete functionality, improve performance or improve design. –CRs cover changes during developing each release as well as between releases (in addition to new requirements stated in the ARS). How this data can be used in our research?How this data can be used in our research? –169 CRs for 4 releases of one system were analyzed. –Exploratory study: Origin (internal vs. external), acceptance rate and classes of changes are studied.

20. 20 07 September 2004 Parastoo Mohagheghi Origin of CRs Perfective/FunctionalPerfective/ Quality Attributes (QA) AdaptivePreventive Other (cost) No407435308 18 CRs have indicated two reasons. Most perfective CRs are to improve QA (also preventive CRs improve quality). 23 of 169 CRs are issued because of customer demands. These and adaptive CRs have external origin (34% of all CRs).

21. 21 07 September 2004 Parastoo Mohagheghi Acceptance Rate over Releases Acceptance rate is increasing (59% in total); i.e. the product is getting more change-prone.

22. 22 07 September 2004 Parastoo Mohagheghi Conclusions Most changes originate from the project organization itself in order to improve quality and enhance functionality. The share of the first group is higher.Most changes originate from the project organization itself in order to improve quality and enhance functionality. The share of the first group is higher. The practice indicates that the iterative realization and improvement of quality attributes has great impact on the evolution of the products.The practice indicates that the iterative realization and improvement of quality attributes has great impact on the evolution of the products. We could not observe any significant difference between reused and non-reused components in the number of CRs per KLOC.We could not observe any significant difference between reused and non-reused components in the number of CRs per KLOC.

23. 23 07 September 2004 Parastoo Mohagheghi Conclusions (continued) Most CRs are accepted and the acceptance rate can have impact on the project plans in terms of decreasing planning precision (observed). Incremental development opens for (more) changes.Most CRs are accepted and the acceptance rate can have impact on the project plans in terms of decreasing planning precision (observed). Incremental development opens for (more) changes. Most CRs are issued pre-delivery, and especially in the short time right after requirement baseline. The organization possibly takes the decision to baseline requirements too early.Most CRs are issued pre-delivery, and especially in the short time right after requirement baseline. The organization possibly takes the decision to baseline requirements too early.

24. 24 07 September 2004 Parastoo Mohagheghi Overview of all studies

25. 25 07 September 2004 Parastoo Mohagheghi Impact of dev. approaches and context Reuse Incr. dev. CBD Large systems Software evolution Change rate Earlier rel. not evolved Granularity Improved incr. Long-lived Effort estimation ROI? Software from a previous rel., Incr. changes in requirements, Large CM No data for components Large CM and testing effort Metricsyesyesyes Different granularity

26. 26 07 September 2004 Parastoo Mohagheghi Mining industrial databases Top-down theory search integrated with bottom-up data exploring.

27. 27 07 September 2004 Parastoo Mohagheghi Assessing development approaches Development approaches should be seen as coherent systems of practices, be chosen depending on the attribute that should be optimized and may trade-off for other practices [IEEE software, MacCormack, 2003 ]

28. 28 07 September 2004 Parastoo Mohagheghi Main Contributions Empirical verification of reuse benefits.Empirical verification of reuse benefits. Increased understanding of the origin and type of changes in each release and between releases.Increased understanding of the origin and type of changes in each release and between releases. Identifying metrics for a combination of reuse and incremental development.Identifying metrics for a combination of reuse and incremental development. A data mining method for exploring industrial databases.A data mining method for exploring industrial databases. An effort estimation method for incremental development.An effort estimation method for incremental development.

29. 29 07 September 2004 Parastoo Mohagheghi Discussion Studies are performed in a real context.Studies are performed in a real context. –Context-independent knowledge is rare in SE and generalization is difficult. Studies combine different methods and data (triangulation).Studies combine different methods and data (triangulation). –Integration of the results is a challenge: physical and conceptual integration challenges. Incremental development is combined with reuse of software in multiple products and CBD.Incremental development is combined with reuse of software in multiple products and CBD. –All aspects of software development need adaptation; e.g. estimation method, software process model, data collection routines etc.

30. 30 07 September 2004 Parastoo Mohagheghi Discussion (continued) Own experience and access to internal dataOwn experience and access to internal data –Confidentiality of some results and other ethical issues should be respected. Some studies are evaluation of others’ observations or methods in a new context (e.g. estimation method, reuse benefits or distribution of maintenance categories), while others contain new observations and methods.Some studies are evaluation of others’ observations or methods in a new context (e.g. estimation method, reuse benefits or distribution of maintenance categories), while others contain new observations and methods. Being exposed to organizational changes leaded to an emerging research design and changes in plans.Being exposed to organizational changes leaded to an emerging research design and changes in plans. Working in the field needs flexibility and a suitably time- frame.Working in the field needs flexibility and a suitably time- frame.

31. 31 07 September 2004 Parastoo Mohagheghi Why such studies are important for industry? Methods should be tested for development in large.Methods should be tested for development in large. Companies are increasingly using mainstream methods, tools and programming languages.Companies are increasingly using mainstream methods, tools and programming languages. The studies analyzed data that was hardly studied before: evaluation of data collection routines and metrics for the company, in addition to concrete results.The studies analyzed data that was hardly studied before: evaluation of data collection routines and metrics for the company, in addition to concrete results.

32. 32 07 September 2004 Parastoo Mohagheghi Acknowledgements Thanks to INCO, Simula and NTNU for financial support.Thanks to INCO, Simula and NTNU for financial support. Thanks to my supervisor, Prof. Reidar Conradi, and colleagues at INCO and IDI.Thanks to my supervisor, Prof. Reidar Conradi, and colleagues at INCO and IDI. Thanks to Ericsson for the permission to perform studies and to publish the results.Thanks to Ericsson for the permission to perform studies and to publish the results. Thanks to the SPIKE seminar and the audience.Thanks to the SPIKE seminar and the audience.