1. ©2011 Rolls-Royce plc The information in this document is the property of Rolls-Royce plc and may not be copied or communicated to a third party, or used for any purpose other than that for which it is supplied without the express written consent of Rolls-Royce plc. This information is given in good faith based upon the latest information available to Rolls-Royce plc, no warranty or representation is given concerning such information, which must not be taken as establishing any contractual or other commitment binding upon Rolls-Royce plc or any of its subsidiary or associated companies. Towards the Integration of Quality Attributes into a Software Product Line Cost Model Andy Nolan 1, Silvia Abrahão 2 Paul Clements 3, John McGregor 4, Sholom Cohen 3 1 Rolls-Royce, UK 2 Universidad Politécnica de Valencia, Spain 3 Software Engineering Institute, USA 4 Clemson University, USA

2. Rolls-Royce 2

3. 3

4. 4 The need for Software Product Lines The Control Systems department is responsible for the Engine Electronic Controllers (EECs) for a range of small and large gas turbine engines for the aerospace industry. The software is developed to DO-178B Level-A standards The company has been developing high integrity software for over 20 years and has extensive data on its processes and productivity. We have the largest order book in history, new engine development places greater demand on the software team (shorter time scales and lower costs) Since 2008, Rolls-Royce has developed a PL which has potential for both the software and hardware aspects of engine design.

5. Rolls-Royce 5 The need for Cost Models Since 2004, we have invested in cost models to predict software cost based on COCOMO II. The cost models have led to an improvement in stability and productivity, with an average of around 11% productivity gain This benefit has occurred because a project has a better understanding of the Quality Attributes that affect cost and risk. When the SPL initiative was launched in 2008, the development of a reliable and comprehensive estimation model was seen as critical to making the right decisions to develop core assets that would bring the greatest business value.

6. Rolls-Royce Recap of SPLC 2009 6

7. Rolls-Royce 7 Overview of the Model

8. Rolls-Royce Recap of SPLC 2010 8

9. Rolls-Royce Environment Team capability Team experience Team complexity and location Process capability Management capability Tool & IT capability Project complexity Integration complexity Development approach Development standard Development domain Cost = Size * Environment Size Lines of Code Complexity 9 Size isn't everything!

10. Rolls-Royce 10 COCOMO II Size & Complexity Quality Attribute Size, Complexity, Testability Maintainability Maturity, Reusability, Portability, Variability...... cost COCOMO SPL 10 The SPLC 2010 paper proposed that the existing cost model had to be extended with other Quality Attributes to get a better understanding of costs & benefits

11. Rolls-Royce The evolution of cost models [2004] First generation of the Rolls-Royce cost model was based on COCOMO II The software product is represented as a single size measure (SLOC). Limited use of the architecture or characteristics of the product being developed [2008] Second generation of the model allowed us to estimate the cost to develop and deploy individual core assets The model estimates the cost-benefit of each core asset based on size, complexity, volatility, and difficulty to develop and deploy [2011] To understand the relevant Quality Attributes of the core assets and the relationship between them To understand and integrate Quality Attributes into the cost model 11

12. Rolls-Royce Selecting Quality Attributes SPLC 2011 12

13. Rolls-Royce Objectives Look at quality attributes and their impact on the cost- benefit of Software Product Lines Every factor (question) a cost model considers is another opportunity to: Refine your understanding, make trade decisions and optimise the project Incorporate quality attributes into the Rolls-Royce Cost Model to analyze: The cost/benefit of having core assets with certain qualities The impact on quality attributes of design decisions Use the cost model to derive design principles for the core asset team For example how much variability should there be and when ius there too much or too little? 13

14. Rolls-Royce Maturity 14 Maturity of a core asset has been defined here as the “degree to which an asset is free from further modification”. A low maturity asset is likely to be exposed to changes and depending on when they manifest, this can lead to high levels of effort to fix defects Maturity becomes a sensitive issue for SPLs especially if products are using low maturity assets. Maturity can be estimated from process exposure and project exposure

15. Rolls-Royce Testability 15 It is an important quality attribute for any type of product, and in particular for safety-critical products. A non-SPL safety-critical product invests around 52% of its total development effort on some form of verification and validation. In a SPL at Rolls-Royce, data shows that RELATIVELY up to 72% of a product’s overall effort will be spent in some form of V&V Testability can be estimated from the number of test cases (decision points) required to exercise the complete asset

16. Rolls-Royce Variability 16 The selection of a specific variation mechanism for a core asset can have an impact on the final product development & deployment cost. The cost of variability in a core asset is calculated by multiplying the cost of deploying the asset (in a specific process) by the cost of using the different variation mechanisms Future Work Need estimation tool based on both the number AND type of variations

17. Rolls-Royce Reusability The reusability of core assets largely determines the success of product line projects. Higher reusability of core assets indicates the potential for a higher return-on-investment. This can be measured retrospectively. 17 Future Work Need to develop a pre-process measure of reusability

18. Rolls-Royce Maintainability Maintainability of core assets is of great importance. It is also important to understand the degree of maintainability that a core asset should have to be cost-effective to the business. Measured by the effort to maintain an asset (normalised by size). 18 Future Work Need to develop a pre-process measure of maintainability

19. Rolls-Royce Trade off analysis SPLC 2011 19

20. Rolls-Royce Quality Attribute Trade-offs 20 Firstly, we created a hypotheses diagram to show the relationship between the Quality Attributes

21. Rolls-Royce Quality Attribute Trade-offs 21 Then we measured the relationship between the attributes (strength and direction)

22. Rolls-Royce Now we can derrive information from the model Quality Attribute Trade-offs 22 ? is it better to have 2 simpler assets or 1 larger asset, to have selectable assets rather than configurable, constrain the requirements or build in variability?

23. Rolls-Royce 23 SPLC 2012?

24. Rolls-Royce Next Steps The Rolls-Royce estimation model will need to be updated to take into account the other quality attributes selected (i.e., maturity, variability and testability) Other quality attributes will be evaluated and the model developed further 24 Model v.2. Without derived relationships Cost Business Sat. Project Sat. Variability Volatility Testability ROI Benefit Maturity Reusability Customer Sat. Maintainability -L -H -M H -H L M M L H H M H M M H M M M H HH H H M H H L M H M

25. Rolls-Royce 25 1st International Workshop on Quantitative Methods in Software Product Line Engineering (QMSPLE 2011) QMSPLE 2011