1. Towards the Integration of Quality Attributes into a Software Product Line Cost Model
Andy Nolan1, Silvia Abrahão2 Paul Clements3,
John McGregor4, Sholom Cohen3
1Rolls-Royce, UK
2Universidad Politécnica de Valencia, Spain
3Software Engineering Institute, USA
4Clemson University, USA

2. 2
Rolls-Royce

4. The need for Software Product Lines4
The Control Systems department is responsible for the Engine Electronic Controllers (EECs) for a range of small and large gas turbine engines for the aerospace industry.
The software is developed to DO-178B Level-A standards
The company has been developing high integrity software for over 20 years and has extensive data on its processes and productivity.
We have the largest order book in history, new engine development places greater demand on the software team (shorter time scales and lower costs)
Since 2008, Rolls-Royce has developed a PL which has potential for both the software and hardware aspects of engine design.

5. The need for Cost Models5
Since 2004, we have invested in cost models to predict software cost based on COCOMO II.
The cost models have led to an improvement in stability and productivity, with an average of around 11% productivity gain
This benefit has occurred because a project has a better understanding of the Quality Attributes that affect cost and risk.
When the SPL initiative was launched in 2008, the development of a reliable and comprehensive estimation model was seen as critical to making the right decisions to develop core assets that would bring the greatest business value.

6. 6
Recap of SPLC 2009

7. Overview of the Model 7 This picture shows the overview of the model.
Step 1: provided the size of each individual asset based on a library of features from past projects.
Step 2: determine the asset value. The assets has been “weighted” to reflect their value rather than size. This was done because it was observed that not all assets are equal % of the assets account for 70% of the costs so, SIZE is not the KEY factor---.
A new step “P-Process Model” was added to model the safety-critical development process and to understand which processes are required to develop an asset and which processes are required to deploy an asset. We also recognized that the process used to develop and deploy an asset varies depending on the asset variation mechanism.
Step 3: was used to map the life of features ACROSS THE FUTURE ENGINE PROGRAMS.
Step 4: was introduced so that the SPL team could understand the COSTS to develop a project using the TRADITIONAL METHODS and then COMPARE these costs with the SPL development costs (see step 10).
Step 5 was introduced to model the additional costs to develop a SPL asset. This information was taken from CO-PL-MO – the RUSE (developing for reuse) and DOCU (additional documentation) factors.
Step 6 was used to map the deployment of SPL assets into projects to understand if there was a net benefit, per asset, when considering the development, deployment and maintenance of assets. The step also revealed those features that would be bespoke to each project.
Step 7 was a mechanical process and generated all the information together to understand the new costs for developing a project based on SPL assets.
Step 8 was added to model the organization costs. This was derived from the SIMPLE model and consists of costs above-and-beyond traditional project development.
Step 9 was added to model the effect of introducing the SPL initiative into the business. With any change comes an initial “shock” followed by a settling in period. COCOMO II was again used to model the business environment.
Step 10 performed the final analysis and compared and contrasted the benefits of the SPL with a more traditional project centric organization.
Rather then write more, this was taken from the paper. I quickly described the 10 steps and showed how we added drag factors to create a “realistic” business case.
This may not be relevant to this paper. I suggest, when presenting, ask the audience to read the paper for more detail. OK!
The model can be used for a variety of purposes such as to communicate with business leaders, model decisions at the business, perform risk analysis, assess the return on investment of improvements, etc.

8. 8
Recap of SPLC 2010

9. Cost = Size * Environment
Size isn't everything!
Environment
Team capability
Team experience
Team complexity and location
Process capability
Management capability
Tool & IT capability
Project complexity
Integration complexity
Development approach
Development standard
Development domain
Size
Lines of Code
Complexity

10. cost cost SPL COCOMO II Size & Complexity Quality Attribute10
cost
COCOMO
cost
COCOMO II
Size & Complexity
Quality Attribute
Size, Complexity, Testability
Maintainability
Maturity, Reusability, Portability, Variability......
SPL
The SPLC 2010 paper proposed that the existing cost model had to be extended with other Quality Attributes to get a better understanding of costs & benefits

11. The evolution of cost models11
[2004] First generation of the Rolls-Royce cost model was based on COCOMO II
The software product is represented as a single size measure (SLOC). Limited use of the architecture or characteristics of the product being developed
[2008] Second generation of the model allowed us to estimate the cost to develop and deploy individual core assets
The model estimates the cost-benefit of each core asset based on size, complexity, volatility, and difficulty to develop and deploy
[2011] To understand the relevant Quality Attributes of the core assets and the relationship between them
To understand and integrate Quality Attributes into the cost model

12. Selecting Quality Attributes12
Selecting Quality Attributes
SPLC 2011

13. Objectives 13
Look at quality attributes and their impact on the cost- benefit of Software Product Lines
Every factor (question) a cost model considers is another opportunity to: Refine your understanding, make trade decisions and optimise the project
Incorporate quality attributes into the Rolls-Royce Cost Model to analyze:
The cost/benefit of having core assets with certain qualities
The impact on quality attributes of design decisions
Use the cost model to derive design principles for the core asset team
For example how much variability should there be and when ius there too much or too little?

14. Maturity 14
Maturity of a core asset has been defined here as the “degree to which an asset is free from further modification”.
A low maturity asset is likely to be exposed to changes and depending on when they manifest, this can lead to high levels of effort to fix defects
Maturity becomes a sensitive issue for SPLs especially if products are using low maturity assets.
Maturity can be estimated from process exposure and project exposure

15. Testability 15
It is an important quality attribute for any type of product, and in particular for safety-critical products.
A non-SPL safety-critical product invests around 52% of its total development effort on some form of verification and validation.
In a SPL at Rolls-Royce, data shows that RELATIVELY up to 72% of a product’s overall effort will be spent in some form of V&V
Testability can be estimated from the number of test cases (decision points) required to exercise the complete asset

16. Need estimation tool based on both the number AND type of variations
Variability 16
The selection of a specific variation mechanism for a core asset can have an impact on the final product development & deployment cost.
The cost of variability in a core asset is calculated by multiplying the cost of deploying the asset (in a specific process) by the cost of using the different variation mechanisms
Future Work
Need estimation tool based on both the number AND type of variations

17. Need to develop a pre-process measure of reusability17
The reusability of core assets largely determines the success of product line projects.
Higher reusability of core assets indicates the potential for a higher return-on-investment. This can be measured retrospectively.
Future Work
Need to develop a pre-process measure of reusability

18. Need to develop a pre-process measure of maintainability18
Maintainability of core assets is of great importance. It is also important to understand the degree of maintainability that a core asset should have to be cost-effective to the business.
Measured by the effort to maintain an asset (normalised by size).
Future Work
Need to develop a pre-process measure of maintainability

19. 19
Trade off analysis
SPLC 2011

20. Quality Attribute Trade-offs20
Firstly, we created a hypotheses diagram to show the relationship between the Quality Attributes

21. Quality Attribute Trade-offs21
Then we measured the relationship between the attributes (strength and direction)

22. Quality Attribute Trade-offs22
Now we can derrive information from the model ?
is it better to
have 2 simpler assets or 1 larger asset, to have selectable assets rather than configurable, constrain the requirements or build in variability?

23. 23
SPLC 2012?

24. Next Steps 24
The Rolls-Royce estimation model will need to be updated to take into account the other quality attributes selected (i.e., maturity, variability and testability)
Other quality attributes will be evaluated and the model developed further
Model v.2. Without derived relationships
Cost
Business Sat.
Project Sat.
Variability
Volatility
Testability
ROI
Benefit
Maturity
Reusability
Customer Sat.
Maintainability -L -H -M H L M

25. 25
1st International Workshop on Quantitative Methods in Software Product Line Engineering (QMSPLE 2011)
QMSPLE 2011