Transcript Towards the Integration of Quality Attributes into a Software Product
Towards the Integration of Quality Attributes into a Software Product Line Cost Model
Andy Nolan 1 , Silvia Abrahão 2 Paul Clements 3 , John McGregor 4 , Sholom Cohen 3 1 Rolls-Royce, UK 2 Universidad Politécnica de Valencia, Spain 3 Software Engineering Institute, USA 4 Clemson University, USA
©2011 Rolls-Royce plc
The information in this document is the property of Rolls-Royce plc and may not be copied or communicated to a third party, or used for any purpose other than that for which it is supplied without the express written consent of Rolls-Royce plc.
This information is given in good faith based upon the latest information available to Rolls-Royce plc, no warranty or representation is given concerning such information, which must not be taken as establishing any contractual or other commitment binding upon Rolls-Royce plc or any of its subsidiary or associated companies.
Rolls-Royce
Rolls-Royce
2
Rolls-Royce
3
4
The need for Software Product Lines
The Control Systems department is responsible for the Engine Electronic Controllers (EECs) for a range of small and large gas turbine engines for the aerospace industry.
The software is developed to DO-178B Level-A standards The company has been developing high integrity software for over 20 years and has extensive data on its processes and productivity.
We have the largest order book in history, new engine development places greater demand on the software team (shorter time scales and lower costs) Since 2008, Rolls-Royce has developed a PL which has potential for both the software and hardware aspects of engine design.
Rolls-Royce
The need for Cost Models
Since 2004, we have invested in cost models to predict software cost based on COCOMO II.
The cost models have led to an improvement in stability and productivity, with an average of around 11% productivity gain This benefit has occurred because a project has a better understanding of the Quality Attributes that affect cost and risk.
When the SPL initiative was launched in 2008, the development of a reliable and comprehensive estimation model was seen as critical to making the right decisions to develop core assets that would bring the greatest business value.
5
Rolls-Royce
Rolls-Royce
Recap of SPLC 2009
6
Overview of the Model
Traditional PL Common Library Step 4 Project Reuse Plan Step 1 Determine Asset Size Step 2 Determine Asset Value Step 3 Asset to Project Mapping P Process Model Step 10 Cost Benefit Step 5 PrL Asset Development Step 6 PL Deployment Plan Step 7 PL Project Costs Step 8 PL Organisation Costs Step 9 PL Environment 7
Rolls-Royce
Rolls-Royce
Recap of SPLC 2010
8
Cost = Size * Environment
Lines of Code Complexity Size
Environment
Team capability Team experience Team complexity and location Process capability Management capability Tool & IT capability Project complexity Integration complexity Development approach Development standard Development domain 9
Rolls-Royce
10 10 cost COCOMO II Size & Complexity Quality Attribute Size, Complexity, Testability Maintainability Maturity, Reusability, Portability, Variability......
SPL COCOMO cost The SPLC 2010 paper proposed that the existing cost model had to be extended with other Quality Attributes to get a better understanding of costs & benefits
Rolls-Royce
The evolution of cost models
[2004] First generation of the Rolls-Royce cost model was based on COCOMO II
The software product is represented as a single size measure (SLOC). Limited use of the architecture or characteristics of the product being developed
[2008] Second generation of the model allowed us to estimate the cost to develop and deploy individual core assets
The model estimates the cost-benefit of each core asset based on size, complexity, volatility, and difficulty to develop and deploy
[2011] To understand the relevant Quality Attributes of the core assets and the relationship between them
To understand and integrate Quality Attributes into the cost model
11
Rolls-Royce
Rolls-Royce
Selecting Quality Attributes
12
SPLC 2011
Objectives
Look at quality attributes and their impact on the cost benefit of Software Product Lines
Every factor (question) a cost model considers is another opportunity to: Refine your understanding, make trade decisions and optimise the project
Incorporate quality attributes into the Rolls-Royce Cost Model to analyze:
The cost/benefit of having core assets with certain qualities
The impact on quality attributes of design decisions
Use the cost model to derive design principles for the core asset team
For example how much variability should there be and when ius there too much or too little?
13
Rolls-Royce
Maturity
Maturity of a core asset has been defined here as the “degree to which an asset is free from further modification” .
A low maturity asset is likely to be exposed to changes and depending on when they manifest, this can lead to high levels of effort to fix defects Maturity becomes a sensitive issue for SPLs especially if products are using low maturity assets. Maturity can be estimated from process exposure and project exposure
Rolls-Royce
14
Testability
It is an important quality attribute for any type of product, and in particular for safety-critical products.
A non-SPL safety-critical product invests around 52% of its total development effort on some form of verification and validation.
In a SPL at Rolls-Royce, data shows that RELATIVELY up to 72% of a product’s overall effort will be spent in some form of V&V Testability can be estimated from the number of test cases (decision points) required to exercise the complete asset
Rolls-Royce
15
Variability
The selection of a specific variation mechanism for a core asset can have an impact on the final product development & deployment cost. The cost of variability in a core asset is calculated by multiplying the cost of deploying the asset (in a specific process) by the cost of using the different variation mechanisms 16 Future Work Need estimation tool based on both the number AND type of variations
Rolls-Royce
Reusability
The reusability of core assets largely determines the success of product line projects.
Higher reusability of core assets indicates the potential for a higher return-on-investment. This can be measured retrospectively.
Future Work Need to develop a pre-process measure of reusability
Rolls-Royce
17
Maintainability
Maintainability of core assets is of great importance. It is also important to understand the degree of maintainability that a core asset should have to be cost-effective to the business.
Measured by the effort to maintain an asset (normalised by size). Future Work Need to develop a pre-process measure of maintainability
Rolls-Royce
18
Rolls-Royce
Trade off analysis
19
SPLC 2011
Quality Attribute Trade-offs Firstly, we created a
hypotheses
diagram to show the relationship between the Quality Attributes
20
Rolls-Royce
Quality Attribute Trade-offs Then we measured the relationship between the attributes (strength and direction)
21
Rolls-Royce
Quality Attribute Trade-offs Now we can derrive information from the model
22
Rolls-Royce
?
is it better to have 2 simpler assets or 1 larger asset, to have selectable assets rather than configurable, constrain the requirements or build in variability?
Rolls-Royce
SPLC 2012?
23
Next Steps
The Rolls-Royce estimation model will need to be updated to take into account the other quality attributes selected (i.e., maturity, variability and testability) Other quality attributes will be evaluated and the model developed further
-H Variability -L -H Volatility Testability Model v.2. Without derived relationships -M L L M M H -H Maturity H H M H H H Reusability M H M M Maintainability M M Cost H H H H Benefit M H H ROI M Customer Sat.
M Project Sat.
L Business Sat.
24
Rolls-Royce
1st International Workshop on Quantitative Methods in Software Product Line Engineering (QMSPLE 2011)
QMSPLE 2011
25
Rolls-Royce