Transcript Selecting COTS Products Using a Requirements

Selecting COTS Products
Using a Requirements-Based
Approach
Jaelson Castro, Carina Alves
Centro de Informática
Universidade Federal de Pernambuco
1
Motivations
 Development of large and complex
systems
 Reusable components or COTS
The potential benefits of using COTS are
increased product reliability and stability, at
shorter development time and reduced cost.
2
What is COTS?
 Commercial-Off-The-Shelf
 There is no agreed definition
 “ COTS are products:
that are sold, leased or licensed to the general public;
that is usually available without source code;
that is supported and evolved by the vendor who
returns the intellectual property rights ”
Software Engineering Institute
3
COTS-Based Development
(CBD)
 Large incentives from industry and
academia
 Improved productivity and quality of
software development
 Emphasizes the acquisition and
integration of COTS components over
in-house development
4
Main Characteristics of CBD
 The nature of COTS suggest new life
cycle models
 The success of COTS-based systems
largely depends on the successful
selection of software products
5
COTS-based systems life
cycle
Evaluation
?
Selection
Adaptation
Integration
Update
?
?
6
The Evaluation Process
 Consists of determining the quality of a
product in the context of its intended use
 Decision making
 Must accommodate uncertainty
 Evaluation activities can span the entire
lifetime of systems
7
The Selection Process
 Critical activity for COTS-Based
systems
 Oriented by evaluation criteria
 COTS candidates must satisfy user
requirements
 Includes an accurate understanding of
the features of each product
8
The Adaptation Process
 COTS are Plug and Pray
 Development of all necessary software
adapters and enhancements to the
selected COTS
 Component Wrapping – includes a
software barrier around the
components; limiting what it can do
9
The Integration Process
 Includes all development efforts
required to interconnect different COTS
into a single integrated system
 Consists of developing additional parts
not supported by available products
 Conventional development efforts
10
The Update Process
 Like other systems, COTS-based
systems need successive updates
 Repair an error
 New required functionality
 Acquisition of new releases
11
Current Problems with COTSBased Development
 Products from different vendors have to be
integrated and tailored to provide complete
system functionality
 Customers have limited access to product´s
internals design
 COTS lifecycle is determined by vendor
When using COTS products, customers are placed
in a situation over which they have no control
12
The impact of these problems can be
minimized if adequate attention is paid to the
process of COTS evaluation and selection
13
The importance of the
Selection Process
 Includes the understanding of user
requirements
 Careful analysis of the capabilities and
limitations of each COTS candidate
 Assessment of products´ quality
14
Main Dimensions of the
Selection Process
15
Selection Process Challenges





Lack of well defined process
Use of inappropriate evaluation criteria
Back-box nature of COTS components
Unclear system expectations
Rapid rate of changes of COTS
This means that any evaluation will
typically have some degree of error
16
Related Works
OTSO
(Off-The-Shelf
Option)
STACE
(SocialTechnical
Approach to
COTS
Evaluation)
PORE
(Procurement
-Oriented
Requirements
Engineering)
 Address fully
Product
Identification
Requirements
Acquisition
Nonfunctional
requirements
description
Product
evaluation
Decision
making
analysis

-
-



*
-

*


*


* Deals with the issue but not fully
– does not deal with the issue
17
The lack of a careful consideration of
non-functional requirements increases the
risks of COTS failure and costs of the
final system
18
Our Contribution
 An approach that addresses the lack of
requirements engineering methods
during COTS evaluation/selection
 Focus on non-functional requirements
analysis to assist the evaluation of COTS
products
19
The CRE Method
 A systematic, repeatable and requirementsdriven approach
 Iterative process of requirements acquisition/
modeling and product evaluation/selection
20
CRE Iterative Process
 The selection is made by rejection,
products that do not meet user
requirements are eliminated
Number
Increasing number and detail
of requirements statements
Decreasing number of
candidate products
Time
21
CRE Main Features
 Goal oriented - each phase is oriented
by predefined goals
 CRE provides templates that include
some guidelines
 Interactive phases – The order is not
rigid
22
The CRE Templates
Template 1
Goals:
Final result:
Information and requirements to be
acquired:
Steps / sequence:
Important considerations:
23
The Phases
 Identification – identify core requirements and
candidates COTS products
 Description – Describe each product and user
requirements
 Evaluation – Analyze products compliance
with requirements
 Acceptance – Verify legal issues
24
Identification
 Define selection goals, include analysis
of influencing factors
User
Requirements
Evaluation Criteria
Application
Architecture
Project objectives
and restrictions
Products
availability
Organization
infrastructure
Metas
Metas
Goals
 Functional
Requirements
 Non-functional
Requirements
 Architecture Issues
 Domain Issues
 Vendor Guaranties
 Market variables
 Products Features
25
 Legal Issues
Identification
 Evaluation criteria definition
 Elicitation of critical requirements
 Interviews and questionnaires techniques
 COTS product searching
 Possible sources: in-house libraries,
Internet, magazines, conferences, vendors.
 Final result - list with all COTS
candidates
26
Description
 Evaluation criteria must be elaborated in
detail
 Non-functional requirements play an
important role to discriminate between similar
functional products (ex. flexibility, reliability,
portability)
 It is usually difficult to check if a product
satisfies non-functional requirements
 Use NFR Framework to refine these
requirements
27
Description
 Feedback mechanism – information exchange
between requirements process and products
description
Requirements
Elicitation
<originates>
Requirements
Description
<allows>
<allows>
Products
Description
<originates>
Products
Search
28
Description
 Requirements document is elaborated
containing (all) relevant information about
stakeholders needs
Requirements Document
Req_ID <unique identifier>
Type <functional, non-functional>
Description <information about requirement>
Priority <vary from 1 to 4>
Contributions <can interact in synergy and conflict>
Comments <additional observations>
29
Description
 CRE method provides situation rules to
deal with requirements conflict
IF <condition> THEN <action>
If Strong_Confl [Req1,Req2] and Req1_Prio > Req2_Prio
Then Deal with Req1
Else If Strong_Confl [Req1,Req2] and Req2_Prio > Req1_Prio
Then Deal with Req2
30
Description
 Checklist to help the elicitation of product
information
Checklist
Products Aspects
Vendor Aspects
Price
Maturity
Conformance with quality
standards
Time delivery
Capacities
Stability
Benefits
Training
Constraints
Reputation
Version control
Support quality
31
Evaluation
 Use of appropriate techniques to assist
decision-making process
 WSM (Weighted Scoring Method)
 Simple but results are not accurate
 AHP (Analytic Hierarchy Process)
 Complex use, provides more confidence in
decisions
32
WSM - Weighted Scoring Method
n
Scorea =  ( weight * scoreaj )
j=1
Where a represents an alternative and n the number of criteria
Conformance
Score
Priority
Weight
Does not meet
the requirement
0
Low
1
Meets with
restrictions
1
Medium
2
Partially meets
2
High
3
Meets
3
Very High
4
33
AHP (Analytic Hierarchy Process)
Goal
Criteria
Alternatives
Cost
Select Product
Vendor
Guaranties
Product A
Functionalities
Product B
Usability
Product C
34
Evaluation
 Perform product demonstration
sessions to obtain detailed information
about COTS features and limitations
 Obtain products compliance score with
evaluation criteria
 Important step during decision-making
process
35
Evaluation
 The cost of each COTS alternative
extends the acquisition price
 Apply cost estimation models
 COCOTS proposed by B. Boehm
 Provides a model to estimate the
associated costs of COTS integration
36
Acceptance
 Negotiation of legal issues with COTS
vendors
 A license should minimally specify:




License grant
Payments to the supplier
Who owns the licensed product
The risks and liability each party assumes
37
Main Contributions
 An effective approach to guide the
selection of COTS products
 An iterative process of requirements
acquisition and product evaluation
 Including a detailed description of nonfunctional requirements
 Support for decision-making analysis
38
Future Work
 Detailed treatment of requirements
prioritization and negotiation
 The interplay between software
architecture and the selection of
multiple COTS
39
Non-Functional Requirements
(NFR)
 Address important issues of quality for
software systems
 Related to constraints on system
services
 Play critical importance during system
development
 Have a global impact on systems
 Referred as “ilities”
40
NFRs Main Features
 Subjective
 interpreted and evaluated differently by different
people
 Relative
 importance may vary depending on the particular
system
 Interacting
 Attempts to achieve one NFR can hurt or help the
achievement of other
41
Non-functional requirements can be
difficult to deal with, yet dealing with
NFRs can be vital for the success
of a software system
42
Non-Functional Requirements
 There is not a unique definition or complete
list of non-functional requirements
 Different researchers use different
terminology
 Previous works




Bohem, 1976
Roman, 1985
Keller, 1990
Standards ISO, ANSI, IEEE
43
Types of Non-Functional
Requirements [sommerville 92]
Non-functional requirements
Process requirements
Delivery requirements
Implementation requirements
Standards requirements
Product requirements
External requirements
Usability requirements
Legal constraints
Reliability requirements
Economic constraints
Safety requirements
Interoperability requirements
Efficiency requirements
Performance requirements
Capacity requirements
44
The NFR Framework
 Proposed by Chung, University of Toronto
 Systematic and global representation of
NFRs
 Process-oriented approach
 Qualitative approach
 Represents NFRs explicitly as softgoals
45
Main Characteristics
 Softgoals - are the basic unit for representing
non-functional requirements
 Interdependencies - state interrelationships
among softgoals
 Methods - offer operationalization techniques
 Correlations - provide catalogues for inferring
possible interactions
46
The notion of Softgoals
 A goal that has no clear definition
 Qualitative reasoning
 Degrees of satisfaction
 Interact in synergy or conflict
 Decomposed through AND or OR relationship
 AND - the softgoal is satisfied if all of its sub-goals are
 OR - the softgoal is satisfied if any of its sub-goals are
47
NFR Framework can be used to
 Acquire knowledge about:
 the particular domain
 functional requirements
 particular kinds of NFRs
 Identify particular NFRs for the domain
 Decompose NFRs
 Identify operationalizations (satisfice)
48
Using the NFR Framework
(cont.)
 To deal with:
 ambiguities
 tradeoffs and priorities
 interdependencies among NFRs
 Select operationalizations
 Support decisions (design rationale)
 Evaluate the impact of decisions
49
Catalogues
 Present knowledge about NFRs
 Sources of knowledge: specialists, developers,
textbooks, developer guides, etc.
 Provide a broad range of expertise
 Types of catalogues:
 NFR types (organize NFRs into specialized
hierarchies)
 method (refine NFRs considering operationalizations)
 correlation (show implicit interdependencies)
50
Catalogue of some NFR types
NFR Types
Performance
Space
Time
Security
Availability Integrity Confidentiality
Accuracy
Completeness
51
Softgoal Interdependency
Graph (SIG)
Secure system
AND contribution
Confidentiality of
system
Integrity of
system
Availability of
system
OR contribution
Operationalization
Identification of
User
Authorization of
User
52
Implicit Interdependency SIG Softgoal Interdependency Graph
Secure [system]
Integrity
[system]
Availability
[system]
User-friendly [system]
Confidentiality
[system]
Accessibility Learnability
[capacities] [user]
Identification Authorization
[user]
[user]
Simplicity
[interface]
negative
interdependency
53
Dealing with Priorities
 Priority softgoals can be identified as:
 Critical – vital for the success of the system
 Dominant – deal with a significant part of the
organization workload
 Make appropriate tradeoffs among
softgoals
54
Identifying a Softgoal as a
Priority SIG - Softgoal
Interdependency Graph
Secure [system]
Integrity
[system]
Availability
[system]
User-friendly [system]
Confidentiality
[system]
Accessibility Learnability
[capacities] [user]
Identification Authorization
[user]
[user]
Simplicity
[interface]
+
Priority Softgoal
! Simplicity
[interface]
55
Recording Design Rationale
 Design decisions should be supported by
well-justified arguments
 Reasons can be stated for making
refinements, for selecting an alternative,
etc.
 A Claim softgoal can rationalize tradeoffs
56
Recording Design Rationale
SIG - Softgoal Interdependency
Graph
Secure [system]
Integrity
[system]
Availability
[system]
User-friendly [system]
Confidentiality
[system]
Accessibility Learnability
[capacities] [user]
Identification Authorization
[user]
[user]
++
+
-
Claim [User authorization
Claim Softgoal
Simplicity
[interface]
! Simplicity
[interface]
will not hurt system simplicity
much]
57
Selecting among alternatives
 The refinement process continues until the
possible solutions are sufficiently detailed
 Evaluate the impact of decisions
 Consider operationalizations and decide
whether a chosen alternative meets a
softgoal sufficiently
58
Evaluating the Impact of
Decisions
 Bottom-up process
 Evaluation of softgoals are represented by
labels (such as  and X)
 Positive contribution
 A satisficed offspring results in a satisficed parent
 A denied offspring results in a denied parent
 Negative contribution
 A satisficed offspring results in a denied parent
 A denied offspring results in a satisficed parent
59
Identifying a Softgoal as a
Priority - SIG
User-friendly [system]
Secure [system]
Integrity
[system]
Availability
[system]
Confidentiality
[system]
Accessibility Learnability
[capacities] [user]

X
Identification Authorization
[user]
[user]
++
-

Claim [User authorization
Simplicity
[interface]
+

! Simplicity
[interface]
will not hurt system simplicity
much]
60