Selecting COTS Products Using a Requirements-Based Approach Jaelson Castro, Carina Alves Centro de Informática Universidade Federal de Pernambuco

Motivations Development of large and complex systems Reusable components or COTS The potential benefits of using COTS are increased product reliability and stability, at shorter development time and reduced cost.

What is COTS? Commercial-Off-The-Shelf There is no agreed definition “ COTS are products: that are sold, leased or licensed to the general public; that is usually available without source code; that is supported and evolved by the vendor who returns the intellectual property rights ” Software Engineering Institute

COTS-Based Development (CBD) Large incentives from industry and academia Improved productivity and quality of software development Emphasizes the acquisition and integration of COTS components over in-house development

Main Characteristics of CBD The nature of COTS suggest new life cycle models The success of COTS-based systems largely depends on the successful selection of software products

COTS-based systems life cycle Evaluation Selection Adaptation Integration Update ?

The Evaluation Process Consists of determining the quality of a product in the context of its intended use Decision making Must accommodate uncertainty Evaluation activities can span the entire lifetime of systems

The Selection Process Critical activity for COTS-Based systems Oriented by evaluation criteria COTS candidates must satisfy user requirements Includes an accurate understanding of the features of each product

The Adaptation Process COTS are Plug and Pray Development of all necessary software adapters and enhancements to the selected COTS Component Wrapping – includes a software barrier around the components; limiting what it can do

The Integration Process Includes all development efforts required to interconnect different COTS into a single integrated system Consists of developing additional parts not supported by available products Conventional development efforts

The Update Process Like other systems, COTS-based systems need successive updates Repair an error New required functionality Acquisition of new releases

Current Problems with COTS-Based Development Products from different vendors have to be integrated and tailored to provide complete system functionality Customers have limited access to product´s internals design COTS lifecycle is determined by vendor When using COTS products, customers are placed in a situation over which they have no control

The impact of these problems can be minimized if adequate attention is paid to the process of COTS evaluation and selection

The importance of the Selection Process Includes the understanding of user requirements Careful analysis of the capabilities and limitations of each COTS candidate Assessment of products´ quality

Main Dimensions of the Selection Process

Selection Process Challenges Lack of well defined process Use of inappropriate evaluation criteria Back-box nature of COTS components Unclear system expectations Rapid rate of changes of COTS This means that any evaluation will typically have some degree of error

Related Works - *  OTSO STACE PORE Product Identification Requirements Acquisition Non-functional requirements description Product evaluation Decision making analysis OTSO (Off-The-Shelf Option)  - STACE (Social-Technical Approach to COTS Evaluation) * PORE (Procurement-Oriented Requirements Engineering)  Address fully * Deals with the issue but not fully – does not deal with the issue

The lack of a careful consideration of non-functional requirements increases the risks of COTS failure and costs of the final system

Our Contribution An approach that addresses the lack of requirements engineering methods during COTS evaluation/selection Focus on non-functional requirements analysis to assist the evaluation of COTS products

The CRE Method A systematic, repeatable and requirements-driven approach Iterative process of requirements acquisition/ modeling and product evaluation/selection

Decreasing number of candidate products CRE Iterative Process The selection is made by rejection, products that do not meet user requirements are eliminated Time Number Increasing number and detail of requirements statements Decreasing number of candidate products

CRE Main Features Goal oriented - each phase is oriented by predefined goals CRE provides templates that include some guidelines Interactive phases – The order is not rigid

The CRE Templates Template 1 Goals: Final result: Information and requirements to be acquired: Steps / sequence: Important considerations:

The Phases Identification – identify core requirements and candidates COTS products Description – Describe each product and user requirements Evaluation – Analyze products compliance with requirements Acceptance – Verify legal issues

Identification Define selection goals, include analysis of influencing factors User Requirements Application Architecture Products availability Organization infrastructure Project objectives and restrictions Metas Goals Evaluation Criteria Functional Requirements Non-functional Requirements Architecture Issues Domain Issues Vendor Guaranties Market variables Products Features Legal Issues

Identification Evaluation criteria definition COTS product searching Elicitation of critical requirements Interviews and questionnaires techniques COTS product searching Possible sources: in-house libraries, Internet, magazines, conferences, vendors. Final result - list with all COTS candidates

Description Evaluation criteria must be elaborated in detail Non-functional requirements play an important role to discriminate between similar functional products (ex. flexibility, reliability, portability) It is usually difficult to check if a product satisfies non-functional requirements Use NFR Framework to refine these requirements

Description Feedback mechanism – information exchange between requirements process and products description

Requirements Document Description Requirements document is elaborated containing (all) relevant information about stakeholders needs Requirements Document Req_ID <unique identifier> Type <functional, non-functional> Description <information about requirement> Priority <vary from 1 to 4> Contributions <can interact in synergy and conflict> Comments <additional observations>

Description CRE method provides situation rules to deal with requirements conflict IF <condition> THEN <action> If Strong_Confl [Req1,Req2] and Req1_Prio > Req2_Prio Then Deal with Req1 Else If Strong_Confl [Req1,Req2] and Req2_Prio > Req1_Prio Then Deal with Req2

Description Checklist to help the elicitation of product information Products Aspects Vendor Aspects Price Maturity Conformance with quality standards Time delivery Capacities Stability Benefits Training Constraints Reputation Version control Support quality

Evaluation Use of appropriate techniques to assist decision-making process WSM (Weighted Scoring Method) Simple but results are not accurate AHP (Analytic Hierarchy Process) Complex use, provides more confidence in decisions

WSM - Weighted Scoring Method å j=1 ( weight * scoreaj ) Scorea = Where a represents an alternative and n the number of criteria Conformance Score Priority Weight Does not meet the requirement Low 1 Meets with restrictions Medium 2 Partially meets High 3 Meets Very High 4

AHP (Analytic Hierarchy Process) Select Product Product A Product B Product C Goal Criteria Alternatives Vendor Guaranties Cost Usability Functionalities

Evaluation Perform product demonstration sessions to obtain detailed information about COTS features and limitations Obtain products compliance score with evaluation criteria Important step during decision-making process

Evaluation The cost of each COTS alternative extends the acquisition price Apply cost estimation models COCOTS proposed by B. Boehm Provides a model to estimate the associated costs of COTS integration

Acceptance Negotiation of legal issues with COTS vendors A license should minimally specify: License grant Payments to the supplier Who owns the licensed product The risks and liability each party assumes

Main Contributions An effective approach to guide the selection of COTS products An iterative process of requirements acquisition and product evaluation Including a detailed description of non-functional requirements Support for decision-making analysis

Future Work Detailed treatment of requirements prioritization and negotiation The interplay between software architecture and the selection of multiple COTS

Non-Functional Requirements (NFR) Address important issues of quality for software systems Related to constraints on system services Play critical importance during system development Have a global impact on systems Referred as “ilities”

NFRs Main Features Subjective Relative Interacting interpreted and evaluated differently by different people Relative importance may vary depending on the particular system Interacting Attempts to achieve one NFR can hurt or help the achievement of other

Non-functional requirements can be difficult to deal with, yet dealing with NFRs can be vital for the success of a software system

Non-Functional Requirements There is not a unique definition or complete list of non-functional requirements Different researchers use different terminology Previous works Bohem, 1976 Roman, 1985 Keller, 1990 Standards ISO, ANSI, IEEE

Types of Non-Functional Requirements [sommerville 92] Process requirements Product requirements External requirements Usability requirements Legal constraints Delivery requirements Reliability requirements Economic constraints Implementation requirements Safety requirements Interoperability requirements Standards requirements Efficiency requirements Performance requirements Capacity requirements

The NFR Framework Proposed by Chung, University of Toronto Systematic and global representation of NFRs Process-oriented approach Qualitative approach Represents NFRs explicitly as softgoals

Main Characteristics Softgoals - are the basic unit for representing non-functional requirements Interdependencies - state interrelationships among softgoals Methods - offer operationalization techniques Correlations - provide catalogues for inferring possible interactions

The notion of Softgoals A goal that has no clear definition Qualitative reasoning Degrees of satisfaction Interact in synergy or conflict Decomposed through AND or OR relationship AND - the softgoal is satisfied if all of its sub-goals are OR - the softgoal is satisfied if any of its sub-goals are

NFR Framework can be used to Acquire knowledge about: the particular domain functional requirements particular kinds of NFRs Identify particular NFRs for the domain Decompose NFRs Identify operationalizations (satisfice)

Using the NFR Framework (cont.) To deal with: ambiguities tradeoffs and priorities interdependencies among NFRs Select operationalizations Support decisions (design rationale) Evaluate the impact of decisions

Catalogues Present knowledge about NFRs Sources of knowledge: specialists, developers, textbooks, developer guides, etc. Provide a broad range of expertise Types of catalogues: NFR types (organize NFRs into specialized hierarchies) method (refine NFRs considering operationalizations) correlation (show implicit interdependencies)

Catalogue of some NFR types Performance NFR Types Time Space Security Confidentiality Integrity Availability Accuracy Completeness

Softgoal Interdependency Graph (SIG) Secure system AND contribution Confidentiality of system Integrity of system Availability of system OR contribution Operationalization Identification of User Authorization of User

Implicit Interdependency SIG - Softgoal Interdependency Graph Secure [system] Integrity [system] Availability Confidentiality Identification [user] Authorization User-friendly [system] Accessibility [capacities] Learnability Simplicity [interface] - negative interdependency

Dealing with Priorities Priority softgoals can be identified as: Critical – vital for the success of the system Dominant – deal with a significant part of the organization workload Make appropriate tradeoffs among softgoals

Identifying a Softgoal as a Priority SIG - Softgoal Interdependency Graph User-friendly [system] Secure [system] Confidentiality [system] Simplicity [interface] Accessibility [capacities] Learnability [user] Integrity [system] Availability [system] + - Identification [user] Authorization [user] ! Simplicity [interface] Priority Softgoal

Recording Design Rationale Design decisions should be supported by well-justified arguments Reasons can be stated for making refinements, for selecting an alternative, etc. A Claim softgoal can rationalize tradeoffs

Recording Design Rationale SIG - Softgoal Interdependency Graph Secure [system] Integrity [system] Availability Confidentiality Identification [user] Authorization User-friendly [system] Accessibility [capacities] Learnability Simplicity [interface] - Claim Softgoal + ! Claim [User authorization will not hurt system simplicity much] ++

Selecting among alternatives The refinement process continues until the possible solutions are sufficiently detailed Evaluate the impact of decisions Consider operationalizations and decide whether a chosen alternative meets a softgoal sufficiently

Evaluating the Impact of Decisions Bottom-up process Evaluation of softgoals are represented by labels (such as  and X) Positive contribution A satisficed offspring results in a satisficed parent A denied offspring results in a denied parent Negative contribution A satisficed offspring results in a denied parent A denied offspring results in a satisficed parent

Identifying a Softgoal as a Priority - SIG User-friendly [system] Secure [system] Confidentiality [system] Simplicity [interface] Accessibility [capacities] Learnability [user] Integrity [system] Availability [system] +  X -  Identification [user] Authorization [user] ++ !  Simplicity [interface] Claim [User authorization will not hurt system simplicity much]