Software Architecture Risk Assessment (SARA) Tool Khader Basha Shaik Problem Report Defense Master of Science in Computer Science Lane Department of Computer Science and Electrical Engineering, West Virginia University This work is funded in part by grants to West Virginia University Research Corp. from the National Science Foundation (NSF-ITR) Program, and from the NASA Office of Safety and Mission Assurance (OSMA) through the NASA Independent Verification and Validation (IV&V) Facility, Fairmont, West Virginia.

Outline Introduction Previous and Related work Problem Statement Objectives Maintainability Risk Product Line Architecture Architecture of SARA Tool Proposed Approach Testing Conclusion and Future Work Demo of SARA Tool Introducing Web Based SARA Tool

Introduction Risk assessment helps projects to avoid unpredicted failures and catastrophic problems. Also, it largely prevents wrong allocation of resources. According to NASA-STD-8719.13A standard, risk is a function of the anticipated frequency of occurrence of an undesired event, the potential severity of resulting consequences, and the uncertainties associated with the frequency and severity. In this research, we present a tool that support architectural level model-based risk assessment, which includes Maintainability based risk Reliability based risk and Requirements based risk. Software architecture explicates the structure of the system in terms of components and interactions among them to accomplish the desired requirements. The architecture quantitative analysis should reflect its pertinent quality attributes and help us to predict the quality of the software products instantiated from it.

Introduction (contd..) SARA Tool extends our earlier Architectural-level Risk Assessment Tool (ARAT) by providing support for more architectural models and different perspective of risk assessment other than reliability- based and requirement based risk. It is built on the maintainability-based risk assessment methodology developed by Walid Abdelmoez and described in his Ph.D. Dissertation as part of the Software Architecture Risk Assessment Project funded by NASA.

Previous and Related work SARA Tool developed in this research is a major extension of an earlier tool called Architecture-level Risk Assessment Tool (ARAT) ARAT estimates the distribution of the scenario, use case and system risk factors on different severity classes which allow us to make a list of critical scenarios in each use case, as well as a list of critical use cases in the system. ARAT supports only Reliability and Requirement Risk.

Previous and Related work (contd..) Software Architecture Analysis Method (SAAM) and Architecture Trade-off Analysis Method (ATAM) developed at the Software Engineering Institute (SEI) at Carnegie-Mellon University (CMU). In both above approaches, the assessment is based on qualitative measures and the experience of the analyst. SDMetrics Tool: It analyzes the structural properties of UML designs. Use object-oriented measures of design size, coupling, and complexity. Doesn’t support Risk Analysis and Product Line Architectures.

Problem Statement The main focus of this research is to develop tool support for quantitative risk assessment of software architectures. The problem addressed in this report is to further develop and extend the Software Architecture Risk Assessment (SARA) tool by providing support for maintainability-based risk assessment and support for the analysis of product line architectures. This tool shall support quantitative analysis that complements the methods developed by the Software Engineering Institute at Carnegie-Melon University (CMU) on the qualitative assessment of software architectures.

Objectives The main objectives of this tool are listed below: To Design, Develop and Implement the tool for different types of Software Architecture-level metrics and risk Assessment. To extract the data from the design diagrams (class diagrams, sequence diagrams and state chart diagrams) by accepting the input files in XMI and .txt format. To implement algorithms, estimating metrics (change propagation and size of change metrics) and risk factors (corrective and adaptive maintainability risk factors) on StarUML, UML RT, and Product Line Architectures based on the methodology proposed.

Objectives (contd..) Complement the ATAM process by providing the quantitative analysis of the product and help to track the quality of software architectures. Ability to identify critical components and scenarios based on estimated their risk factors. Providing high flexibility and extensibility, so that the tool can support other risk assessment perspectives such as performance-based risk and other architecture metrics, and other input formats. Portability and scalability.

Maintainability Risk In accordance with NASA-STD-8719 standard, we define maintainability-based risk is as a combination of two factors: the probability performing maintenance tasks and the impact of performing these tasks . Accordingly, Maintainability-based Risk for a component is defined as: Probability of changing the component* Maintenance impact of changing the component.

Maintainability Risk (contd..) Maintainability Risk Methodology

Product Line Architecture A software product line architecture is the encompassing architecture for the family of products that make up the product line and specifies what is common, and what variations are explicitly allowed among them. Variabilities are characteristics that may vary from a product to another. The main challenge in the context of software Product Lines (PL) approach is to model and implement these variabilities. One of the main concepts behind Product line architecture is software reuse through managing variability between the products in the PL.

Product Line Architecture (contd..) Stereotypes: Kernel. Kernel in PLs represents the mandatory features for the PL members. i.e.: they cannot be omitted in products. The stereotype <<kernel>> is used to specify Kernel in UML class diagrams. Optionality. Optionality in PLs means that some features are optional for the PL members. i.e.: they can be omitted in some products. The stereotype <<optional>> is used to specify optionality in UML class diagrams. Variation. We model variation point using UML inheritance and stereotypes: each variation point will be defined by an abstract class and a set of subclasses. The abstract class will be defined with the stereotype <<variation>> and each subclass will be stereotyped<<variant>>. Example of PLA for Micro-oven model in shown in the next side

Class Diagram of Micro-oven PLA model

Architecture of SARA Tool

Import Architecture Desc file Control Flow diagram for Maintainability risk Calculation Module Data Parser Module Database access Module Storing data into database Retrieving data from database Calculation Module SARA Tool Database Display Module Change Propagation and Size of Change Calculation Initial Change Probability Calculation (Optional) Maintainability Risk Calculation Import Architecture Desc file

Process Flow Chart of PLA module in SARA Tool Is the architecture quality of the target software? satisfied the specification requirement? Build StarUML PLA model of target System SARA Tool Import modal data (XMI) into SARA Tool Create Instances (Product Lines) Store the Product Lines in Repository Size of Change Analysis Maintainability Risk Preprocess each Product Line and save data is Repository Change Propagation Analysis Identify optimal Product Line Process Flow Chart of PLA module in SARA Tool No

Use case diagram of maintainability-based risk functionality of the SARA tool

Class Diagram of SARA Tool

Proposed Approach Models supported by the Tool For Maintainability Risk StarUML UMLRT PLA For Reliability Risk For Requirement Risk Extract Architectural Description of Models Modules in SARA Tool Import Module Metrics Module Risk Module User Interface Module

Proposed Approach (contd..) Import Module Architecture description files to be imported to the SARA Tool workspace The input formats used by the SARA Tool are: For StarUML model : XMI and .txt Files For UMLRT model : .txt Files Unlike UMLRT, StarUML is an open source UML/MDA Tool. Any user can just develop his Architecture Model and use SARA Tool for Risk Assessment.

Proposed Approach (contd..) Metrics module in SARA Tool The transaction methods in Java call various algorithms to compute metrics. StarUML model Change propagation Size of change Initial Change Probabilities Error propagation Size Coupling Complexity UMLRT model

Proposed Approach (contd..) Risk module in SARA Tool Maintainability Risk Reliability Risk Requirement Risk User Interface module in SARA Tool Swings are used to show the outputs to the user. Results are shown to the analyst in both table and bar chart format. A third party tool- Espress Chart has been used to display results in bar chart format

Testing Testing was done on the following modules with different Case Studies Import Module Metrics Module Risk Module User Interface Module Case Studies Used (Khader, please give the source of each) CM1 Model (From the MDP project, NASA IV&V) Pacemaker Model Game of life Model Micro-oven PLA Model

Case Study:CM-1: Class Diagram from StarUML

Some of the Sequence Diagrams of CM-1 Model :

Some of the Sequence Diagrams of CM-1 Model : (Khder there is no need for this slide, you already showed examples of sequence diagrams in previous slide) HK TIMESYNC

------Let us go to Tool demo now XMI file exported from StarUML ------Let us go to Tool demo now

Conclusion and Future Work Software Architecture Risk Assessment (SARA) Tool is designed and implemented as a tool for computing and analyzing architectural level risk factors like Maintainability Risk, Reliability Risk and Requirement Risk. Efforts are made in implementing the methodology on Product Line Architectures for analyzing all possible instances and coming out with better architecture with minimum risk (Khader add a bullet here on the case studies you used to demonstrate the utility of te tool)

Conclusion and Future Work (contd..) Among our venues of further research, we are considering To add other risk assessment perspectives like performance-based risk. To support reliability and requirement risk for StarUML model( do we have reliability risk supported already in the current version?). To support more input formats for the tool and test with multiple case studies. To support evaluation of Product Line Architectures with multiple case studies. To make SARA Tool a complete version of web based open source tool. Support Data Mining analysis process with statistical data and risk factors.

DEMO of the SARA Tool…… Creating a new Model Import file(XML) for Maintainability Risk: File imported: Case Study-1: Class Diagram of CM-1 Model: Sequence Diagrams of CM-1 Model DEMO of the SARA Tool…… For Size of Change: Results for SC For ICP: Two Options: 1.Import File 2.Choose Options: calculating ICP both Corrective ICP and Adaptive ICP Now Calculating Maintainability Risk: Results for Corrective MR Results for Adaptive MR:

Thank You