Software Architecture Prof.Dr.ir. F. Gielen Architecture Trade-off Analysis Method ATAMSM Vakgroep Informatietechnologie – IBCN

Context for Evaluating Architectures. Before we consider specific methods for evaluating software architectures we will consider some context : SA will tell you important properties of a system even before it exists Architects know the effects (or can estimate) of design (architectural) decisions Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Why Evaluate Architectures? All design involves trade-off’s. A software architecture is the earliest life-cycle artifact that embodies significant design decisions and trade-offs. The earlier that risks are identified, the earlier that mitigation strategies can be developed potentially avoid the risks altogether. The earlier that defects are found, the less it costs to remove them. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Cost / Benefits of Evaluation of SA Staff time that is required of the participants AT&T (Lucent) performed ~300 full scale SA Evaluations Average cost 70 staff-days ATAM method averages 36 staff-days + stakeholders Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Cost / Benefits of Evaluation of SA Financial Benefits AT&T estimated that the 300 Evaluations saved 10% in project costs Anecdotal information of evaluations savings Company avoided multi-million dollar purchase when evaluation showed inadequacy Anecdotes on evaluations that should have been done Estimate of rewrite of system would take two years; it took three rewrites and seven years Large engineering DB system; design decisions prevented integration testing. Cancelled after $20,000,000 invested Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Dimensions of complexity Higher technical complexity - Embedded, real-time, distributed, fault-tolerant - Custom, unprecedented, architecture reengineering - High performance An average software project: - 5-10 people - 10-15 month duration - 3-5 external interfaces - Some unknowns & risks Defense Weapon System Telecom Switch National Air Traffic Control System Commercial Compiler Embedded Automotive Software Large-Scale Organization/Entity Simulation Lower management complexity - Small scale - Informal - Single stakeholder - “Products” CASE Tool Higher management complexity - Large scale - Contractual - Many stake holders - “Projects” Small Scientific Simulation IS Application Distributed Objects (Order Entry) Enterprise IS (Family of IS Applications) Defense MIS System IS Application GUI/RDB (Order Entry) Business Spreadsheet Lower technical complexity - Mostly 4GL, or component-based - Application reengineering - Interactive performance Walker Royce, Rational Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Non-Financial Benefits Forced Preparation for the review: Presenters will focus on clarity Captured Rationale Evaluation will focus on questions Answering yields “explanations of design decisions” Useful throughout the software life cycle After the fact capturing rationale much more difficult “Why was that done?” Validation of requirements Discussion of how well a requirement is met opens discussion Some requirements easy to demand, hard to satisfy Improved architectures Iterative improvement Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Planned or Unplanned Evaluations Normal part of software life cycle Built into the project’s work plans, budget and schedule Scheduled right after completion of SA (or ???) Planned evaluations are “proactive” and “team-building” Unplanned As a result of some serious problem “mid-course correction” Challenge to technical authority of Team Unplanned evaluations are “reactive” and “tension-filled” Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Properties of Successful Evaluations Clearly articulated goals and requirements Controlled Scope Cost-effectiveness Key personnel availability Competent Evaluation Team Managed Expectations Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Outputs of an Architecture Evaluation Prioritized statement of quality attribute requirements Mapping of approaches to quality attributes Risks and non-risks Risks are potentially problematic architectural decisions Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Documenting risks and non-risks consists of: An architectural decision (that has not been made). A specific quality attribute response being addressed by that decision. A rationale for the positive or negative effect that the decision has on satisfying the quality attribute. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Technological Risk Levels Amount of uncertainty 1. DISASTER Believed to have never been done by anyone, anywhere in the world! 2. VERY MAJOR Done by somebody, somewhere, once 3. MAJOR Done elsewhere a few times, but never yet in-house 4. MEDIUM Done commonly or by people who now are in-house 5. MINOR Done often, including in-house Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

QA response and consequence: Example of a Risk Decision to be made: The rules for writing business logic modules in the second tier of your three tier client-server system are not clearly articulated. QA response and consequence: This could result in replication of functionality thereby compromising modifiability of the third tier. Rationale for the negative effect: Unarticulated rules for writing business logic can result in unintended and undesired coupling of components Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Architecture Tradeoff Analysis Method (ATAM) “We evaluate the services that anyone renders to us according to the value he puts on them, not according to the value they have for us.” --- Friedrich Nietzche Evaluating an architecture is a complicated undertaking. Large system  large architecture ABC + Nietzche’s quote yields: A software system is intended to support business goals and Evaluation needs to make connections between goals and design decisions Multiple stakeholders  acquiring all perspectives requires careful management Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Architecture Tradeoff Analysis Method ATAM is an architecture evaluation method that: focuses on multiple quality attributes illuminates points in the architecture where quality attribute trade-offs occur. generates a context for ongoing quantitative analysis. utilises an architecture’s vested stakeholders as authorities on the quality attribute goals. The purpose of ATAM is: to assess the consequences of architectural decisions in light of quality attribute requirements and business goals. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Figure taken from http://www.sei.cmu.edu/ata/ata_method.html Conceptual Flow ATAM Figure taken from http://www.sei.cmu.edu/ata/ata_method.html Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Participants in ATAM Evaluation Team Project Decision makers group external to the project 3-5 persons need to be recognized as unbiased outsiders Project Decision makers the architect, customer, project manager people empowered to speak for the development project with the authority to make decisions Architecture stakeholders including developers, testers, field engineers…, users, customers builders of systems interacting with this one. Find at least 10 of them ! Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Evaluation Team Roles and Responsibilities Team leader Sets up evaluation (set evaluation contract) forms team interfaces with client Oversees the writing of the evaluation report Evaluation leader Runs the evaluation Facilitates elicitation of scenarios Administers selection and prioritization of scenarios process Facilitates evaluation of scenarios against architecture Scenario scribe Records scenarios on a flip chart Captures (and insists on) agreed wording Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Evaluation Team Roles and Responsibilities Proceedings scribe Captures proceedings in electronic form Raw scenarios with motivating issues Resolution of each scenario when applied to the architecture Generates a list of adopted scenaios Timekeeper Helps evaluation leader stay on schedule Controls the time devoted to each scenario Questioner raises issues of architectural interest stakeholders may not have thought of Process observer Keeps notes on how the evaluation process could be improved Process enforcer helps the evaluation leader stay “on process” Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

A concise presentation of the architecture Outputs of the ATAM (1/2) A concise presentation of the architecture Frequently there is too much information. ATAM will force a “one hour” presentation of the architecture forcing it to be concise and clear. Articulation of Business Goals Quality requirements in terms of collections of scenarios Mapping of architectural decisions to quality attribute requirements A set sensitivity and tradeoff points Eg. A backup database positively affects reliability (sensitivity point with respect to reliability) However it negatively affects performance (tradeoff) Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

A set of risks and non-risks Outputs of the ATAM (2/2) A set of risks and non-risks A risk is an architectural decision that may lead to undesirable consequences with respect to a stated quality attribute requirement A non-risk is an architectural decision that, after analysis, is deemed to be safe Identified risks can form the basis of a “architectural risk mitigation” plan A set of risk themes Examine the collection of risks produced looking for themes that are the result of systematic weaknesses in the architecture Other outputs – secondary More documentation, rationale, sense of community between stakeholders, architect, … Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

ATAM Phases Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Team leadership/ key project decision makers Over a few weeks ATAM planning Phase Activity Participants Duration Preparation Team leadership/ key project decision makers Over a few weeks 1 Evaluation Evaluation team and project decision makers 1 day + hiatus of 2 to 3 weeks 2 Evaluation with Stakeholders Ditto + stakeholders 2 days 3 Follow-up Evaluation team and client 1 week Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Issues that must be resolved in Phase 0: Phase 0: Partnership Client : someone who can exercise control of the project whose architecture is being evaluated Perhaps a manger Perhaps someone in an organization considering a purchase Issues that must be resolved in Phase 0: The client must understand the evaluation method and process. (train the customer) The client should describe the system and architecture. A “go/no-go” decision is made here by the evaluation team leader. Statement of work is negotiated. Issues of proprietary information resolved. IPR & NDA Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Forming the evaluation team Holding an evaluation kickoff meeting Phase 0: Preparation Forming the evaluation team Holding an evaluation kickoff meeting Assignment of roles Roles not necessarily one-to-one The minimum size evaluation team is 4 (3+1) One person can be process observer, timekeeper and questioner Team leader’s responsibilities are mostly “outside” the evaluation. He can double up. (Often the evaluation leader.) Questioners should be chosen to represent the spectrum of expertise in performance, modifiability, … Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Phase 1: Activities in Addition to the steps Organizational meeting of evaluation team and key project personnel Form schedule The right people attend the meetings They are prepared (know what is expected of them) They have the right attitude Besides carrying out the steps the team needs to gather as much information as possible to determine Whether the remainder of the evaluation is feasible Whether more architectural documentation is required Which stakeholders should be present for phase 2 Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

ATAM steps: phase 1 Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Evaluation leader presents the ATAM to all participants Step 1: Present the ATAM Evaluation leader presents the ATAM to all participants To explain the process To answer questions To set the context and expectations for the process Using standard presentation discuss ATAM steps in brief and outputs of the evaluation Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 2: Present Business Drivers Everyone needs to understand the primary business drivers motivating/guiding the development of the system: NABC Need: What is the important customer and market need? Approach: What is your unique approach for addressing this need ? Benefits: What are the benefits (per cost) from this approach ? Competition: How are those benefits superior to the competition and the alternatives ? Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 2: Example – Video on demand Need: Movie rental is a 500 Mio Euro business. The part people dislike is to return tapes and late fees. Approach: We will provide VOD via the cable system with access to all the tittles of MovieMAX. The system uses existing channels and hardware. Customers need no new investments and pay the same price for a movie as in the rental shop. Benefits: End-user: no need to return movies; no more late fees. Same functions as with a DVD player: fast forward, trailers, ..etc. Customer: Higher revenue per movie with higher margin; 20% market share expected. Competition: competition : we have patented the distribution and VCR like features for VOD alternatives : on-line rentals have higher handling costs (0.75 Euro per movie). Sending the tape back is as inconvenient as returning it. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 2: Present Business Drivers in practice A project decision maker presents a system overview from the business perspective The NABC. The system’s core functionality Relevant constraints: technical, managerial, economic, or political The architectural drivers the quality attributes that shape the architecture Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 3: Present the Architecture (1/2) The lead architect makes the architecture presentation at the appropriate level of detail (~20 slides; 60 minutes) Architectural drivers and existing standards/models/approaches for meeting these(2-3 slides) Important Architectural Information (4-8 slides) Context diagram Module or layer view Component and connector view Deployment view Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 3: Present the Architecture (cont) Architectural approaches, patterns, or tactics employed (3-6 slides) Use of COTS (commercial off the shelf) products Trace of 1-3 most important use cases scenarios Trace of 1-3 most important change scenarios Architectural issues/risks with respect to meeting the driving architectural requirements Glossary Should cover technical constraints such as operating system, hardware and middleware Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 4: Identify Architectural Approaches The ATAM analyzes an architecture by focusing on its architectural approaches. Styles and patterns are captured here by the evaluation team. The evaluation team should explicitly asked the architect to name the patterns, styles and approaches. The list is cataloged and documented. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 5: Generate Quality Attribute Utility Tree Quality attributes are articulated in detail: Evaluation team works with project decision makers to elicit the most important quality attributes as scenarios. Root: Utility is an expression of the benefit to the user or customer. Level 1: Quality attributes are the second level. The initial set comes from the business drivers (Step 2). Level 2: Attribute refinements. Level 3: Short scenarios (stimulus, context, response) Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Quality Attribute Utility Tree Example Figure taken from Linda Northrop’s presentation for CSEE&T 2004, March 2004 Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 5: Generate Quality Attribute Utility Tree Scenario prioritisation Project decision makers give a relative ranking based on importance (business goal). Architect gives a ranking based on difficulty to satisfy the scenario. (technical goal) Each scenario has a (H,H), (H,M),…(L,M),(L,L) priority assigned. Based on this a selection of scenarios is made for the purpose of evaluation. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 6:Analyze Architectural Approaches The evaluation team examines the highest priority scenarios one at a time and the architect is asked how the architecture supports each one. architectural decisions sensitivity points tradeoff risks, Key is to have sufficient information to link architectural decisions to quality requirements Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Example of Scenario Analysis :JBoss Scenario U1: Deploy a modified library to the JBoss application server within five minutes. The Results of Analysis of Scenario U1 D1: Using the unified class loaders to facilitate hot redeployment. D2: Using the deployment ScannerThread to periodically check for changes in the deployment directory. D3: The deployment unit is not locked during the deployment. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

An Example of Scenario Analysis T1: Tradeoff between reusability and the support of multiple versions of components. The unified class loaders support the reuse of the components among different deployment packages at run-time, but they limit the support for the multiple versions of the components . S1: If the interval for checking is too long, the total deployment time will be longer. If the interval for checking is too short, CPU resources will be wasted. R1: If the copy takes too long time to complete (large library or remote copy), JBoss may deploy an incomplete library. R2: If the library is changed during the deployment, JBoss may deploy a library in an inconsistent state. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

ATAM steps: phase 2 with all stakeholders Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 7: Brainstorm and Prioritize Scenarios The Hiatus Evaluation team distills what has been learned and informally contacts architecture for more information where needed Scenario brainstorming Take input from a large group Compare this list with the utility tree of phase 1 prioritize the stakeholder scenario list Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 8: Analyze Architectural Approaches Similar to step 6 Applied to the highest ranking scenarios of step 7 Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Step 9: Present results Present the outputs Identify risk themes focus on utility, risks, sensitivity and tradeoff Identify risk themes group risks into categories identify impacted business drivers define mitigation plan action before the risk becomes a problem Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

Figure taken from http://www.sei.cmu.edu/ata/ata_method.html Summary: ATAM Flow Vakgroep Informatietechnologie – Onderzoeksgroep IBCN Figure taken from http://www.sei.cmu.edu/ata/ata_method.html

Summary: When to Use the ATAM ? The ATAM can be used throughout the life cycle when there is a software architecture to evaluate. ATAM can be used: after an architecture has been specified but there is little or no code to evaluate architectural alternatives to evaluate the architecture of an existing system Vakgroep Informatietechnologie – Onderzoeksgroep IBCN

The result is improved architectures. ATAM benefits The benefits of performing ATAM evaluations include: clarified quality attribute requirements improved architecture documentation basis for architectural decisions identification of risks early in the life cycle increased communication among stakeholders The result is improved architectures. Vakgroep Informatietechnologie – Onderzoeksgroep IBCN