1 The ATAM A Comprehensive Method for rchitecture Evaluation & The CBAM A Quantitative Approach to Architecture Design Deci $ ion Making CSSE 377 Software Architecture & Design 2 Week 6, Day 1, including Ch (opening parts) in Bass’s book

2 Feedback on Project 4 so far Describe to implementers how you are improving the testing for tomorrow’s retests. Part 3 Intro on Analyzing Archs Main topic: Chapter 11 & 12 in SA  (Opening parts of each chapter) Today – Tomorrow – Have your implementers try the new testing process in class. Vote on Friday for final Project 4 deliverable Biweekly Quiz # 3 Then… Fall Break! Background: Red sky at morning Sailors take warning! Red sky at night Sailors’ delight.

3 Analyzing architectures Need to know why, when, and where to do this activity Tougher than design review of a program –who is left, who’s both “knowledgeable” and “objective” about the overall design? We’ll describe Bass’s methods, and how AT&T did this, for comparison Intro

4 ATAM - What it is “A Comprehensive Method for Architecture Evaluation” One of several review methods to “guess” the success of a design ahead of time This one is more in depth than some, because it also tries to help do the design, not just critique it SA Ch 11 – The ATAM ATAM = Architecture Tradeoff Analysis Method

5 Stolen from older disciplines! Here’s an “architecture review” going on in architecture. Other engineers do similar things. From SA Ch 11 – The ATAM

6 Participants in the ATAM The Evaluation Team Project Decision Makers Architecture Stakeholders –You have to decide carefully on how to include stakeholders! –What will their inclusion do to the review? Clients Users Others affected by the system SA Ch 11 – The ATAM

7 Outputs of the ATAM A concise presentation of the architecture Articulation of the business goals Quality requirements in terms of collection of scenarios Mapping of architectural decisions to quality requirements A set of identified sensitivity and tradeoff points A set of risks and non-risks A set of risk themes SA Ch 11 – The ATAM

8 Phases of the ATAM Phase 0 – Partnership and presentation Phase 1 – Evaluation Phase 2 – Evaluation (continued, with project stakeholders!) Phase 3 – Follow-up  See book for details on each of these SA Ch 11 – The ATAM

9 Phase 1 - Strategic role of QA’s Phase 1, step 5 – The QA’s are a key part of generating a Utility Tree, ranking by both importance and difficulty of implementation! H, M, or L for each, as shown: SA Ch 11 – The ATAM

10 By the way, what’s “Utility”? It’s an economics concept: –Utility is a measure of the relative satisfaction from, or desirability of, consumption of various goods and services. –Utility is an abstract concept rather than a concrete, observable quantity (like $). –Utility often suggests a “value” more than cost. E.g., in the “importance” variable on the previous slide. Probably you know this, if you had: –SL 151 Principles of Economics For more info, see, for example, SA Ch 11 – The ATAM

11 Next… Analyze architectural approaches for achieving the stand-out items in the tree. This takes a lot of time! An external team sticks around to participate in high-level design work. SA Ch 11 – The ATAM

12 In Phase 2… Create “scenarios” by brainstorming with stakeholders. –Like the ones you’ve been doing for QA’s –Only not as formal –Like, “At peak load, the system is able to complete 150 normalized transactions per second.” Prioritize scenarios by voting! Compare to utility tree done by development team’s decision makers. Analyze arch approaches to these scenarios & present results. Top scenarios (and ways to achieve them) become the design. SA Ch 11 – The ATAM

13 Case study – the Nightingale Project -- Read it yourself. -- I’ll put one question about it on the biweekly quiz! -- It’s a real system, but they changed the company info for confidentiality. -- It was a system that got the company into a big market, but it was hard to adapt to new customers. -- Alternative, tabular form of “Utility Tree,” – pp SA Ch 11 – The ATAM

14 The AT&T Arch Review Process For comparison with ATAM: –Usually a 2-day event, with a review team of outside experts and a trained leader. –A project team presented the problem and their architectural solution. The review team asked questions. Everyone wrote issues on “snow cards.” –The review team huddled, organized the cards, and decided on criticality. –They presented their findings back to the project team. –In a couple weeks, they sent a full report, with recommendations. –Projects reported savings of about 10% from this process. Ref – p. 263 of Bass’s book. SA Ch 11 – The ATAM

15 CBAM = “Cost Benefit Analysis Method” An SEI reference on this: SA Ch 12 – The CBAM Architectural goals: P = Performance A = Availability S = Security M = Modifiability etc.

16 Here’s the problem in a nutshell… SA Ch 12 – The CBAM In software development, we’re almost always confused… Why? Basically, our confusion is mostly about both “What to do” and also “How to do it.” How do you measure the “goodness” of any one feature you might develop or basic design you might try or tool you might buy? The answer… This of course is the WingTite

17 Which brings up… It’s ok to be confused in science & engineering. Who says so? Richard Feynman, for example.  For more on him, see for example, ~feynman/plenty.html Here he is, in 1959, explaining why we should probably spend a lot of money to develop microelectromechanical devices.  SA Ch 12 – The CBAM

18 Convert everything to Utility ! The issue shown here is “how much” of each quality attribute we need.  Extends the Utility Tree’s H, M, L ratings into quantities. Question – What’s the x-axis here? SA Ch 12 – The CBAM The most common case!

19 Answer – The x- axis here is adding successive scenarios, in order of priority. But, when the book gets to p. 313, it tends to interpret the x-axis more traditionally, as “cost” of creating these (vs. the y-axis of value). SA Ch 12 – The CBAM The most common case! This scenario (a) could then be interpreted to mean that as investment cost increases, the resulting benefits taper off. What you’d expect most of the time.

20 Besides utility, the other side is cost. The formulas – See p. 313! They assume you can really guess closely enough to apply them with any confidence. Calculating ROI… Implementing it  SA Ch 12 – The CBAM Your quality attribute scenarios lead to decisions about “utility.”

21 From our discussion of reliability (CSSE Wk 2 Day 4 - Availability-2.ppt, slide 8) – An example of combining economics and technical expertise: Deciding “how reliable” different parts of your system should be: Gold code – Has to be really reliable. The stuff that has to work right no matter what. Examples? Silver code – Typical “high use” code. Examples? Bronze code – We want it to test out as working correctly, but if it has problems, they will be less severe than for other parts of the system. Examples? SA Ch 12 – The CBAM

22 Case study – NASA ECS Project -- Read it yourself -- I’ll put one question about it on the biweekly quiz! -- It’s a real system – Google for it. SA Ch 12 – The CBAM

23 Today’s daily quiz… Put together a quality attribute “tree” for your project –Like Slide 9 If you were your project’s “clients,” what changes would you recommend!? You’ll get it back (to save) –Use it in next week’s enhancement to your arch doc.