1. Architecture Evaluation Cost Benefit Analysis Method (CBAM)

2. Overview 2

3. Motivation A set of architectural approaches/decisions/tactics/… can be applied –Add a redundant Web server? –Manage event rate towards Inventory? –Adhere to standard SQL? Resources for implementing decisions are finite –We need to choose? –Criteria? 3

4. Utility-Reponse Curves Some efforts... –Make little difference –Make a lot difference Key point –Variations of response measure 0: Useless 100:Excellent 4

5. Cost Benefit Analysis Method (CBAM) Economics-based approach to architecture decision analysis –Benefit of an architectural strategy, Bi Weighted and summed individual utility of strategy –Cost of architectural strategy, Ci –Return On Investment (ROI) for each strategy –Prioritize based on Ri May follow an ATAM evaluation –Uses a number of artefacts produced in an ATAM evaluation Business drivers Scenarios Architectural approaches 5

6. CBAM Steps 1. Collate Scenarios –Collect a set of scenarios (e.g., from ATAM) –Choose 1/3 based on importance according to business goals 2. Refine Scenarios –Focus on stimulus-response measures –Elicit worst-case, current, desired, and best- case response levels for each scenario 3. Prioritize Scenarios –100 votes for each stakeholder for scenarios –Top scenario is assigned weight 1.0 – the rest a weight relative to that 6

7. CBAM Steps 4. Assign Utility –Determine utility for each quality attribute response level for scenarios Utility = benefit of achieving the RM for QA 5. Develop Strategies –Develop strategies that address scenarios –Determine expected resulting response level 6. Determine Utility –Determine utility for strategy based on extrapolation 7

8. CBAM Steps 7. Calculate Benefit –Normalize utility increase with weights of scenarios to get benefit –Sum benefits for each scenario to get total benefit of strategy 8. Choose Strategies –Determine cost and schedule implications of each strategy –Calculate ROI as a ratio between benefit and cost –Choose strategies according to rank 9. Confirm Results –Consider alignment with business goals 8

9. Example: A POS system 9 Credits: Klaus Marius Hansen

10. 1. Collate Scenarios ScenarioScenario Description 1The barcode scanner fails; continue in degraded mode 2The inventory system fails and the failure is detected 3The POS system should be extended to handle “supermarket” domains as well as “small shop” domains 4The POS systems scans a new item; item is looked up, price updated 10

11. 2. Refine Scenarios 11 Response Goals ScenarioWorstCurrentDesiredBest 10 sec 5 min60 min 2Never 3 sec2,5 sec 32 years1 year6 months2 months 420 sec4 sec2 sec0,5 sec *) time before going from degraded mode to failure *)

12. 3. Prioritize Scenarios 12 Response Goals ScenarioVotesWorstCurrentDesiredBest 1750 sec 5 min60 min 250Never 3 sec2,5 sec 31002 years1 year6 months2 months 47520 sec4 sec2 sec0,5 sec

13. 4. Assign Utility 13 Utility Scores ScenarioVotesWorstCurrentDesiredBest 17510 60100 2500085100 3 10408090 47502090100

14. 5. Develop Strategies 14 StrategyNameScenarios Affected Current Respons e Expected Response 1Heartbeat and cache 10 sec5 min 2Never3 sec 44 sec1 sec 2Virtual machine31 year3 months 44 sec6 sec 3Concurrency44 sec2 sec

15. 6. Determine Utility 15 StrategyNameScenarios Affected Current Utility Expected Utility 1Heartbeat and cache 11060 20100 42090 2Virtual machine34095 42010 3Concurrency42085

16. 7. Calculate Benefits 16 StrategyScenario Affected Scenario Weight BenefitNormalized Benefit Total Benefit 1175503750 2501005000 47570525014000 23100555500 475-10-7504750 3475654875

17. 8. Choose Strategies 17 Strategy Cost Total Benefit Strategy ROI Strategy Rank 120014000702 2120047503,963 3404875121,861 Choose top rank, implement, loop until no more funds...

18. Results A ranking of design decisions with respect to ROI Provides support for structured decision making Side effects –Discussions of costs and benefits –Clarification of scenarios and requirements Discussion –Uncertainties from lot of sources are multiplied quite a lot in this model  18

19. Summary Economy-based approach to evaluation of architectural strategies –Architectural strategy = set of design decisions Requires –Business drivers –Precise scenarios –Architectural strategies –Cost model Result –Ranking of architectural strategies according to ROI 19