Architecture Evaluation Cost Benefit Analysis Method (CBAM)

Overview 2

Motivation A set of architectural approaches/decisions/tactics/… can be applied –Add a redundant Web server? –Manage event rate towards Inventory? –Adhere to standard SQL? Resources for implementing decisions are finite –We need to choose? –Criteria? 3

Utility-Reponse Curves Some efforts... –Make little difference –Make a lot difference Key point –Variations of response measure 0: Useless 100:Excellent 4

Cost Benefit Analysis Method (CBAM) Economics-based approach to architecture decision analysis –Benefit of an architectural strategy, Bi Weighted and summed individual utility of strategy –Cost of architectural strategy, Ci –Return On Investment (ROI) for each strategy –Prioritize based on Ri May follow an ATAM evaluation –Uses a number of artefacts produced in an ATAM evaluation Business drivers Scenarios Architectural approaches 5

CBAM Steps 1. Collate Scenarios –Collect a set of scenarios (e.g., from ATAM) –Choose 1/3 based on importance according to business goals 2. Refine Scenarios –Focus on stimulus-response measures –Elicit worst-case, current, desired, and best- case response levels for each scenario 3. Prioritize Scenarios –100 votes for each stakeholder for scenarios –Top scenario is assigned weight 1.0 – the rest a weight relative to that 6

CBAM Steps 4. Assign Utility –Determine utility for each quality attribute response level for scenarios Utility = benefit of achieving the RM for QA 5. Develop Strategies –Develop strategies that address scenarios –Determine expected resulting response level 6. Determine Utility –Determine utility for strategy based on extrapolation 7

CBAM Steps 7. Calculate Benefit –Normalize utility increase with weights of scenarios to get benefit –Sum benefits for each scenario to get total benefit of strategy 8. Choose Strategies –Determine cost and schedule implications of each strategy –Calculate ROI as a ratio between benefit and cost –Choose strategies according to rank 9. Confirm Results –Consider alignment with business goals 8

Example: A POS system 9 Credits: Klaus Marius Hansen

1. Collate Scenarios ScenarioScenario Description 1The barcode scanner fails; continue in degraded mode 2The inventory system fails and the failure is detected 3The POS system should be extended to handle “supermarket” domains as well as “small shop” domains 4The POS systems scans a new item; item is looked up, price updated 10

2. Refine Scenarios 11 Response Goals ScenarioWorstCurrentDesiredBest 10 sec 5 min60 min 2Never 3 sec2,5 sec 32 years1 year6 months2 months 420 sec4 sec2 sec0,5 sec *) time before going from degraded mode to failure *)

3. Prioritize Scenarios 12 Response Goals ScenarioVotesWorstCurrentDesiredBest 1750 sec 5 min60 min 250Never 3 sec2,5 sec years1 year6 months2 months sec4 sec2 sec0,5 sec

4. Assign Utility 13 Utility Scores ScenarioVotesWorstCurrentDesiredBest

5. Develop Strategies 14 StrategyNameScenarios Affected Current Respons e Expected Response 1Heartbeat and cache 10 sec5 min 2Never3 sec 44 sec1 sec 2Virtual machine31 year3 months 44 sec6 sec 3Concurrency44 sec2 sec

6. Determine Utility 15 StrategyNameScenarios Affected Current Utility Expected Utility 1Heartbeat and cache Virtual machine Concurrency42085

7. Calculate Benefits 16 StrategyScenario Affected Scenario Weight BenefitNormalized Benefit Total Benefit

8. Choose Strategies 17 Strategy Cost Total Benefit Strategy ROI Strategy Rank , ,861 Choose top rank, implement, loop until no more funds...

Results A ranking of design decisions with respect to ROI Provides support for structured decision making Side effects –Discussions of costs and benefits –Clarification of scenarios and requirements Discussion –Uncertainties from lot of sources are multiplied quite a lot in this model  18

Summary Economy-based approach to evaluation of architectural strategies –Architectural strategy = set of design decisions Requires –Business drivers –Precise scenarios –Architectural strategies –Cost model Result –Ranking of architectural strategies according to ROI 19