CS 510, Fall 2017 Edited & presented by Jim Alstad Software Decision Analysis Techniques Chapters 10-20 in Software Engineering Economics (SEE) Updated to MedFRS Barry Boehm, USC-CSSE CS 510, Fall 2017 Edited & presented by Jim Alstad

Topics in Software Decision Analysis Techniques Economics – The study of how people make decisions in resource-limited situations Macroeconomics (not covered in this course) Inflation, taxation, balance of payments Microeconomics Make-or-buy, pricing, how much to build Software economics decisions Make-or-buy: Software Product How many options to build? Which DP architecture to use? How much testing (prototyping, specifying) is enough? How much software to re-use? Which new features to add first? Fall 2017 © USC-CSSE

Outline of The Four Lectures (Chapters in SEE) 10-12. Context; MedFRS TPS Example; Cost-Effectiveness Analysis Models, optimization, production functions, economies of scale decision criteria 13-15. Multiple-Goal Decision Analysis I Net value, marginal analysis, present value, figures of merit. 16-18. Multiple-Goal Decision Analysis II Goals as constraints, constrained optimization, system analysis, unquantifiable goals 19-20. Dealing with Uncertainties Expected value, utility functions, statistical decision theory, value of information Fall 2017 © USC-CSSE

Master Key to Software Engineering Economics Decision Analysis Techniques All decision criteria (DCs) convertible to present $? Yes Use standard optimization, net value techniques (chapters 10, 13) outcome Is of decision highly sensitive to assumptions? (chapter 17) No Use standard constrained- optimization techniques (chapter 16) Yes Yes All Non-$ DCs expressible as constraints? Find, use less sensitive solution No Use cost-benefit (CB) decision making techniques (chapters 11, 12) All Non- $ DCs expressed as single “benefit” criterion? Yes End No Use figure of-merit techniques, CB techniques (chapter 15) Use present value techniques to convert future $ to present $ (chapter 14) Yes When $ are a mix of present and future cash flows All Non-$ DCs quantifiable? Use statistical decision theory techniques (Chapters 19,20) When some DCs involve uncertainties No Use techniques for reconciling non-quantifiable DCs (chapter 18) Fall 2017 © USC-CSSE

Chapters 10-12 Cost-Effectiveness Analysis Introduction Example: MedFRS transaction processing system (TPS) Performance models Cost-performance models Production functions: economies of scale Cost-effectiveness decision criteria Summary Fall 2017 © USC-CSSE

MedFRS TPS Context Current Ensayo IT processing capacity less than needed in crises Top performance less than 1000 transactions/second (tr/sec) Need 2000 tr/sec soon Need growth to 4000 tr/sec COTS server capability can provide over 2000 tr/sec, but mostly for business applications And can’t achieve 4000 tr/sec MedFRS considering developing or outsourcing its own server software Fall 2017 © USC-CSSE

MedFRS TPS Architecture variant on SEE architecture (Fig 10-1) … … … … … … … FRS Devices … … … MedFRS communications hub vehicles (CHVs) Ensayo Regional Server … Area Concentrators 1 N Medical DB DB Server Patient DB Fall 2017 © USC-CSSE

MedFRS TPS Concept of Operation MedFRS devices send patient trauma information to CHVs CHVs validate information and forward it to area concentrators at regional MedFRS server Usually about 10 local concentrators per area N area concentrators use MedFRS server to determine best near-term treatment Send back to device operators via CHVs Multiprocessor overhead due to resource contention, coordination Fall 2017 © USC-CSSE

COTS vs. New Development Cost Tradeoff: MedFRS TPS Build special version of MedFRS regional server systems functions To reduce COTS server software overhead, improve transaction throughput Server systems software size: 20,700 SLOC Transaction prioritization, status monitoring COTS license tradeoffs vs. number of area concentrators N Need 10 N licenses for CHVs $1K each for acquisition, $1K each for 5-year maintenance Fall 2017 © USC-CSSE

COTS/New Development Cost Tradeoff Analysis COTS $K New Development $K Software Cost to acquire 100 606 Integrate & test 100 Included Run-time licenses 10N Not applicable 5-year maintenance 10N 151 200 + 20N 757 Server 250 + 20N 250+20N Total 450 + 40N 1007 + 20N Fall 2017 © USC-CSSE

COTS/New Development Cost Tradeoff 1200 1000 Life Cycle Cost, $K 800 New 600 400 200 10 20 30 40 50 Number of Area Concentrators, N Now, we need to address the benefit tradeoffs Fall 2017 © USC-CSSE

MedFRS TPS Decision Problem 1: How Many Regional Concentrators in Server? Performance Parameters (for COTS OS) N, number of processors N = ? S, processor speed (MOPS/sec) S = 1000 P, processor overhead (MOPS/sec) P = 200 M, multiprocessor overhead factor M = 80 [overhead=M(N-1) MOPS/sec] T, transaction processing load (MOPS/TR) T = 1.0 Performance (TR/sec) MOPS/sec available for processing MOPS/TR N[S-P-M(N-1)] T E(N) = E(N) = Fall 2017 © USC-CSSE

TPS Performance, E(N) MOPS/sec available for processing MOPS/sec required E(N) = N [ 1000-200-80(N-1)] 1.0 = N [1000-200+80)-80N2 = 880N – 80N2 = 80N(11-N) dE(N) dN 160N* = 880 N* = 5.5 E(N*) = 2440 = N E(N) 1 800 2 1440 3 1920 4 2240 5 2400 6 7 11 0 = = 880 - 160N* Fall 2017 © USC-CSSE

TPS throughout: E(N) versus # of processors, N 1 800 2 1440 3 1920 4 2240 5 2400 6 7 8 1960 E(N) (tr/sec) Number of processors N Fall 2017 © USC-CSSE

TPS Decision Problem 2: Which Operating System? Option A (Accept Available OS) B (Build New OS) Cost ($k) 450 + 40N 1007 + 20N Multiprocessor overhead factor M O/H = M(N-1) 80 40 Option B: N(1000-200-40(N-1)) E(N) = = 840N-40N2 1.0 = 40N(21-N) For N = 10, E(N) = 40(10)(11) = 4400 Fall 2017 © USC-CSSE

Cost-Effectiveness Comparison: TPS Options A, B Option B E(N) = 40N(21 – N) C(N) = 1007 + 20N 5000 4000 3000 2000 1000 450 650 1207 500 1000 1500 Cost C, $K Fall 2017 © USC-CSSE

Production Function Achievable output = F (input consumed) -Assuming only technologically efficient pairs: No higher level of output achievable, using given input X Output Input PF is nonnegative PF is nondecreasing Fall 2017 © USC-CSSE

Segments of Typical Production Function Output Input High payoff Diminishing returns Investment Fall 2017 © USC-CSSE

Value of software product to organization Natural speech input Tertairy application functions Animated displays Secondary application functions User amenities Value of software product to organization Main application functions Basic application functions Operating System Data management system Investment High-payoff Diminishing returns Cost of software product Fall 2017 © USC-CSSE

Software “Gold-Plating” (Adding More Than Is Necessary) Agents with attitudes Animated displays “everything for everybody” Modularity, info. hiding Measurement, diagnostics Frequently Gold-Plating Instant response Pinpoint accuracy Unbalanced systems Usually Not Gold-Plating Humanized input Humanized output Sometimes Gold-Plating Highly generalized control, data structures Sophisticated command languages General-purpose utilities Automatic trend analysis Fall 2017 © USC-CSSE

Modular Transaction Processing System Module 1 E(2 x 3) = 2 x E(3) = 3840 vs. 2400 P11 P12 P13 1 Trans. in Processed Transaction out P21 P22 P23 2 Fall 2017 © USC-CSSE Module 2

Software Project Diseconomies of Scale SLOC Output 1 + x PM = C (KSLOC) Person – Months Input The best way to combat diseconomies of scale is to Reduce the Scale Fall 2017 © USC-CSSE

Cost-Effectiveness Decision Criteria Maximum available budget Minimum performance requirement Maximum effectiveness/cost ratio Maximum effectiveness – cost difference Return on investment (ROI) Composite alternatives Fall 2017 © USC-CSSE

Production Functions for TPS Options A, B 1207 2400 4400 X Y Q Z P E (tr/sec) 1007 650 450 Fall 2017 © USC-CSSE

Maximum Effectiveness/Cost Ratio 1200 2000 1600 800 400 2400 100 200 300 500 600 700 R L Eff/Cost = 8 K Eff/Cost = 3.69 C, $K E (tr/sec) Fall 2017 © USC-CSSE

Maximum Effectiveness-Cost Difference 1000 2000 3000 4000 5000 500 1500 2400 1750 650 1207 3193 4400 B-Build new OS A-Accept available OS Cost C, $K Throughput E(TR/sec) Fall 2017 © USC-CSSE

Return on Investment ROI E-C C Cost C ($K) 1 2 3 4 5 500 1000 1500 2.67 2.60 A B Fall 2017 © USC-CSSE

Production Function for TPS Composite Alternative 2400 4400 E C, $K E (r/sec) 450 1007 1207 650 Fall 2017 © USC-CSSE

Summary – Cost-Effectiveness Analysis Microeconomic concepts help structure, resolve software decision problems Cost-effectiveness Production functions Economies of scale C-E decision criteria No single decision criterion dominates others Each is best for some situations Need to perform sensitivity analysis: Slightly altered situation doesn’t yield bad decision Fall 2017 © USC-CSSE