1. University of Southern California Center for Systems and Software Engineering Value-Based Software Engineering CS 577a Software Engineering I Barry Boehm Fall 2012

2. University of Southern California Center for Systems and Software Engineering ©USC-CSSE2 Why Software Projects Fail

3. University of Southern California Center for Systems and Software Engineering ©USC-CSSE3 Software Testing Business Case Vendor proposition –Our test data generator will cut your test costs in half –We’ll provide it to you for 30% of your test costs –After you run all your tests for 50% of your original cost, you are 20% ahead Any concerns with vendor proposition?

4. University of Southern California Center for Systems and Software Engineering ©USC-CSSE4 Software Testing Business Case Vendor proposition –Our test data generator will cut your test costs in half –We’ll provide it to you for 30% of your test costs –After you run all your tests for 50% of your original cost, you are 20% ahead Any concerns with vendor proposition? –Test data generator is value-neutral* –Every test case, defect is equally important –Usually, 20% of test cases cover 80% of business case * As are most current software engineering techniques

5. University of Southern California Center for Systems and Software Engineering ©USC-CSSE5 20% of Features Provide 80% of Value: Focus Testing on These (Bullock, 2000) % of Value for Correct Customer Billing Customer Type 100 80 60 40 20 51015 Automated test generation tool - all tests have equal value

6. University of Southern California Center for Systems and Software Engineering ©USC-CSSE6 Value-Based Testing Provides More Net Value Net Value NV 60 40 20 0 -20 20401006080 -40 (100, 20) Percent of tests run Test Data Generator Value-Based Testing (30, 58) % TestsTest Data GeneratorValue-Based Testing CostValueNVCostValueNV 0300-30000 103510-25105040 204020-20207555 304530-15308858 405040-10409454 ….….….….….….….….….….….….….…. 10080100+20100 0

7. University of Southern California Center for Systems and Software Engineering ©USC-CSSE7 Motivation for Value-Based SE Current SE methods are basically value-neutral –Every requirement, use case, object, test case, and defect is equally important –Object oriented development is a logic exercise –“Earned Value” Systems don’t track business value –Separation of concerns: SE’s job is to turn requirements into verified code –Ethical concerns separated from daily practices Value-neutral SE methods are increasingly risky –Software decisions increasingly drive system value –Corporate adaptability to change achieved via software decisions –System value-domain problems are the chief sources of software project failures

8. University of Southern California Center for Systems and Software Engineering ©USC-CSSE8 20% of Fires Cause 80% of Property Loss: Focus Fire Dispatching on These? 100 80 60 40 20 % of Property Loss % of Fires 20406080100

9. University of Southern California Center for Systems and Software Engineering ©USC-CSSE9 Penumbra Negotiation Example: Fire Dispatching System Dispatch to minimize value of property loss –Neglect safety, least-advantaged property owners English-only dispatcher service –Neglect least-advantaged immigrants Minimal recordkeeping –Reduced accountability Tight budget; design for nominal case –Neglect reliability, safety, crisis performance

10. University of Southern California Center for Systems and Software Engineering ©USC-CSSE10 Conclusions So Far Value considerations are software success- critical “Success” is a function of key stakeholder values –Risky to exclude key stakeholders Values vary by stakeholder role Non-monetary values are important –Fairness, customer satisfaction, trust

11. University of Southern California Center for Systems and Software Engineering ©USC-CSSE11 Key Definitions Value (from Latin “valere” – to be worth 1.A fair or equivalent in goods, services, or money 2.The monetary worth of something 3.Relative worth, utility or importance Software validation (also from Latin “valere”) –Validation: Are we building the right product? –Verification: Are we building the product right?

12. University of Southern California Center for Systems and Software Engineering ©USC-CSSE12 7 Key Elements of VBSE 1.Benefits Realization Analysis 2.Stakeholders’ Value Proposition Elicitation and Reconciliation 3.Business Case Analysis 4.Continuous Risk and Opportunity Management 5.Concurrent System and Software Engineering 6.Value-Based Monitoring and Control 7.Change as Opportunity

13. University of Southern California Center for Systems and Software Engineering ©USC-CSSE13 - Stakeholder value propositions (win conditions) The Model-Clash Spider Web: Master Net

14. University of Southern California Center for Systems and Software Engineering ©USC-CSSE14 Theory W: Enterprise Success Theorem – And informal proof Theorem: Your enterprise will succeed if and only if it makes winners of your success-critical stakeholders Proof of “if”: Everyone that counts is a winner. Nobody significant is left to complain. Proof of “only if”: Nobody wants to lose. Prospective losers will refuse to participate, or will counterattack. The usual result is lose-lose.

15. University of Southern California Center for Systems and Software Engineering ©USC-CSSE15 Theory W: WinWin Achievement Theorem Making winners of your success-critical stakeholders requires: i.Identifying all of the success-critical stakeholders (SCSs). ii.Understanding how the SCSs want to win. iii.Having the SCSs negotiate a win-win set of product and process plans. iv.Controlling progress toward SCS win-win realization, including adaptation to change.

16. University of Southern California Center for Systems and Software Engineering ©USC-CSSE16 VBSE Theory 4+1 Structure

17. University of Southern California Center for Systems and Software Engineering ©USC-CSSE17 VBSE Component Theories Theory W (Stakeholder win-win) –Enterprise Success Theorem, Win-Win Achievement Theorem Dependency Theory (Product, process, people interdependencies) –Systems architecture/performance theory, costing and scheduling theory; organization theory Utility Theory –Utility functions, bounded rationality, Maslow need hierarchy, multi-attribute utility theory Decision Theory –Statistical decision theory, game theory, negotiation theory, theory of Justice Control Theory –Observability, predictability, controllability, stability theory

18. University of Southern California Center for Systems and Software Engineering ©USC-CSSE18 Dependency Theory - Example

19. University of Southern California Center for Systems and Software Engineering ©USC-CSSE19 Decision Theory – Example - RE due to inadequate plans - RE due to market share erosion - Sum of risk exposures

20. University of Southern California Center for Systems and Software Engineering ©USC-CSSE20 Control Theory - Example Value Realization Feedback Control

21. University of Southern California Center for Systems and Software Engineering ©USC-CSSE21 Initial VBSE Theory: 4+1 Process – With a great deal of concurrency and backtracking

22. University of Southern California Center for Systems and Software Engineering ©USC-CSSE22 Example Project: Sierra Mountainbikes –Based on what would have worked on a similar project Quality leader in specialty area Competitively priced Major problems with order processing –Delivery delays and mistakes –Poor synchronization of order entry, confirmation, fulfillment –Disorganized responses to problem situations –Excess costs; low distributor satisfaction

23. University of Southern California Center for Systems and Software Engineering ©USC-CSSE23 Order Processing Project Goals Goals: Improve profits, market share, customer satisfaction via improved order processing Questions: Current state? Root causes of problems? Keys to improvement? Metrics: Balanced Scorecard of benefits realized, proxies –Customer satisfaction ratings; key elements (ITV: in- transit visibility) –Overhead cost reduction –Actual vs. expected benefit and cost flows, ROI

24. University of Southern California Center for Systems and Software Engineering ©USC-CSSE24 Initial VBSE Theory: 4+1 Process, Step 1 – With a great deal of concurrency and backtracking

25. University of Southern California Center for Systems and Software Engineering ©USC-CSSE25 Frequent Protagonist Classes Sierra Moutainbikes: Susan Swanson, new CEO – Bicycle champion, MBA, 15 years’ experience – Leads with goals, open agenda

26. University of Southern California Center for Systems and Software Engineering ©USC-CSSE26 Initial VBSE Theory: 4+1 Process, Step 2 – With a great deal of concurrency and backtracking

27. University of Southern California Center for Systems and Software Engineering ©USC-CSSE27 DMR/BRA* Results Chain INITIATIVE OUTCOME Implement a new order entry system ASSUMPTION Contribution Order to delivery time is an important buying criterion Reduce time to process order Reduced order processing cycle (intermediate outcome) Increased sales Reduce time to deliver product *DMR Consulting Group’s Benefits Realization Approach

28. University of Southern California Center for Systems and Software Engineering ©USC-CSSE28 Expanded Order Processing System Benefits Chain

29. University of Southern California Center for Systems and Software Engineering ©USC-CSSE29 Initial VBSE Theory: 4+1 Process, Step 3 – With a great deal of concurrency and backtracking

30. University of Southern California Center for Systems and Software Engineering ©USC-CSSE30 Initial VBSE Theory: 4+1 Process, Step 4 – With a great deal of concurrency and backtracking

31. University of Southern California Center for Systems and Software Engineering ©USC-CSSE31 The Model-Clash Spider Web: Master Net - Stakeholder value propositions (win conditions)

32. University of Southern California Center for Systems and Software Engineering ©USC-CSSE32 EasyWinWin OnLine Negotiation Steps

33. University of Southern California Center for Systems and Software Engineering ©USC-CSSE33 Red cells indicate lack of consensus. Oral discussion of cell graph reveals unshared information, unnoticed assumptions, hidden issues, constraints, etc.

34. University of Southern California Center for Systems and Software Engineering ©USC-CSSE34 Initial VBSE Theory: 4+1 Process, Step 5 – With a great deal of concurrency and backtracking

35. University of Southern California Center for Systems and Software Engineering ©USC-CSSE35 Project Strategy and Partnerships Partner with eServices, Inc. for order processing and fulfillment system –Profit sharing using jointly-developed business case Partner with key distributors to provide user feedback –Evaluate prototypes, beta-test early versions, provide satisfaction ratings Incremental development using MBASE/RUP anchor points –Life Cycle Objectives; Architecture (LCO; LCA) –Core Capability Drivethrough (CCD) –Initial; Full Operational Capability (IOC; FOC) Architect for later supply chain extensions

36. University of Southern California Center for Systems and Software Engineering ©USC-CSSE36 Business Case Analysis Estimate costs and schedules –COCOMO II and/or alternative for software –PRICE H or alternative for hardware –COSYSMO for systems engineering Estimate financial benefits –Increased profits –Reduced operating costs Compute Return on Investment –ROI = (Benefits – Costs) / Costs –Normalized to present value Identify quantitative metrics for other goals –Customer satisfaction ratings Ease of use; In-transit visibility; overall –Late delivery percentage

37. University of Southern California Center for Systems and Software Engineering ©USC-CSSE37 MilestoneDue DateBudget ($K)Cumulative Budget ($K) Inception Readiness1/1/200400 Life Cycle Objectives1/31/2004120 Life Cycle Architecture3/31/2004280400 Core Capability Drivethrough7/31/20046501050 Initial Oper. Capability: SW9/30/20043501400 Initial Oper. Capability: HW9/30/200421003500 Developed IOC12/31/20045004000 Responsive IOC3/31/20055004500 Full Oper. Cap’y CCD7/31/20057005200 FOC Beta9/30/20054005600 FOC Deployed12/31/20054006000 Annual Oper. & Maintenance3800 Annual O&M; Old System7600 Order Processing System Schedules and Budgets

38. University of Southern California Center for Systems and Software Engineering ©USC-CSSE38 Order Processing System: Expected Benefits and Business Case

39. University of Southern California Center for Systems and Software Engineering ©USC-CSSE39 Initial VBSE Theory: 4+1 Process, Step 7 – With a great deal of concurrency and backtracking

40. University of Southern California Center for Systems and Software Engineering ©USC-CSSE40 A Real Earned Value System Current “earned value” systems monitor cost and schedule, not business value –Budgeted cost of work performed (“earned”) –Budgeted cost of work scheduled (“yearned”) –Actual costs vs. schedule (“burned”) A real earned value system monitors benefits realized –Financial benefits realized vs. cost (ROI) –Benefits realized vs. schedule - Including non-financial metrics –Actual costs vs. schedule

41. University of Southern California Center for Systems and Software Engineering ©USC-CSSE41 Value-Based Expected/Actual Outcome Tracking Capability

42. University of Southern California Center for Systems and Software Engineering ©USC-CSSE42 C. Baldwin & K. Clark, Design Rules: The Power of Modularity, MIT Press, 1999. S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Gruenbacher (eds.), Value-Based Software Engineering, Springer, 2005 (to appear). D. Blackwell and M. Girshick, Theory of Games and Statistical Decisions, Wiley, 1954. B. Boehm, C. Abts, A.W. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece, Software Cost Estimation with COCOMO II, Prentice Hall, 2000. B. Boehm and L. Huang, “Value-Based Software Engineering: A Case Study, Computer, March 2003, pp. 33-41. B. Boehm, and R. Ross, Theory-W Software Project Management: Principles and Examples, IEEE Trans. SW Engineering., July 1989, pp. 902-916. W. Brogan, Modern Control Theory, Prentice Hall, 1974 (3 rd ed., 1991). P. Checkland, Systems Thinking, Systems Practice, Wiley, 1981. C. W. Churchman, R. Ackoff, and E. Arnoff, An Introduction to Operations Research, Wiley, 1957. R. M. Cyert and J.G. March, A Behavioral Theory of the Firm, Prentice Hall, 1963. C. G. Hempel and P. Oppenheim, Problems of the Concept of General Law, in (eds.) A. Danto and S. Mogenbesser, Philosophy of Science, Meridian Books, 1960. References - I

43. University of Southern California Center for Systems and Software Engineering ©USC-CSSE43 R. Kaplan & D. Norton, The Balanced Scorecard: Translating Strategy into Action, Harvard Business School Press, 1996. R. L. Keeney and H. Raiffa, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Cambridge University Press, 1976. A. Maslow, Motivation and Personality, Harper, 1954. J. Rawls, A Theory of Justice, Belknap/Harvard U. Press, 1971, 1999. J. Thorp and DMR, The Information Paradox, McGraw Hill, 1998. R. J. Torraco, Theory-building research methods, in R. A. Swanson & E. F. Holton III (eds.), Human resource development handbook: Linking research and practice pp. 114–137, Berrett-Koehler, 1997. S. Toulmin, Cosmopolis: The Hidden Agenda of Modernity, U. of Chicago Press, 1992 reprint edition. J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, 1944. A. W. Wymore, A Mathematical Theory of Systems Engineering: The Elements, Wiley, New York, 1967. References - II