1. University of Southern California Center for Software Engineering C S E USC Barry Boehm, USC CS 510 Lecture Fall 2011 Value-Based Software Engineering: Case Study and Value-Based Control

2. University of Southern California Center for Software Engineering C S E USC August 29, 20112 Outline Value-based software engineering (VBSE) process framework Application to case study; key practices Conclusions and references ©USC-CSSE

3. University of Southern California Center for Software Engineering C S E USC August 29, 20113 Initial VBSE Theory: 4+1 Engine: Theory W (stakeholder win-win): What values are important? –Enterprise Success Theorem –Theory of Justice –Win-Win Equilibrium and Negotiation Four Supporting Theories –Dependency Theory: How do dependencies affect value realization? –Results chains; value chains; cost/schedule/performance tradeoffs –Utility Theory: How important are the values? –Multi-attribute utility; Maslow need hierarchy –Decision Theory: How do values determine decisions? –Investment theory; game theory; statistical decision theory –Control Theory: How to monitor and control value realization –Feedback control; adaptive control; spiral risk control ©USC-CSSE

4. University of Southern California Center for Software Engineering C S E USC August 29, 20114 VBSE Theory 4+1 Structure ©USC-CSSE

5. University of Southern California Center for Software Engineering C S E USC August 29, 20115 Initial VBSE Theory: 4+1 Process – With a great deal of concurrency and backtracking ©USC-CSSE

6. University of Southern California Center for Software Engineering C S E USC Need Incremental Approach to VBSE Process Cannot make all decisions in a single pass –Emergent technology: social networking –Emergent requirements: outdo competitors –TBD standards: information exchange; infrastructure –Evolving platforms: mobile devices –Emerging stakeholders: strategic partners, suppliers Best to incrementally define scope, features, details –But do best-effort architecting for the future Avoid easiest-first increments on large, critical projects Concurrently determine requirements and solutions Best to incrementally develop product features –Avoid obsolete deliveries –Early focus on critical infrastructure, high-value features August 29, 20116©USC-CSSE

7. University of Southern California Center for Software Engineering C S E USC August 29, 20117 Outline Value-based software engineering (VBSE) process framework Application to case study; key practices Conclusions and references ©USC-CSSE

8. University of Southern California Center for Software Engineering C S E USC August 29, 20118 Example Project: Sierra Mountainbikes –Based on what would have worked on a similar project Quality leader in specialty area Competitively priced Major problems with order processing –Delivery delays and mistakes –Poor synchronization of order entry, confirmation, fulfillment –Disorganized responses to problem situations –Excess costs; low distributor satisfaction ©USC-CSSE

9. University of Southern California Center for Software Engineering C S E USC August 29, 20119 Order Processing Project Goals Goals: Improve profits, market share, customer satisfaction via improved order processing Questions: Current state? Root causes of problems? Keys to improvement? Metrics: Balanced Scorecard of benefits realized, proxies –Customer satisfaction ratings; key elements (ITV: in-transit visibility) –Overhead cost reduction –Actual vs. expected benefit and cost flows, ROI ©USC-CSSE

10. University of Southern California Center for Software Engineering C S E USC August 29, 201110 Initial VBSE Theory: 4+1 Process, Steps 1 and 2 – With a great deal of concurrency and backtracking ©USC-CSSE

11. University of Southern California Center for Software Engineering C S E USC August 29, 201111 Frequent Protagonist Classes Without a pro-active protagonist, a project won’t start Sierra Moutainbikes: Susan Swanson, new CEO – Bicycle champion, MBA, 15 years’ experience – Leads with goals, open agenda ©USC-CSSE

12. University of Southern California Center for Software Engineering C S E USC August 29, 201112 DMR/BRA* Results Chain INITIATIVE OUTCOME Implement a new order entry system ASSUMPTION Contribution Order to delivery time is an important buying criterion Reduce time to process order Reduced order processing cycle (intermediate outcome) Increased sales Reduce time to deliver product *DMR Consulting Group’s Benefits Realization Approach ©USC-CSSE

13. University of Southern California Center for Software Engineering C S E USC August 29, 201113 Expanded Order Processing System Benefits Chain ©USC-CSSE

14. University of Southern California Center for Software Engineering C S E USC August 29, 201114 Initial VBSE Theory: 4+1 Process, Steps 3 and 4 – With a great deal of concurrency and backtracking ©USC-CSSE

15. University of Southern California Center for Software Engineering C S E USC August 29, 201115 The Model-Clash Spider Web: Master Net - Stakeholder value propositions (win conditions) ©USC-CSSE

16. University of Southern California Center for Software Engineering C S E USC August 29, 201116 There is No Universal Quality-Value Metric Different stakeholders rely on different value attributes –Protection: safety, security, privacy –Robustness: reliability, availability, survivability –Quality of Service: performance, accuracy, ease of use –Adaptability: evolvability, interoperability –Affordability: cost, schedule, reusability Value attributes continue to tier down –Performance: response time, resource consumption (CPU, memory, comm.) Value attributes are scenario-dependent –5 seconds normal response time; 2 seconds in crisis Value attributes often conflict –Most often with performance and affordability ©USC-CSSE

17. University of Southern California Center for Software Engineering C S E USC August 29, 201117 Major Information System Quality Stakeholders Information Suppliers -citizens, companies Information Brokers Information Consumers -financial services, -decisions, education, Mission Controllers Developers, Acquirers, Administrators Dependents Information System - passengers, patients news media entertainment -pilots, distribution controllers ©USC-CSSE

18. University of Southern California Center for Software Engineering C S E USC August 29, 201118 Overview of Stakeholder/Value Dependencies Strength of direct dependency on value attribute **- Critical ; *-Significant; blank-insignificant or indirect Attributes Stakeholders ** * * * * * * ** * Protection Robustness Quality of Service Adaptability Affordability Developers, Acquirers Mission Controllers, Administrators Info. Consumers Info. Brokers Info. Suppliers, Dependents ©USC-CSSE

19. University of Southern California Center for Software Engineering C S E USC August 29, 201119 Stakeholders Elaboration of Stakeholder/Value Dependencies ©USC-CSSE

20. University of Southern California Center for Software Engineering C S E USC August 29, 201120 Cost, Schedule, Quality: Pick any Two? C QS ©USC-CSSE

21. University of Southern California Center for Software Engineering C S E USC August 29, 201121 Tradeoffs Among Cost, Schedule, and Reliability: COCOMO II -- Cost/Schedule/RELY: “pick any two” points (RELY, MTBF (hours)) For 100-KSLOC set of features Can “pick all three” with 77-KSLOC set of features ©USC-CSSE

22. University of Southern California Center for Software Engineering C S E USC August 29, 201122 Cost, Schedule, Quality: Pick any Two? C QS C QS Consider C, S, Q as Independent Variable – Feature Set as Dependent Variable ©USC-CSSE

23. University of Southern California Center for Software Engineering C S E USC August 29, 201123 C, S, Q as Independent Variable Determine Desired Delivered Defect Density (D4) –Or a value-based equivalent Prioritize desired features –Via QFD, IPT, stakeholder win-win Determine Core Capability –90% confidence of D4 within cost and schedule –Balance parametric models and expert judgment Architect for ease of adding next-priority features –Hide sources of change within modules (Parnas) Develop core capability to D4 quality level –Usually in less than available cost and schedule Add next priority features as resources permit Versions used successfully on 32 of 34 USC digital library projects ©USC-CSSE

24. University of Southern California Center for Software Engineering C S E USC August 29, 201124 Implications for Quality Engineering There is no universal quality metric to optimize Need to identify system’s success-critical stakeholders –And their quality priorities Need to balance satisfaction of stakeholder dependencies –Stakeholder win-win negotiation –Quality attribute tradeoff analysis Need value-of-quality models, methods, and tools –COCOMO example: cost-schedule-quality- size tradeoffs –Stakeholder win-win negotiation tools ©USC-CSSE

25. University of Southern California Center for Software Engineering C S E USC August 29, 201125 Win-lose Generally Becomes Lose-lose Actually, nobody wins in these situations ©USC-CSSE

26. University of Southern California Center for Software Engineering C S E USC August 29, 201126 Key Concepts Win Condition: Objective which makes a stakeholder feel like a winner Issue: Conflict or constraint on a win condition Option: A way of overcoming an issue Agreement: Mutual commitment to an option or win condition ©USC-CSSE

27. University of Southern California Center for Software Engineering C S E USC August 29, 201127 Win Condition Agreement Option Issue involves addresses adopts covers WinWin Negotiation Model WinWin Equilibrium State - All Win Conditions covered by Agreements - No outstanding Issues ©USC-CSSE

28. University of Southern California Center for Software Engineering C S E USC August 29, 201128 EasyWinWin OnLine Negotiation Steps ©USC-CSSE

29. University of Southern California Center for Software Engineering C S E USC August 29, 201129 Red cells indicate lack of consensus. Oral discussion of cell graph reveals unshared information, unnoticed assumptions, hidden issues, constraints, etc. ©USC-CSSE

30. University of Southern California Center for Software Engineering C S E USC August 29, 201130 WikiWinWin: Identify and Resolve Issues ©USC-CSSE

31. University of Southern California Center for Software Engineering C S E USC August 29, 201131 Initial VBSE Theory: 4+1 Process, Step 5 – With a great deal of concurrency and backtracking ©USC-CSSE

32. University of Southern California Center for Software Engineering C S E USC August 29, 201132 Project Strategy and Partnerships Partner with eServices, Inc. for order processing and fulfillment system –Profit sharing using jointly-developed business case Partner with key distributors to provide user feedback –Evaluate prototypes, beta-test early versions, provide satisfaction ratings Incremental development using MBASE/RUP anchor points –Life Cycle Objectives; Architecture (LCO; LCA) –Core Capability Drivethrough (CCD) –Initial; Full Operational Capability (IOC; FOC) Architect for later supply chain extensions ©USC-CSSE

33. University of Southern California Center for Software Engineering C S E USC August 29, 201133 Business Case Analysis Estimate costs and schedules –COCOMO II and/or alternative for software –PRICE H or alternative for hardware –COSYSMO for systems engineering Estimate financial benefits –Increased profits –Reduced operating costs Compute Return on Investment –ROI = (Benefits – Costs) / Costs –Normalized to present value Identify quantitative metrics for other goals –Customer satisfaction ratings Ease of use; In-transit visibility; overall –Late delivery percentage ©USC-CSSE

34. University of Southern California Center for Software Engineering C S E USC August 29, 201134 MilestoneDue DateBudget ($K)Cumulative Budget ($K) Inception Readiness1/1/200400 Life Cycle Objectives1/31/2004120 Life Cycle Architecture3/31/2004280400 Core Capability Drivethrough7/31/20046501050 Initial Oper. Capability: SW9/30/20043501400 Initial Oper. Capability: HW9/30/200421003500 Developed IOC12/31/20045004000 Responsive IOC3/31/20055004500 Full Oper. Cap’y CCD7/31/20057005200 FOC Beta9/30/20054005600 FOC Deployed12/31/20054006000 Annual Oper. & Maintenance3800 Annual O&M; Old System7600 Order Processing System Schedules and Budgets ©USC-CSSE

35. University of Southern California Center for Software Engineering C S E USC August 29, 201135 Order Processing System: Expected Benefits and Business Case ©USC-CSSE

36. University of Southern California Center for Software Engineering C S E USC August 29, 201136 Initial VBSE Theory: 4+1 Process, Steps 6 and 7 – With a great deal of concurrency and backtracking ©USC-CSSE

37. University of Southern California Center for Software Engineering C S E USC August 29, 201137 A Real Earned Value System Current “earned value” systems monitor cost and schedule, not business value –Budgeted cost of work performed (“earned”) –Budgeted cost of work scheduled (“yearned”) –Actual costs vs. schedule (“burned”) A real earned value system monitors benefits realized – Financial benefits realized vs. cost (ROI) –Benefits realized vs. schedule - Including non-financial metrics –Actual costs vs. schedule ©USC-CSSE

38. University of Southern California Center for Software Engineering C S E USC August 29, 201138 Value-Based Expected/Actual Outcome Tracking Capability ©USC-CSSE

39. University of Southern California Center for Software Engineering C S E USC August 29, 201139 Conclusions Current SE methods are basically value- neutral Value-neutral SE methods are increasingly risky VBSE agenda making progress, but major challenges remain –Evolving VBSE theory –Creating VB counterparts for value-neutral SE methods ©USC-CSSE

40. University of Southern California Center for Software Engineering C S E USC August 29, 201140 C. Baldwin & K. Clark, Design Rules: The Power of Modularity, MIT Press, 1999. S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Gruenbacher (eds.), Value-Based Software Engineering, Springer, 2005. B. Boehm, “Value-Based Software Engineering,” ACM Software Engineering Notes, March 2003. B. Boehm, C. Abts, A.W. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece, Software Cost Estimation with COCOMO II, Prentice Hall, 2000. B. Boehm and L. Huang, “Value-Based Software Engineering: A Case Study, Computer, March 2003, pp. 33-41. B. Boehm & K. Sullivan, “Software Economics: A Roadmap,” The Future of Software Economics, A. Finkelstein (ed.), ACM Press, 2000. B. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Addison Wesley, 2003 (to appear). B. Boehm, L. Huang, A. Jain. R. Madachy, “ The ROI of Software Dependability: The iDAVE Model”, IEEE Software Special Issue on Return on Investment, May/June 2004. M. Denne and J. Cleland-Huang, Software by Numbers, Prentice Hall, 2004. References - I ©USC-CSSE

41. University of Southern California Center for Software Engineering C S E USC August 29, 201141 S. Faulk, D. Harmon, and D. Raffo, “Value-Based Software Engineering (VBSE): A Value-Driven Approach to Product-Line Engineering,” Proceedings, First Intl. Conf. On SW Product Line Engineering, August 2000. R. Kaplan & D. Norton, The Balanced Scorecard: Translating Strategy into Action, Harvard Business School Press, 1996. D. Reifer, Making the Software Business Case, Addison Wesley, 2002. K. Sullivan, Y. Cai, B. Hallen, and W. Griswold, “The Structure and Value of Modularity in Software Design,” Proceedings, ESEC/FSE, 2001, ACM Press, pp. 99-108. J. Thorp and DMR, The Information Paradox, McGraw Hill, 1998. S. Tockey, Return on Software, Addison Wesley, 2004. Economics-Driven Software Engineering Research (EDSER) web site: www.edser.org MBASE web site : sunset.usc.edu/research/MBASE References - II ©USC-CSSE