Value-Based Software Engineering: Case Study and Value-Based Control Barry Boehm, USC CS 510 Lecture Fall 2009

Outline Value-based software engineering (VBSE) process framework Application to case study; key practices Conclusions and references 07/09/09 ©USC-CSE

Initial VBSE Theory: 4+1 Engine: Theory W (stakeholder win-win): What values are important? Enterprise Success Theorem Theory of Justice Win-Win Equilibrium and Negotiation Four Supporting Theories Dependency Theory: How do dependencies affect value realization? Results chains; value chains; cost/schedule/performance tradeoffs Utility Theory: How important are the values? Multi-attribute utility; Maslow need hierarchy Decision Theory: How do values determine decisions? Investment theory; game theory; statistical decision theory Control Theory: How to monitor and control value realization Feedback control; adaptive control; spiral risk control 07/09/09 ©USC-CSE

VBSE Theory 4+1 Structure 07/09/09 ©USC-CSE

Initial VBSE Theory: 4+1 Process – With a great deal of concurrency and backtracking 07/09/09 ©USC-CSE

Need Incremental Approach to VBSE Process Cannot make all decisions in a single pass Emergent technology: social networking Emergent requirements: outdo competitors TBD standards: information exchange; infrastructure Evolving platforms: mobile devices Emerging stakeholders: strategic partners, suppliers Best to incrementally define scope, features, details But do best-effort architecting for the future Avoid easiest-first increments on large, critical projects Concurrently determine requirements and solutions Best to incrementally develop product features Avoid obsolete deliveries Early focus on critical infrastructure, high-value features 07/09/09 ©USC-CSE

Outline Value-based software engineering (VBSE) process framework Application to case study; key practices Conclusions and references 07/09/09 ©USC-CSE

Example Project: Sierra Mountainbikes Based on what would have worked on a similar project Quality leader in specialty area Competitively priced Major problems with order processing Delivery delays and mistakes Poor synchronization of order entry, confirmation, fulfillment Disorganized responses to problem situations Excess costs; low distributor satisfaction 07/09/09 ©USC-CSE

Order Processing Project Goals Goals: Improve profits, market share, customer satisfaction via improved order processing Questions: Current state? Root causes of problems? Keys to improvement? Metrics: Balanced Scorecard of benefits realized, proxies Customer satisfaction ratings; key elements (ITV: in-transit visibility) Overhead cost reduction Actual vs. expected benefit and cost flows, ROI 07/09/09 ©USC-CSE

Initial VBSE Theory: 4+1 Process, Steps 1 and 2 – With a great deal of concurrency and backtracking 07/09/09 ©USC-CSE

Frequent Protagonist Classes Without a pro-active protagonist, a project won’t start Sierra Moutainbikes: Susan Swanson, new CEO Bicycle champion, MBA, 15 years’ experience Leads with goals, open agenda 07/09/09 ©USC-CSE

DMR/BRA* Results Chain Order to delivery time is an important buying criterion ASSUMPTION INITIATIVE OUTCOME Contribution Contribution OUTCOME Reduced order processing cycle (intermediate outcome) Implement a new order entry system Reduce time to process order Increased sales Reduce time to deliver product *DMR Consulting Group’s Benefits Realization Approach 07/09/09 ©USC-CSE

Expanded Order Processing System Benefits Chain 07/09/09 ©USC-CSE

Initial VBSE Theory: 4+1 Process, Steps 3 and 4 – With a great deal of concurrency and backtracking 07/09/09 ©USC-CSE

The Model-Clash Spider Web: Master Net - Stakeholder value propositions (win conditions) 07/09/09 ©USC-CSE

There is No Universal Quality-Value Metric Different stakeholders rely on different value attributes Protection: safety, security, privacy Robustness: reliability, availability, survivability Quality of Service: performance, accuracy, ease of use Adaptability: evolvability, interoperability Affordability: cost, schedule, reusability Value attributes continue to tier down Performance: response time, resource consumption (CPU, memory, comm.) Value attributes are scenario-dependent 5 seconds normal response time; 2 seconds in crisis Value attributes often conflict Most often with performance and affordability 07/09/09 ©USC-CSE

Major Information System Quality Stakeholders Dependents - passengers, patients Information Brokers -financial services, Information Suppliers -citizens, companies news media Information System Information Consumers -decisions, education, entertainment Mission Controllers -pilots, distribution controllers Developers, Acquirers, Administrators 07/09/09 ©USC-CSE

Overview of Stakeholder/Value Dependencies Strength of direct dependency on value attribute **- Critical ; *-Significant; blank-insignificant or indirect Quality of Service Robustness Adaptability Protection Affordability Attributes Stakeholders Info. Suppliers, Dependents ** * * Info. Brokers ** ** ** ** * Info. Consumers * ** * * Mission Controllers, Administrators * ** ** ** Developers, Acquirers * * ** ** 07/09/09 ©USC-CSE

Elaboration of Stakeholder/Value Dependencies Stakeholders 07/09/09 ©USC-CSE

Cost, Schedule, Quality: Pick any Two? 07/09/09 ©USC-CSE

Tradeoffs Among Cost, Schedule, and Reliability: COCOMO II (RELY, MTBF (hours)) For 100-KSLOC set of features Can “pick all three” with 77-KSLOC set of features -- Cost/Schedule/RELY: “pick any two” points 07/09/09 ©USC-CSE

Cost, Schedule, Quality: Pick any Two? Consider C, S, Q as Independent Variable Feature Set as Dependent Variable C C S Q S Q 07/09/09 ©USC-CSE

C, S, Q as Independent Variable Determine Desired Delivered Defect Density (D4) Or a value-based equivalent Prioritize desired features Via QFD, IPT, stakeholder win-win Determine Core Capability 90% confidence of D4 within cost and schedule Balance parametric models and expert judgment Architect for ease of adding next-priority features Hide sources of change within modules (Parnas) Develop core capability to D4 quality level Usually in less than available cost and schedule Add next priority features as resources permit Versions used successfully on 32 of 34 USC digital library projects 07/09/09 ©USC-CSE

Implications for Quality Engineering There is no universal quality metric to optimize Need to identify system’s success-critical stakeholders And their quality priorities Need to balance satisfaction of stakeholder dependencies Stakeholder win-win negotiation Quality attribute tradeoff analysis Need value-of-quality models, methods, and tools COCOMO example: cost-schedule-quality- size tradeoffs Stakeholder win-win negotiation tools 07/09/09 ©USC-CSE

Win-lose Generally Becomes Lose-lose Actually, nobody wins in these situations 07/09/09 ©USC-CSE

Key Concepts Win Condition: Objective which makes a stakeholder feel like a winner Issue: Conflict or constraint on a win condition Option: A way of overcoming an issue Agreement: Mutual commitment to an option or win condition 07/09/09 ©USC-CSE

WinWin Negotiation Model Win Condition Issue involves covers addresses Agreement Option adopts WinWin Equilibrium State - All Win Conditions covered by Agreements - No outstanding Issues 07/09/09 ©USC-CSE

EasyWinWin OnLine Negotiation Steps 07/09/09 ©USC-CSE

Red cells indicate lack of consensus. Oral discussion of cell graph reveals unshared information, unnoticed assumptions, hidden issues, constraints, etc. 07/09/09 ©USC-CSE

WikiWinWin: Identify and Resolve Issues 07/09/09 ©USC-CSE

Initial VBSE Theory: 4+1 Process, Step 5 – With a great deal of concurrency and backtracking 07/09/09 ©USC-CSE

Project Strategy and Partnerships Partner with eServices, Inc. for order processing and fulfillment system Profit sharing using jointly-developed business case Partner with key distributors to provide user feedback Evaluate prototypes, beta-test early versions, provide satisfaction ratings Incremental development using MBASE/RUP anchor points Life Cycle Objectives; Architecture (LCO; LCA) Core Capability Drivethrough (CCD) Initial; Full Operational Capability (IOC; FOC) Architect for later supply chain extensions 07/09/09 ©USC-CSE

Business Case Analysis Estimate costs and schedules COCOMO II and/or alternative for software PRICE H or alternative for hardware COSYSMO for systems engineering Estimate financial benefits Increased profits Reduced operating costs Compute Return on Investment ROI = (Benefits – Costs) / Costs Normalized to present value Identify quantitative metrics for other goals Customer satisfaction ratings Ease of use; In-transit visibility; overall Late delivery percentage 07/09/09 ©USC-CSE

Order Processing System Schedules and Budgets Milestone Due Date Budget ($K) Cumulative Budget ($K) Inception Readiness 1/1/2004 Life Cycle Objectives 1/31/2004 120 Life Cycle Architecture 3/31/2004 280 400 Core Capability Drivethrough 7/31/2004 650 1050 Initial Oper. Capability: SW 9/30/2004 350 1400 Initial Oper. Capability: HW 2100 3500 Developed IOC 12/31/2004 500 4000 Responsive IOC 3/31/2005 4500 Full Oper. Cap’y CCD 7/31/2005 700 5200 FOC Beta 9/30/2005 5600 FOC Deployed 12/31/2005 6000 Annual Oper. & Maintenance 3800 Annual O&M; Old System 7600 07/09/09 ©USC-CSE

Order Processing System: Expected Benefits and Business Case 07/09/09 ©USC-CSE

Initial VBSE Theory: 4+1 Process, Steps 6 and 7 – With a great deal of concurrency and backtracking 07/09/09 ©USC-CSE

A Real Earned Value System Current “earned value” systems monitor cost and schedule, not business value Budgeted cost of work performed (“earned”) Budgeted cost of work scheduled (“yearned”) Actual costs vs. schedule (“burned”) A real earned value system monitors benefits realized Financial benefits realized vs. cost (ROI) Benefits realized vs. schedule - Including non-financial metrics Actual costs vs. schedule 07/09/09 ©USC-CSE

Value-Based Expected/Actual Outcome Tracking Capability 07/09/09 ©USC-CSE

Conclusions Current SE methods are basically value-neutral Value-neutral SE methods are increasingly risky VBSE agenda making progress, but major challenges remain Evolving VBSE theory Creating VB counterparts for value-neutral SE methods 07/09/09 ©USC-CSE

References - I C. Baldwin & K. Clark, Design Rules: The Power of Modularity, MIT Press, 1999. S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Gruenbacher (eds.), Value-Based Software Engineering, Springer, 2005. B. Boehm, “Value-Based Software Engineering,” ACM Software Engineering Notes, March 2003. B. Boehm, C. Abts, A.W. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece, Software Cost Estimation with COCOMO II, Prentice Hall, 2000. B. Boehm and L. Huang, “Value-Based Software Engineering: A Case Study, Computer, March 2003, pp. 33-41. B. Boehm & K. Sullivan, “Software Economics: A Roadmap,” The Future of Software Economics, A. Finkelstein (ed.), ACM Press, 2000. B. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Addison Wesley, 2003 (to appear). B. Boehm, L. Huang, A. Jain. R. Madachy, “ The ROI of Software Dependability: The iDAVE Model”, IEEE Software Special Issue on Return on Investment, May/June 2004. M. Denne and J. Cleland-Huang, Software by Numbers, Prentice Hall, 2004. 07/09/09 ©USC-CSE

References - II S. Faulk, D. Harmon, and D. Raffo, “Value-Based Software Engineering (VBSE): A Value-Driven Approach to Product-Line Engineering,” Proceedings, First Intl. Conf. On SW Product Line Engineering, August 2000. R. Kaplan & D. Norton, The Balanced Scorecard: Translating Strategy into Action, Harvard Business School Press, 1996. D. Reifer, Making the Software Business Case, Addison Wesley, 2002. K. Sullivan, Y. Cai, B. Hallen, and W. Griswold, “The Structure and Value of Modularity in Software Design,” Proceedings, ESEC/FSE, 2001, ACM Press, pp. 99-108. J. Thorp and DMR, The Information Paradox, McGraw Hill, 1998. S. Tockey, Return on Software, Addison Wesley, 2004. Economics-Driven Software Engineering Research (EDSER) web site: www.edser.org MBASE web site : sunset.usc.edu/research/MBASE 07/09/09 ©USC-CSE