1. Value-Based Software Engineering: Case Study and Value-Based Control
Barry Boehm, USC
CS 510 Lecture
Fall 2009

2. Outline Value-based software engineering (VBSE) process framework
Application to case study; key practices
Conclusions and references
07/09/09
©USC-CSE

3. Initial VBSE Theory: 4+1
Engine: Theory W (stakeholder win-win): What values are important?
Enterprise Success Theorem
Theory of Justice
Win-Win Equilibrium and Negotiation
Four Supporting Theories
Dependency Theory: How do dependencies affect value realization?
Results chains; value chains; cost/schedule/performance tradeoffs
Utility Theory: How important are the values?
Multi-attribute utility; Maslow need hierarchy
Decision Theory: How do values determine decisions?
Investment theory; game theory; statistical decision theory
Control Theory: How to monitor and control value realization
Feedback control; adaptive control; spiral risk control
07/09/09
©USC-CSE

4. VBSE Theory 4+1 Structure
07/09/09
©USC-CSE

5. Initial VBSE Theory: 4+1 Process – With a great deal of concurrency and backtracking
07/09/09
©USC-CSE

6. Need Incremental Approach to VBSE Process
Cannot make all decisions in a single pass
Emergent technology: social networking
Emergent requirements: outdo competitors
TBD standards: information exchange; infrastructure
Evolving platforms: mobile devices
Emerging stakeholders: strategic partners, suppliers
Best to incrementally define scope, features, details
But do best-effort architecting for the future
Avoid easiest-first increments on large, critical projects
Concurrently determine requirements and solutions
Best to incrementally develop product features
Avoid obsolete deliveries
Early focus on critical infrastructure, high-value features
07/09/09
©USC-CSE

7. Outline Value-based software engineering (VBSE) process framework
Application to case study; key practices
Conclusions and references
07/09/09
©USC-CSE

8. Example Project: Sierra Mountainbikes
Based on what would have worked on a similar project
Quality leader in specialty area
Competitively priced
Major problems with order processing
Delivery delays and mistakes
Poor synchronization of order entry, confirmation, fulfillment
Disorganized responses to problem situations
Excess costs; low distributor satisfaction
07/09/09
©USC-CSE

9. Order Processing Project Goals
Goals: Improve profits, market share, customer satisfaction via improved order processing
Questions: Current state? Root causes of problems? Keys to improvement?
Metrics: Balanced Scorecard of benefits realized, proxies
Customer satisfaction ratings; key elements (ITV: in-transit visibility)
Overhead cost reduction
Actual vs. expected benefit and cost flows, ROI
07/09/09
©USC-CSE

10. Initial VBSE Theory: 4+1 Process, Steps 1 and 2 – With a great deal of concurrency and backtracking
07/09/09
©USC-CSE

11. Frequent Protagonist Classes Without a pro-active protagonist, a project won’t start
Sierra Moutainbikes: Susan Swanson, new CEO
Bicycle champion, MBA, 15 years’ experience
Leads with goals, open agenda
07/09/09
©USC-CSE

12. DMR/BRA* Results Chain
Order to delivery time is
an important buying criterion
ASSUMPTION
INITIATIVE
OUTCOME
Contribution
Contribution
OUTCOME
Reduced order processing cycle
(intermediate outcome)
Implement a new order
entry system
Reduce time to process order
Increased sales
Reduce time to deliver product
*DMR Consulting Group’s Benefits Realization Approach
07/09/09
©USC-CSE

13. Expanded Order Processing System Benefits Chain
07/09/09
©USC-CSE

14. Initial VBSE Theory: 4+1 Process, Steps 3 and 4 – With a great deal of concurrency and backtracking
07/09/09
©USC-CSE

15. The Model-Clash Spider Web: Master Net
- Stakeholder value propositions (win conditions)
07/09/09
©USC-CSE

16. There is No Universal Quality-Value Metric
Different stakeholders rely on different value attributes
Protection: safety, security, privacy
Robustness: reliability, availability, survivability
Quality of Service: performance, accuracy, ease of use
Adaptability: evolvability, interoperability
Affordability: cost, schedule, reusability
Value attributes continue to tier down
Performance: response time, resource consumption (CPU, memory, comm.)
Value attributes are scenario-dependent
5 seconds normal response time; 2 seconds in crisis
Value attributes often conflict
Most often with performance and affordability
07/09/09
©USC-CSE

17. Major Information System Quality Stakeholders
Dependents
- passengers, patients
Information Brokers
-financial services,
Information Suppliers
-citizens, companies
news media
Information System
Information Consumers
-decisions, education,
entertainment
Mission Controllers
-pilots, distribution
controllers
Developers, Acquirers, Administrators
07/09/09
©USC-CSE

18. Overview of Stakeholder/Value Dependencies
Strength of direct dependency on value attribute
**- Critical ; *-Significant; blank-insignificant or indirect
Quality of Service
Robustness
Adaptability
Protection
Affordability
Attributes
Stakeholders
Info. Suppliers, Dependents ** * *
Info. Brokers ** ** ** ** *
Info. Consumers * ** * *
Mission Controllers, Administrators * ** ** **
Developers, Acquirers * * ** **
07/09/09
©USC-CSE

19. Elaboration of Stakeholder/Value Dependencies
Stakeholders
07/09/09
©USC-CSE

20. Cost, Schedule, Quality: Pick any Two?
07/09/09
©USC-CSE

21. Tradeoffs Among Cost, Schedule, and Reliability: COCOMO II
(RELY, MTBF (hours))
For 100-KSLOC set of features
Can “pick all three” with 77-KSLOC set of features
-- Cost/Schedule/RELY:
“pick any two” points
07/09/09
©USC-CSE

22. Cost, Schedule, Quality: Pick any Two?
Consider C, S, Q as Independent Variable
Feature Set as Dependent Variable C C S Q S Q
07/09/09
©USC-CSE

23. C, S, Q as Independent Variable
Determine Desired Delivered Defect Density (D4)
Or a value-based equivalent
Prioritize desired features
Via QFD, IPT, stakeholder win-win
Determine Core Capability
90% confidence of D4 within cost and schedule
Balance parametric models and expert judgment
Architect for ease of adding next-priority features
Hide sources of change within modules (Parnas)
Develop core capability to D4 quality level
Usually in less than available cost and schedule
Add next priority features as resources permit
Versions used successfully on 32 of 34 USC digital library projects
07/09/09
©USC-CSE

24. Implications for Quality Engineering
There is no universal quality metric to optimize
Need to identify system’s success-critical stakeholders
And their quality priorities
Need to balance satisfaction of stakeholder dependencies
Stakeholder win-win negotiation
Quality attribute tradeoff analysis
Need value-of-quality models, methods, and tools
COCOMO example: cost-schedule-quality- size tradeoffs
Stakeholder win-win negotiation tools
07/09/09
©USC-CSE

25. Win-lose Generally Becomes Lose-lose
Actually, nobody wins in these situations
07/09/09
©USC-CSE

26. Key Concepts
Win Condition: Objective which makes a stakeholder feel like a winner
Issue: Conflict or constraint on a win condition
Option: A way of overcoming an issue
Agreement: Mutual commitment to an option or win condition
07/09/09
©USC-CSE

27. WinWin Negotiation Model
Win Condition
Issue
involves
covers
addresses
Agreement
Option
adopts
WinWin Equilibrium State - All Win Conditions covered by Agreements - No outstanding Issues
07/09/09
©USC-CSE

28. EasyWinWin OnLine Negotiation Steps
07/09/09
©USC-CSE

29. Red cells indicate lack of consensus.
Oral discussion of cell graph reveals unshared information, unnoticed assumptions, hidden issues, constraints, etc.
07/09/09
©USC-CSE

30. WikiWinWin: Identify and Resolve Issues
07/09/09
©USC-CSE

31. Initial VBSE Theory: 4+1 Process, Step 5 – With a great deal of concurrency and backtracking
07/09/09
©USC-CSE

32. Project Strategy and Partnerships
Partner with eServices, Inc. for order processing and fulfillment system
Profit sharing using jointly-developed business case
Partner with key distributors to provide user feedback
Evaluate prototypes, beta-test early versions, provide satisfaction ratings
Incremental development using MBASE/RUP anchor points
Life Cycle Objectives; Architecture (LCO; LCA)
Core Capability Drivethrough (CCD)
Initial; Full Operational Capability (IOC; FOC)
Architect for later supply chain extensions
07/09/09
©USC-CSE

33. Business Case Analysis
Estimate costs and schedules
COCOMO II and/or alternative for software
PRICE H or alternative for hardware
COSYSMO for systems engineering
Estimate financial benefits
Increased profits
Reduced operating costs
Compute Return on Investment
ROI = (Benefits – Costs) / Costs
Normalized to present value
Identify quantitative metrics for other goals
Customer satisfaction ratings
Ease of use; In-transit visibility; overall
Late delivery percentage
07/09/09
©USC-CSE

34. Order Processing System Schedules and Budgets
Milestone
Due Date
Budget ($K)
Cumulative Budget ($K)
Inception Readiness
1/1/2004
Life Cycle Objectives
1/31/2004
120
Life Cycle Architecture
3/31/2004
280
400
Core Capability Drivethrough
7/31/2004
650
1050
Initial Oper. Capability: SW
9/30/2004
350
1400
Initial Oper. Capability: HW
2100
3500
Developed IOC
12/31/2004
500
4000
Responsive IOC
3/31/2005
4500
Full Oper. Cap’y CCD
7/31/2005
700
5200
FOC Beta
9/30/2005
5600
FOC Deployed
12/31/2005
6000
Annual Oper. & Maintenance
3800
Annual O&M; Old System
7600
07/09/09
©USC-CSE

35. Order Processing System: Expected Benefits and Business Case
07/09/09
©USC-CSE

36. Initial VBSE Theory: 4+1 Process, Steps 6 and 7 – With a great deal of concurrency and backtracking
07/09/09
©USC-CSE

37. A Real Earned Value System
Current “earned value” systems monitor cost and schedule, not business value
Budgeted cost of work performed (“earned”)
Budgeted cost of work scheduled (“yearned”)
Actual costs vs. schedule (“burned”)
A real earned value system monitors benefits realized
Financial benefits realized vs. cost (ROI)
Benefits realized vs. schedule
- Including non-financial metrics
Actual costs vs. schedule
07/09/09
©USC-CSE

38. Value-Based Expected/Actual Outcome Tracking Capability
07/09/09
©USC-CSE

39. Conclusions Current SE methods are basically value-neutral
Value-neutral SE methods are increasingly risky
VBSE agenda making progress, but major challenges remain
Evolving VBSE theory
Creating VB counterparts for value-neutral SE methods
07/09/09
©USC-CSE

40. References - I
C. Baldwin & K. Clark, Design Rules: The Power of Modularity, MIT Press, 1999.
S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Gruenbacher (eds.), Value-Based Software Engineering, Springer, 2005.
B. Boehm, “Value-Based Software Engineering,” ACM Software Engineering Notes, March 2003.
B. Boehm, C. Abts, A.W. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece, Software Cost Estimation with COCOMO II, Prentice Hall, 2000.
B. Boehm and L. Huang, “Value-Based Software Engineering: A Case Study, Computer, March 2003, pp
B. Boehm & K. Sullivan, “Software Economics: A Roadmap,” The Future of Software Economics, A. Finkelstein (ed.), ACM Press, 2000.
B. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed, Addison Wesley, 2003 (to appear).
B. Boehm, L. Huang, A. Jain. R. Madachy, “ The ROI of Software Dependability: The iDAVE Model”, IEEE Software Special Issue on Return on Investment, May/June 2004.
M. Denne and J. Cleland-Huang, Software by Numbers, Prentice Hall, 2004.
07/09/09
©USC-CSE

41. References - II
S. Faulk, D. Harmon, and D. Raffo, “Value-Based Software Engineering (VBSE): A Value-Driven Approach to Product-Line Engineering,” Proceedings, First Intl. Conf. On SW Product Line Engineering, August 2000.
R. Kaplan & D. Norton, The Balanced Scorecard: Translating Strategy into Action, Harvard Business School Press, 1996.
D. Reifer, Making the Software Business Case, Addison Wesley, 2002.
K. Sullivan, Y. Cai, B. Hallen, and W. Griswold, “The Structure and Value of Modularity in Software Design,” Proceedings, ESEC/FSE, 2001, ACM Press, pp
J. Thorp and DMR, The Information Paradox, McGraw Hill, 1998.
S. Tockey, Return on Software, Addison Wesley, 2004.
Economics-Driven Software Engineering Research (EDSER) web site:
MBASE web site : sunset.usc.edu/research/MBASE
07/09/09
©USC-CSE