CS 577b Software Engineering II -- Introduction 24 April 2017 Risk Management CS 510, Fall 2014 Barry Boehm © 2002-6 USC Center for Software Engineering

Outline Risk Management Case Studies Agile Risk Management

Importance of Risk Management Avoiding Disasters 80-20 rule Avoiding Rework Rework of erroneous requirements, design, code typically consumes 40-50% of the total costs of software development Prime opportunity to improve productivity Avoiding Overkill Focus effort on the critical areas that make a difference Stimulating Win-Win Situations “Best and final offer” negotiation on software contracts leads to cheapest wins

USC-CSSE Top-10 Risk Delphi, 2014 Unrealistic budgets, schedules (94) Inadequate or misunderstood requirements (92) Misaligned stakeholders (62) Bad architectural decisions (50) Bad NDI (COTS, clouds, open source) decisions (45) Personnel shortfalls (43) Weak or no analysis of alternatives (33) Unresolved quality-attribute conflicts (27) Immature technology (26) Lack of customer/CONOPS understanding (20)

Straining Computer Science Capabilities (1989) Distributed Processing Artificial Intelligence Domains Human-machine performance Algorithm speed and accuracy Information privacy and security protection High reliability and fault tolerance

Decision-Driver If decisions had been driven by factors other than technical and management achievability, frequently be the source of critical software-risk item. Politically driven decisions Choice of equipment, subcontractor, schedule & budget, allocation of responsibilities Marketing-driven decisions Gold plating, choice of eq, schedule & budget

Decision – Driver (cont) Solution-driven vs problem-driven decisions In-house components and tools, product champion Short-term vs Long-term decisions Staffing – available basis rather than the qualified ones Software reuse – often flaky and incompatible Premature review – fixed review date

Pareto 80-20 phenomena 80% of the contribution comes from 20% of the contributors 20% of the modules contribute 80% of the cost 20% of the modules contribute 80% of the errors 20% of the errors cause 80% of the down time 20% of the errors consume 80% of the cost to fix 20% of the modules consume 80% of the execution time

Uncertainty Areas and unresolved questions Mission requirements Vague top-level mission statements but no clear definition Life-cycle concept of option Boundaries between maintenance and incremental development or preplanned product improvement System performance drivers Where to measure (CPU, main memory, communication) UI characteristics IKIWISI Interfacing system characteristics Sharing common resources? Common modules?

Dealing with compound risks Pushing technology on more than one front Too many new technology at the same time Pushing technology with key staff shortages Meeting vague user requirements on an ambitious schedule Untried hardware with an ambitious schedule Unstable interfaces with an untried subcontractor

Make or Buy Decision Analysis COST EASY $1,200K 0.4 = (0.4*1200K) + (0.6*1800K) = $1,560K HARD 0.6 $1,800K BUILD SMALL MOD $700K Geographic DBMS REUSE 0.5 =$1,450K 0.5 EASY 0.3 $1,500K LARGE MOD BUY HARD $2,500K 0.7 SMALL MOD 0.6 $1,000K =$1,400K 0.4 LARGE MOD $2,000K

Do IV&V or not? $1.3M $2M $0.18M $0.82 M $0.30 M $0 $20M Size(Loss) = $0.5 M P(Loss) = 0.36, Find CE 0.04 Don’t Find CE $20.5M DO IV&V 0.6 NO CE $0.5M NO IV&V 0.3 Find CE Size(Loss) = $0 M 0.1 Don’t Find CE $20M 0.6 NO CE $0M CE = Critical Error

Opportunity Tree Vendor A or Vendor B Impact: positive $ = gain, negative $ = loss http://getpmpcertified.blogspot.in/2013/02/expected-monetary-value-analysis-emv.html

VALUE Fix or rebuild http://www.mindtools.com/dectree.html

VALUE http://www.mindtools.com/dectree.html

http://mosaic.cnfolio.com/B302CW2009B152

Risk Reduction Leverage (RRL) BEFORE - AFTER RISK REDUCTION COST Spacecraft Example LONG DURATION TEST FAILURE MODE TESTS LOSS (UO) PROB (UO) RE $20M 0.2 $4M $20M 0.2 $4M B B PROB (UO) RE 0.05 $1M 0.07 $1.4M A A COST $2M $0.26M 4-1 4- 1.4 = 1.5 = 10 RRL 2 0.26 10/14/05 ©USC-CSE

Risk Assessment Tool

Top 10 tips for risk management 10. Identify and assess risks 9. Know the numbers 8. Risks are interrelated, recognize the relationship 7. Continually reassess risks 6. Commit adequate resources 5. Review the cost of risk mitigation 4. Reduce exposure 3. Assess the risk/return ratio 2. Monitor the quantum shifts in risk levels 1. Create a risk aware culture http://www.risklimited.com/risk-tips.htm

Risk/Return Ratio Let's say a trader purchases 100 shares of XYZ Company at $20 and places a stop-loss order at $15 to ensure that her losses will not exceed $500. Let's also assume that this trader believes that the price of XYZ will reach $30 in the next few months. In this case, the trader is willing to risk $5 per share to make an expected return of $10 per share after closing her position. Since the trader stands to make double the amount that she has risked, she would be said to have a 1:2 risk/reward ratio Purchase: 100 shares * $20 Stop-loss at $15; expectation at $30 Risk $5 to make $10 1:2 risk reward ratio

Agile Risk Burn-down chart http://www.mountaingoatsoftware.com/blog/managing-risk-on-agile-projects-with-the-risk-burndown-chart http://www.scrumalliance.org/community/articles/2013/2013-may/risk-management-in-agile

Agile Risk Policy Do risky stuff first and fail early http://www.guerillaprojectmanagement.org/2009/01/beneficiile-agile-pentru-project.html

Sample risks from risk register

Potential risk – Product Size Number of users of the product? Number of projected changes to the requirements for the product? Before delivery? After delivery? http://groups.engin.umd.umich.edu/CIS/course.des/cis375/projects/risktable/risks.htm

Business impact risks Affect of this product on company revenue? Governmental constraints on the construction of the product? Costs associated with late delivery? Costs associated with a defective product? http://groups.engin.umd.umich.edu/CIS/course.des/cis375/projects/risktable/risks.htm

Customer related risks Does the customer have a solid idea of what is required? Does the customer understand the software engineering process? http://groups.engin.umd.umich.edu/CIS/course.des/cis375/projects/risktable/risks.htm

Development environment risks Are all the software tools integrated with one another? Are testing tools available and appropriate for the product to be built? http://groups.engin.umd.umich.edu/CIS/course.des/cis375/projects/risktable/risks.htm

Process issue risks Are staff members signed-up to the software process as it is documented and willing to use it? Are the results of each formal technical review documented, including defects found and resources used? http://groups.engin.umd.umich.edu/CIS/course.des/cis375/projects/risktable/risks.htm