Risk Analysis 19 th September, 2014

Risk vs. Problem Risk  An “uncertain event that “may” happen with some probability Problem  The above event “has” happened! (i.e., the risk materialized)  Or something with 100% probability of occurrence

Reactive Management System Integration Time A and B don’t talk to each other Too pre-occupied working on A & B Solution? – Write wrappers – Make them talk (using CORBA, ESB etc.,) Result: – 3 month schedule overrun = $100K contract penalty – $300K cost overrun

Proactive Management (Risk) A and B is what we’ll be using Chance they won’t talk to each other: – Probability won’t talk = P(Loss) = 50% (from past experience) If found they can’t talk at integration: – Size of Loss = S(L) = $300K + $100K = $400K Risk Exposure: – RE = P(L) * S(L) = 0.5 * $400K = $200K

Risk vs. Opportunity RiskOpportunity RE = Probability(Loss) * Size(Loss)OE = Probability(Gain) * Size(Gain)

Top 10 Risks: 1989 vs Personnel shortfalls 2. Schedules and budgets 3. Wrong software functions 4. Wrong user interface 5. Gold plating 6. Requirements changes 7. Externally-furnished components 8. Externally-performed tasks 9. Real-time performance 10. Straining computer science Personnel shortfalls 2. Schedules, budgets, process 3. COTS, external components 4. Requirements mismatch 5. User interface mismatch 6. Architecture, performance, quality 7. Requirements changes 8. Legacy software 9. Externally-performed tasks 10. Straining computer science

Top 10 Risks: 1995 vs Personnel shortfalls1. Customer-developer-user team cohesion 2. Schedules, budgets, process2. Personnel shortfalls 3. COTS, external components3. Architecture complexity; quality tradeoffs 4. Requirements mismatch4. Budget and schedule constraints 5. User interface mismatch5. COTS and other independently evolving systems 6. Architecture, performance, quality6. Lack of domain knowledge 7. Requirements changes7. Process Quality Assurance 8. Legacy software8. Requirements volatility; rapid change 9. Externally-performed tasks9. User interface mismatch 10. Straining computer science10. Requirements mismatch

Top 10 Risks: Inflated expectations 2.Success-critical stakeholder’s lack of involvement or value conflicts 3.Under-defined plans and requirements 4.Architecture/reuse/non-developmental item (ND) conflicts 5.Personnel shortfalls 6.Immature or obsolete processes; unbridled requirements volatility 7.Human-system integration shortfalls 8.Legacy asset incompatibilities 9.Unbalanced nonfunctional requirements (-ilities) 10.Immature Technology

Primary Risks in 577ab

Personnel Shortfalls Commitment (this is team member’s last course; only needs C to graduate) Compatibility Communication problems Skill deficiencies (management, web design, Java, Perl, CGI, …)

Schedule Project scope too large for 24 weeks Initial Operational Capability (IOC) scope/content too big Critical path items that can cause delay: – COTS – Platforms – ARBs Requirements/UI mismatch

COTS and External Components COTS – Immaturity of framework – Inexperience of team members – Incompatibility with application, platform or other COTS – Lack of control over COTS development GOTS/ROTS – Compatibility analysis needs to be done – Benchmarking – Inspections – Reference checking

Understanding Risk Management How to do it?

Stakeholder Risk Profiles Understanding & Addressing People’s Utility Functions – Risk Averse – Risk Neutral – Risk Seeking  Implication(s): Very people oriented process Continuous and Iterative Must have plans/processes in place to identify, manage and mitigate risks Utility Benefit 0,0 1,1 Risk averse for losses and Risk Seeking for Gains It is possible to have negative utilities  Losses 19

Risk/Opportunity Assessment Identify relevant risks – Checklists, creativity, top-down decomposition, … Analyze – Probability of Loss/Gain – Size of Loss/Gain Prioritize – Risk Exposure – Risk Reduction Leverage (RRL)

Risk Reduction Leverage Gives the ROI of the “countermeasures” being considered for mitigating risk Computed as: Useful for prioritizing various mitigation strategies =

Risk Mitigation Buying Information – Gain more insight by prototyping, exploring further etc., or actually purchasing something Risk Avoidance – Remove risk from critical path of project Risk Transfer – From my head to yours OR – Share risk Risk Reduction – Lower probability or magnitude of loss Risk Acceptance – Accept and move on (assuming risk is manageable)

Risk Analysis: Example Unsatisfactory Outcome (UO) Prob (UO) Loss (UO) Risk Exposure A. S/ W error kills experiment B. S/ W error loses key data C. Fault tolerance features cause unacceptable performance D. Monitoring software reports unsafe condition as safe E. Monitoring software reports safe condition as unsafe F. Hardware delay causes schedule overrun G. Data reduction software errors cause extra work H. Poor user interface causes inefficient operation I. Processor memory insufficient J. DBMS software loses derived data

Risk Management Plans A “lean” plan with the following outline: 1.Objectives (Why?) 2.Deliverables and milestones (What and When?) 3.Responsibilities (Who and Where?) 4.Approach (How?) 5.Resources (How much?) Detailed example: Pg. 246 ICSM Textbook

Software Risk Management Techniques Source of Risk (1995)Risk Management Techniques 1. Personnel shortfallsStaffing with top talent; key personnel agreements; team- building; training; tailoring process to skill mix; walkthroughs 2. Schedules, budgets, Process Detailed, multi-source cost and schedule estimation; design to cost; incremental development; software reuse; requirements descoping; adding more budget and schedule; outside reviews 3. COTS, external components Benchmarking; inspections; reference checking; compatibility prototyping and analysis 4. Requirements mismatch Requirements scrubbing; prototyping; cost-benefit analysis; design to cost; user surveys 5. User interface mismatch Prototyping; scenarios; user characterization (functionality; style, workload); identifying the real users ©USC-CSSE26

Software Risk Management Techniques Source of Risk (1995)Risk Management Techniques 6. Architecture, performance, quality Simulation; benchmarking; modeling; prototyping; instrumentation; tuning 7. Requirements changesHigh change threshold: information hiding; incremental development (defer changes to later increments) 8. Legacy softwareReengineering; code analysis; interviewing; wrappers; incremental deconstruction 9. COTS, Externally- performed tasks Pre-award audits, award-fee contracts, competitive design or Prototyping 10. Straining computer science Technical analysis; cost-benefit analysis; prototyping; reference checking ©USC-CSSE27

Validation Results on Process Adoption Incidents of Process Selection and Direction Changes ©USC-CSSE28 #of teamsResults on Project Process Selection 8/14Selected the right process pattern from the beginning 3/14Unclear project scope ; re-select right at the end of the Exploration phase 1/14Minor changes on project scope ; right at the end of the Valuation phase 1/14Major change in Foundations phase 1/14Infeasible project scope

Key Takeaways Risk management starts on day one! ICSM provides a process framework for early risk resolution – Stakeholder identification – WinWin Reconciliation – Anchor point milestones/reviews Risk Analysis helps determine “how much is enough” – How much planning, prototyping, designing, testing, formalizing etc., Fairly simple yet highly valuable! Various tools/techniques exists in the wild