Risk Management Lucas Taylor Fermilab Risk Manager 3 rd February Director's Review -- Risk Management L. Taylor, 3rd February 2016

Risk Management Lucas Taylor Fermilab Risk Manager 3 rd February Director's Review -- Risk Management L. Taylor, 3rd February 2016 Terminology Risk: Event with a probability that can have impacts on the project objectives Threat: risk with negative impacts Opportunity: risk with positive impacts Risk Management: management of uncertainty to help achieve objectives, by minimizing probabilities and impacts of threats and maximizing opportunities

 Particle Physicist (PhD)  CMS, L3, Pierre Auger Observatory, UA1  Higgs discovery, (g-2) of tau, tau neutrino mass, Vtb, rare decays, b-lifetime, mixing, hadron production …  Project Management Professional (PMP)  CMS detector upgrades, CMS Head of Communications, construction of Control Rooms, LHC Grid, CMS Software… Current roles:  Fermilab Risk Manager  Initiated Fermilab-wide Enterprise Risk Management program  Standardized the Fermilab project risk procedures  Established common Fermilab risk register and risk analysis tools  Performed risk workshops, risk analysis, MC modeling, reviews: CMS Phase 1 and HL-LHC Upgrades, LBNF/DUNE, Mu2e...  Deputy Project Manager, CMS Phase 1 Upgrades 3 Lucas Taylor Director's Review -- Risk Management L. Taylor, 3rd February 2016

 Introduction  Risk Management Process Including:  Identification of high-level risks  Analysis of risk impacts  Cost and schedule contingency  Next steps  Summary 4 Outline Director's Review -- Risk Management L. Taylor, 3rd February 2016 Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

ISO ANSI PMBOK DOE NSF  Project follows the “Fermilab Risk Management Procedure for Projects” (link) – consistent with:link  ANSI PMBOK methodology  NSF Large Facilities Manual  DOE Risk Management Guide Fermilab Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve 5 Basis of Risk Management Process Director's Review -- Risk Management L. Taylor, 3rd February 2016

 Scope contingency  ~10% of scope to be identified as scope contingency Defer work until we’re sure the budget and schedule allow it  Careful choice of threshold KPPs and objective KPPs Example: threshold KPP for detector delivery to CERN (de- couple from LHC schedule!) and objective KPP for installation  Cost estimate uncertainty contingency  M&S and labor cost risk – depends on design maturity  Estimate uncertainty contingency (as % of base cost)  Management of specific risks  Risks identified and analyzed  Project Risk Register  Mitigation and risk response plans are developed  Cost risk contingency from preliminary risk analysis  Schedule risk contingency from preliminary analysis 6 Risk Management Strategy Director's Review -- Risk Management L. Taylor, 3rd February 2016 PEP BoEs This talk Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 PM, L2s, Control Account Managers, SMEs  Risk identification, analysis, mitigations and response planning, risk monitoring and control  PM chairs the Risk Management Board (RMB)  Risk Manager  Coordinational role  Establishes risk processes, register, and tools  Performs risk modeling and contingency analysis  Lab Management and Funding Agencies  Advise project and monitor the risk process  Affirm decisions and use of contingency 7 Roles and Responsibilities Director's Review -- Risk Management L. Taylor, 3rd February 2016 Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 Initial risk identification has been performed  Focus on high-level, project-wide risks  Reviewed risks from previous, similar projects o Especially CMS Phase 1 Upgrades  Detector experts starting to assess detailed risks  15 high-level threats and 3 opportunities  Risks managed in Fermilab Risk Register tool  Lab-wide risk DB with (multi-user) web interface  Risk is characterized by o Unique ID, Title, Description, Project / WBS / Owner o Probabilities and impacts – technical, cost, schedule o Mitigations and response plans … 8 Risk Identification Director's Review -- Risk ManagementL. Taylor, 3rd February 2016 Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 Risk interviews in 2016 will identify lower rank, technical risks 9 Risk Identification Director's Review -- Risk Management L. Taylor, 3rd February high-level opportunities 15 high- level threats Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 Estimate risk probabilities and impacts, typically by parametric scaling from other, similar projects 10 Qualitative Risk Analysis Director's Review -- Risk Management L. Taylor, 3rd February 2016 Bins are tailored to the Project’s cost and duration Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 18 high-/medium-rank risks (15 threats and 3 opportunities) We will pursue the lower level risks through risk interviews and workshops in Qualitative Risk Analysis Director's Review -- Risk Management L. Taylor, 3rd February 2016

 14 of 18 high-/medium-level risks correlate across the Project 12 Qualitative Risk Analysis Director's Review -- Risk Management L. Taylor, 3rd February 2016

 Perform Monte Carlo (MC) modeling of risk events in the Project schedule to help  Identify cost and schedule risk drivers  Adjust schedule to eliminate or reduce risks  Assess overall impacts on Project objectives  Determine how much contingency is needed to deliver within budget and on time (at 90 % C.L.) o Schedule contingency (float) o Cost contingency for risks  We have experience in doing this …  Current MC analysis is simplistic but will mature as the resource-loaded schedule is developed 13 Quantitative Risk Analysis Director's Review -- Risk ManagementL. Taylor, 3rd February 2016 Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

Cost and schedule drivers Costs of risks and contingency Completion dates and float Examples from LBNF / DUNE 14 Quantitative Risk Analysis Director's Review -- Risk ManagementL. Taylor, 3rd February 2016 P6 Project Plan – project activities, in time-sequence. P6 DB P6 DB Full Schedule Full Schedule Summary Schedule Risk Model simulates risk events, with ranges of impacts, in project schedule Risk Model (Primavera Risk Analysis - PRA) Risk Model (Primavera Risk Analysis - PRA) Risk DB Risk DB Risk Register (Web tool) Risk Register (Web tool) DocDB More Details (documents) More Details (documents) Risk Register -- probabilities, impacts, and response plans

 Contingency (at 90 %C.L) for 18 risks analyzed = 8.4 M$  Estimated contingency for not-yet-analyzed risks = 4.2 M$  Assume ~1/3 of total as for CMS Phase 1 Upgrades  Total cost risk contingency (DOE + NSF) = 12.6 M$  NSF cost risk contingency (assume ~36% of total) = 4.5 M$ 15 Quantitative Risk Analysis Director's Review -- Risk Management L. Taylor, 3rd February 2016 Run MC for 10,000 iterations 1.Risks do or don’t happen according to probability 2.Choose cost and schedule impacts (single value / flat range / triangle function) 3.Re-compute the P6 schedule 4.Determine Project end date 5.Aggregate total cost of risks

 Without any risks, the Summary Schedule has:  Technical work complete:March 2026  Administrative closeout:September 2026  Simple risk MC aggregates delays from risks occurring in different phases to yield probability distribution of Project finish date  Including risks will delay completion by < 21 months at 90% C.L.  Administrative closeout is before July 2028 (at 90% C.L.)  Possible CD-4 date: September 2028  30 months of schedule contingency after technical work is complete  24 months of schedule contingency after administrative closeout 16 Quantitative Risk Analysis Director's Review -- Risk Management L. Taylor, 3rd February months

 Contingency estimates are consistent with similar projects 17 Quantitative Risk Analysis Director's Review -- Risk Management L. Taylor, 3rd February 2016

 Risk mitigation plans – pre-emptive actions to reduce risk probability and/or impact  Mitigation activities are incorporated in the project plan along with associated costs and durations  Risk response plans – actions taken only if a risk occurs, to reduce the impacts  Not part of the baseline plan  Costs and delays covered by risk contingency  Risk mitigation and response plans will be developed – and modeled – during 2016/17  Risk workshops with SMEs and management  Summarized in risk register, details in DocDB  Involve external experts to improve effectiveness 18 Risk Mitigation and Response Plans Director's Review -- Risk Management L. Taylor, 3rd February 2016 Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 Risk monitoring and control happens in the context of the Project Management Board  Risk is a key input to effective decision making  Changes to Project plans to deal with risks (and any use of contingency) follows the Project’s standard change control process  The PMB reports to, and coordinates risk handling with, the Lab, DOE and NSF  Project Management Group (PMG)  Project Oversight Group (POG)  Integrated Project Team (IPT) 19 Risk Monitoring and Control Director's Review -- Risk ManagementL. Taylor, 3rd February 2016 Plan Risk Management Identify risks Identify risks Develop risk responses Monitor and Control risks Monitor and Control risks Analyze risks Iterate / Communicate / Review / Improve

 Spring 2016 – complete the lower-level risk identification  Risk interviews with stakeholders  Review the Risk Breakdown Structure – look for gaps  Summer 2016 – Risk Workshop  Review entire risk register; capture any additional risks  Estimate risk probabilities and impacts (and document justifications)  Identify and document initial risk mitigation and response plans  Fall 2016 – perform quantitative MC risk analysis  Model risks in the context of the P6 project schedule  Improve project plan to reduce impacts of risks  Assess schedule sensitivity to duration estimate uncertainty  Determine indirect burn rate costs from delays caused by risks o Standing army costs and escalation  Determine cost and schedule contingency at 90% C.L. 20 Risk Management – Next Steps in 2016 Director's Review -- Risk Management L. Taylor, 3rd February 2016

 The Project has established a solid risk management process  Follows the “Fermilab Risk Management Procedures for Projects”  Consistent with ANSI “PMBOK” and “NSF Large Facilities Manual”  High-level and cross-cutting risks have been identified  Risk probabilities and (cost and schedule) impacts have been estimated parametrically based on other projects’ experience  Impacts of all risks on Project objectives have been determined using a MC of risk events in the context of the project schedule  NSF cost contingency for risk is 4.5 M$ (~9% of base cost)  Schedule contingency for risk is 2.5 years (~25% of duration)  Contingency estimates are consistent with similar projects  Risk activities are well advanced for this stage of the Project and the required, upcoming risk work is already foreseen 21 Risk Management – Summary Director's Review -- Risk Management L. Taylor, 3rd February 2016