Uncertainty Analysis in Capital Investment Planning and Execution Carlos M. Alvarado, PE CH2M, Transportation Business Group Senior Financial Analyst, St. Louis, Missouri

Uncertainty affects key parts of the strategic planning process A Typical Strategic Planning Process Focus of this presentation Project Execution Investment Decisions Infrastructure Supporting Service Lines Project Plan Activities/Worksteps Month A M J J A S O N D J F M Program Management Service Lines I.1 Confirm stakeholders objectives II.1 Review, refine, and detail Prioritized Needs II.2 Develop Council key business III.1 Develop methodology to Tolling III.2 Map Masterplan aligning III.3 Develop telec III.4 Develop Vision Facilities III.5 Fine tune Tunnels Other Systems Risk Management Plan Develop Vision Based on Strategy Bridges Stations Program Charter # xx (Quarter 4 2005) Primary Objectives Deliverables Rail Vehicles Primary Objectives Program Charter # xx (Quarter 4 2005) Deliverables Guideway External Skills Bus (3-4 Max) Metrics Project Key External Skills Other (3-4 Max) Metrics Project Key Project Sponsor ICT Council Resources Council Project Manager ICT Council Project Sponsor Council Project Manager Council Other Operating Model

The purpose of the presentation is to describe the role of uncertainty when making investment decisions and during project execution Describe how to incorporate uncertainty principles into capital investment planning and execution 1 Discuss how to identify key variables how to apply uncertainty to these variables 2 Discuss how to mitigate project risk based on probabilistic cost overruns forecasts 3

Why is uncertainty analysis important? New Start requires an Alternatives Analysis and recommends the use uncertainty analysis for key “quantified measures” such as ridership or travel time savings1 It’s usually required by third-party auditors It’s just good practice May no longer be required to the same extent under FAST.

Let’s start with the capital investment decision Describe how to incorporate uncertainty principles into capital investment planning and execution 1 Discuss how to identify key variables how to apply uncertainty to these variables 2 Discuss how to mitigate transportation project risk based on probabilistic cost overruns forecasts 3

Should an investment be made? 1 EXAMPLE Toll Booths Open Road Tolling …for a toll bridge what might be the financial impact of going from toll booths to open road tolling?

Analysis of Alternatives Criteria and Scoring Financial Impact is a key criterion when comparing capital investment alternatives 1 ILLUSTRATIVE Analysis of Alternatives Criteria and Scoring Financial Operational Implementation Customer Needs Riskiness Baseline Alternative #1  Alternative #2 Rating Definitions: Alternative is far superior than Baseline Alternative is better than Baseline Alternative is comparable to Baseline Alternative is inferior to the Baseline Alternative is much more inferior than Baseline

The Financial impact is based on comparing the proposed Alternative(s) against the Baseline 1 Revenue Revenue Net financial impact Cost Cost Subtract Equals Proposed Alternative Baseline < 0 < Negative financial impact Positive financial impact

In the case of the open road tolling example, there will be uncertainty around some parameters 1 EXAMPLE Toll Booths Open Road Tolling Based on previous experience, possibilities include cost overruns, smaller-than-expected savings, others…

Uncertainty is modeled into the financial forecast and may have an impact on the results 1 Net Present Value Savings Impact for Cashless Tolling Project Waterfall Chart Results ILLUSTRATIVE Net impact might be negative (uncertainty) +$X4 +$X3 Uncertainty +$X2 +$X1 ($Y) ($Z) $0 Operational Expenses Savings Impact Revenue Loss Impact Capital Investment TOTAL

To model uncertainty a probabilistic approach is required 1 Impact of Uncertainty Analysis (Monte Carlo simulation) on Financial Calculations ILLUSTRATIVE Model Without Considering Uncertainty Considering Uncertainty Key Parameter 1 Point estimate Triangular distribution Key Parameter 2 Point estimate Normal distribution Output Point estimate solution Cumulative probability distribution Output Conclusion Model output misleadingly suggests certainty (e.g., revenue is $xx) Output is qualified by likelihood of occurrence (e.g., 75% confident that revenue will be more than $xx)

Next, let’s discuss how to identify the key uncertainty variables and how to model them Describe how to incorporate uncertainty principles into capital investment planning and execution 1 Discuss how to identify key variables how to apply uncertainty to these variables 2 Discuss how to mitigate transportation project risk based on probabilistic cost overruns forecasts 3

To model uncertainty we rely on a Monte Carlo simulation tool Start with an Ordinary Financial Model…. …and then Apply Crystal Ball Add-In1 2 @Risk® is an alternative

These tools run a useful diagnostic to reveal the most critical variables through a “tornado” chart Impact of Key Model Parameters on Overall Project Costs 2 ILLUSTRATIVE Tornado charts illustrate how each key parameter impacts results while other key variables assumptions are kept at their base (expected) value

In the case of the open toll example, the key financial parameter was revealed to be “leakage” 2 EXAMPLE Toll Booths Open Road Tolling Leakage is the volume of vehicles that pass through the open road tolled bridge but are not charged a toll

Likelihood Distributions That Can Be Assigned to Key Variables To model uncertainty, apply a suitable distribution function to the key financial parameters… 2 Likelihood Distributions That Can Be Assigned to Key Variables

Cumulative Distribution Results …which will reveal the likelihood that the net financial impact of implementing an alternative is within management’s threshold 2 Cumulative Distribution Results ILLUSTRATIVE What is the likelihood that the financial impact of implementing alternative will be negative?

Finally, let’s discuss how to incorporate uncertainty principles while executing a project Describe how to incorporate uncertainty principles into capital investment planning and execution 1 Discuss how to identify key variables how to apply uncertainty to these variables 2 Discuss how to mitigate transportation project risk based on probabilistic cost overruns forecasts 3

Project execution is the last part of the strategic planning process 3 Strategic Planning Process Focus of this section Project Execution Investment Decisions Infrastructure Supporting Service Lines Project Plan Activities/Worksteps Month A M J J A S O N D J F M Program Management Service Lines I.1 Confirm stakeholders objectives II.1 Review, refine, and detail Prioritized Needs II.2 Develop Council key business III.1 Develop methodology to Tolling III.2 Map Masterplan aligning III.3 Develop telec III.4 Develop Vision Facilities III.5 Fine tune Tunnels Other Systems Risk Management Plan Develop Vision Based on Strategy Bridges Stations Program Charter # xx (Quarter 4 2005) Primary Objectives Program Charter # xx (Quarter 4 2005) Deliverables Rail Vehicles Primary Objectives Deliverables Guideway External Skills Bus (3-4 Max) Metrics Project Key External Skills Other (3-4 Max) Metrics Project Key Project Sponsor ICT Council Resources Council Project Manager ICT Council Project Sponsor Council Project Manager Council Other Operating Model

During execution, want to avoid cost overruns which are usually tied to risks inherent in project execution 3 PMI’s PMBOK® recommends a formal risk management process Risk Management Process Overview Risk Management Planning Risk Response Planning Risk Monitoring & Control Risk Identification Risk Analysis

A key output of the risk management process is a Risk Register 3 Typical Components of Risk Register Category Responsible Party Risk Current Status Risk Statement Risk Criticality Source of Risk Risk Likelihood Consequence of Risk Risk Handling Strategy Date Identified Other Date Retired

Risk Register Cost Impact Expected Additional Cost Due to Risks For our purposes, we are most interested on the cost impact of the risks on the register 3 Risk Register Cost Impact ILLUSTRATIVE Risk Title Risk Statement Current Status Probability % Cost Impact ($) Low High Most Likely Cost1 Probable Impact2 Risk 1 Utility relocation... Active 30 50 1.5M 10.0M 12.0M 4.0M Risk 2 Claims for delays… 1.0M 3.0M … …… …. .. Risk N Statement N XX X.XM Expected Additional Cost Due to Risks XX.XM Field where uncertainty analysis distribution is applied. In this case, apply triangular distributions. Field where probability distribution “forecast” are calculated.

A typical uncertainty analysis output shows the probability of costs not exceeding a certain amount 3 Cumulative Frequency Chart ILLUSTRATIVE 70% probability is a common threshold Probability of “not exceeding”

The cumulative frequency output can be used to assess potential project cost overruns 3 ILLUSTRATIVE In this example, 43% of scenarios the likely cost is higher than the total resources allocated…. …resulting in a shortfall of contingency funds in some instances 43% of scenarios shortfall 57% of scenarios

In summary, I believe uncertainty analysis might be useful for transportation planners Acknowledges that the success of a project may depend on parameters that cannot be predicted with certainty (e.g., ridership, revenues, travel time savings, etc.) Reveals and focuses management effort on the most critical variables / parameters (e.g., leakage from open road tolling example) Helps keep close tabs on risk that may result in cost overruns (and/or delays) during project execution

Contact information Carlos M. Alvarado, PE Senior Financial Analyst CH2M Transportation Business Group carlos.alvarado@ch2m.com (703) 342-9088

Extra slides

To compare the Baseline against the Alternative, define a revenue/cost element structure 1 ILLUSTRATIVE Financial Impact Analysis – Revenue & Cost Element Structure Unit Pricing Revenue Element Quantity Revenues Revenue Element Revenue Element Baseline or New Alternative Labor Materials Op Ex Costs Outsourced Contract Cost Element Capital Investment CapEx Costs Capital Investment