Uncertainty in transport models

Slides:



Advertisements
Similar presentations
McGraw-Hill/Irwin Copyright © 2013 by The McGraw-Hill Companies, Inc. All rights reserved. A PowerPoint Presentation Package to Accompany Applied Statistics.
Advertisements

HOW DO LAND USE TRENDS AFFECT CBA OUTCOME? Peter Almström Svante Berglund Maria Börjesson Daniel Jonsson 27 november 2009, 1.
Interim Guidance on the Application of Travel and Land Use Forecasting in NEPA Statewide Travel Demand Modeling Committee October 14, 2010.
Assessing Uncertainty when Predicting Extreme Flood Processes.
Design of Experiments Lecture I
for Traffic Forecast Analysis, London | TIS.PT – Transportes Inovação e Sistemas, S.A. Slide 1 | 29 Transportes, Inovação e Sistemas,
Exploring uncertainty in cost effectiveness analysis NICE International and HITAP copyright © 2013 Francis Ruiz NICE International (acknowledgements to:
Correlation and regression
Sensitivity and Scenario Analysis
Decision Making: An Introduction 1. 2 Decision Making Decision Making is a process of choosing among two or more alternative courses of action for the.
Session 11: Model Calibration, Validation, and Reasonableness Checks
Marakas: Decision Support Systems, 2nd Edition © 2003, Prentice-Hall Chapter Chapter 4: Modeling Decision Processes Decision Support Systems in the.
Simple Linear Regression
Simulation.
GreenSTEP Statewide Transportation Greenhouse Gas Model Cutting Carbs Conference December 3, 2008 Brian Gregor ODOT Transportation Planning Analysis Unit.
Robert M. Saltzman © DS 851: 4 Main Components 1.Applications The more you see, the better 2.Probability & Statistics Computer does most of the work.
Decision analysis and Risk Management course in Kuopio
13 th TRB Transportation Planning Applications Conference May 11, 2011 Risk Assessment & Sensitivity Analysis of Traffic and Revenue Projections for Toll.
Estimating Risks Associated with Travel Demand Forecasts 14 th TRB Transportation Planning Applications Conference May 2013 Thomas Adler, RSG Michael Doherty,
1 D r a f t Life Cycle Assessment A product-oriented method for sustainability analysis UNEP LCA Training Kit Module k – Uncertainty in LCA.
Transit Estimation and Mode Split CE 451/551 Source: NHI course on Travel Demand Forecasting (152054A) Session 7.
Overview of Project Main objective of study is to assess the impact of delay at border crossings and resulting changes in user benefits and broad macroeconomic.
Lecture 12 Statistical Inference (Estimation) Point and Interval estimation By Aziza Munir.
Presentation Ling Zhang Date: Framework of the method 1 Using Distribution Fitting for Assumptions 2 monte – carlo simulation 3 compare different.
Modeling in the “Real World” John Britting Wasatch Front Regional Council April 19, 2005.
Transportation Planning, Transportation Demand Analysis Land Use-Transportation Interaction Transportation Planning Framework Transportation Demand Analysis.
Generic Approaches to Model Validation Presented at Growth Model User’s Group August 10, 2005 David K. Walters.
Comparing Dynamic Traffic Assignment Approaches for Planning
Client Name Here - In Title Master Slide Data Requirements to Support Road Pricing Analyses Johanna Zmud, Ph.D. NuStats Partners, LP Expert Forum on Road.
Chapter 4 MODELING AND ANALYSIS. Model component Data component provides input data User interface displays solution It is the model component of a DSS.
Various topics Petter Mostad Overview Epidemiology Study types / data types Econometrics Time series data More about sampling –Estimation.
Cost drivers, cost behaviour and cost estimation
MBA7025_01.ppt/Jan 13, 2015/Page 1 Georgia State University - Confidential MBA 7025 Statistical Business Analysis Introduction - Why Business Analysis.
Dynamic Origin-Destination Trip Table Estimation for Transportation Planning Ramachandran Balakrishna Caliper Corporation 11 th TRB National Transportation.
Handbook on Residential Property Price Indices Chapter 5: Methods Jan de Haan UNECE/ILO Meeting, May 2010.
MBA7020_01.ppt/June 13, 2005/Page 1 Georgia State University - Confidential MBA 7020 Business Analysis Foundations Introduction - Why Business Analysis.
SHRP2 C10A Final Conclusions & Insights TRB Planning Applications Conference May 5, 2013 Columbus, OH Stephen Lawe, Joe Castiglione & John Gliebe Resource.
Presented to Time of Day Subcommittee May 9, 2011 Time of Day Modeling in FSUTMS.
Comparative Analysis of Traffic and Revenue Risks Associated with Priced Facilities 14 th TRB National Transportation Planning Applications Conference.
Lesson 4 ODOT Best Practices. Best Practices Our Challenge: Best Practices for Action-Focused and Resource-Focused analyses are distinct Distinct best.
© English Matthews Brockman Business Planning in Personal Lines using DFA A Talk by Mike Brockman and Karl Murphy 2001 Joint GIRO/CAS Conference.
Risk Analysis in Capital Budgeting. Nature of Risk Risk exists because of the inability of the decision-maker to make perfect forecasts. the risk associated.
1 1 Slide Simulation Professor Ahmadi. 2 2 Slide Simulation Chapter Outline n Computer Simulation n Simulation Modeling n Random Variables and Pseudo-Random.
The World Bank Toll Road Revenue Forecast Quality Assurance/Quality Control.
Uncertainty and Reliability Analysis D Nagesh Kumar, IISc Water Resources Planning and Management: M6L2 Stochastic Optimization.
Matrix Models for Population Management & Conservation March 2014 Lecture 10 Uncertainty, Process Variance, and Retrospective Perturbation Analysis.
ATDM Analytical Methods for Urban Streets Urban Streets Subcommittee Meeting January 10, 2016 David Hale.
1 Life Cycle Assessment A product-oriented method for sustainability analysis UNEP LCA Training Kit Module k – Uncertainty in LCA.
Transportation Planning Asian Institute of Technology
Quantify Uncertainty in Travel Forecasts
Uncertainty Analysis in Emission Inventories
HVAC-4 Deemed Measures Uncertainty Study, Year 2
Prepared by Lloyd R. Jaisingh
Chapter 10 Verification and Validation of Simulation Models
Chapter 4. Modeling Transportation Demand and Supply
Professor S K Dubey,VSM Amity School of Business
Travel Demand Forecasting: Mode Choice
Chapter 10 - Monte Carlo Simulation and the Evaluation of Risk
Introduction to Instrumentation Engineering
Choice of Methods and Instruments
Trip Generation II Meeghat Habibian Transportation Demand Analysis
Price and Volume Measures
Trip Generation II Meeghat Habibian Transportation Demand Analysis
Decision and Risk Analysis
Estimation Error and Portfolio Optimization
2009 Minnesota MPO Conference August 11, 2009
Evaluating Impacts: An Overview of Quantitative Methods
Poverty and Social Impact Analysis: a User’s Guide – Economic tools
A Stochastic Approach to Occupant Pre-Movement in Fires
Introduction to Decision Sciences
Presentation transcript:

Uncertainty in transport models IDA workshop 7th May 2014

Presentation outline Rationale Uncertainty in transport models How uncertainty is quantified Case study General comments Uncertainty in transport models 07-05-2014

Rationale Transport models consist of equations combining exogenous variables (input) and coefficient (parameters) that express how the model output depends on the exogenous variables (De Jong et al. 2007) Transport models play a prominent role in many decision making processes; the output they generate, transport demand, is a key input for a wide range of analyses including: Cost Benefit Analysis (CBA) / Multi Criteria Analysis (MCA), e.g.: urban development planning strategy, sustainable mobility polices evaluation, etc. financial analysis for new (tolled) infrastructures Thus, wrong transport demand (traffic) forecast might have considerable socio/economic consequences (e.g. misplacement of public resources) Uncertainty in transport models 07-05-2014

Rationale Transport models have been criticized for lack of accuracy; an extensive literature has shown that often there is a considerable inaccuracy between forecasted and observed traffic volumes Source: Flyvbjerg et al. 2006 Uncertainty in transport models 07-05-2014

Rationale The main sources of inaccuracy can be broadly divided between technical and political/decision process Inaccuracy Technical Uncertainty Political Project selection positive bias toward projects with high forecasted traffic Big projects, more difficult to assess, preferred to small ones Disagreement among stakeholders might cause delays making the model results not up to date Standard model used for a problem it is not developed for, then overlooking core causal relationships Wrong assumptions on the transport system, e.g. (i) expectations on how competitors would react to new fare and service (strategy ) (ii) other projects being postponed (network) Uncertainty in transport models 07-05-2014

Rationale Transport models have inherent uncertainty : reproduce complex systems generating transport supply (i.e. services, infrastructure, regulation) and transport demand (i.e. traffic) receive inputs from other complex system (models): economics, sociology, psychology, technology, etc. The consequence is that transport models output is unpredictable, so it is not possible to use a deterministic approach In fact uncertainty can be defined as “any departure from the unachievable ideal of complete deterministic knowledge of the relevant system” (Walker et al. 2003) “(...) modelled output is better expressed as a central estimate and an overall range of uncertainty margins articulated in terms of values and likelihood of occurrence” (Boyce 1999), rather than as a point value Uncertainty in transport models 07-05-2014

Uncertainty in transport models Model definition Uncertainty related to the definition of the boundaries of the system being modelled (context) e.g. the geographical size of the area of interest, future changes in the transport demand environment Model specification Uncertainty about the assumed model structure and the technical implementation of the model (e.g. the methodology chosen) Model calibration Uncertainty about input (data collected) and parameters (calibrated variables) Model validation Uncertainty propagated to the transport model output (sequential frameworks) Adapted from Leleur 2000 Uncertainty in transport models 07-05-2014

How uncertainty is quantified Different levels of uncertainty require/allow different uncertainty calculation techniques: Determinism Statistical uncertainty Scenario uncertainty Recognised ignorance Ignorance (Indeterminacy) 1) Scenario hypoteses + sensitivity tests Adapted from Walker et al. 2003 2) Confidence intervals + sensitivity tests 3) Stochastic sampling methods + sensitivity tests (Monte Carlo Simulation, Bootstrap) Uncertainty in transport models 07-05-2014

How uncertainty is quantified Deterministic approach Scenario analysis Confidence intervals Stochastic simulation Uncertainty in transport models 07-05-2014

Case study: Næstved model Uncertainty analysis (model input and parameters) through MCS combined with sensitivity analysis Analysis focuses on: uncertainty propagation pattern effects on output uncertainty of uncertainty location (input, parameters and total) effects on output uncertainty of network congestion three levels of generated traffic (1,1.5 and 3 times base traffic) Stochastic sampling (multivariate LHS,100 draws): variables mean values: from the model variability: Coefficient of Variation (CV) 0.1 and 0.3 (CV=St Dev /Mean) probability distribution: input (Triangular) parameters (Log-normal) Sensitivity tests (100 runs) Uncertainty in transport models 07-05-2014

Case study: Næstved model Uncertainty in transport models 07-05-2014

Case study: Næstved model Uncertainty in transport models 07-05-2014

Case study: Næstved model Uncertainty in transport models 07-05-2014

Case study: NTM forecasts Uncertainty analysis (model socio-economic input growth rate forecasts 2015/2025) through MCS combined with sensitivity analysis : population, employment, GDP (real) and fuel prices (both petrol and diesel) Analysis focuses on: uncertainty propagation pattern over time aggregated and for different input Stochastic sampling (multivariate LHS,100 draws): variables mean values: official forecasts variability: standard deviations calculated by 5 years intervals from time series probability distribution: Normal distribution This approach is meant to reflect an increased level of uncertainty throughout the forecasted period Uncertainty in transport models 07-05-2014

Case study: NTM forecasts Uncertainty in transport models 07-05-2014

Case study: NTM forecasts Uncertainty in transport models 07-05-2014

Conclusions Transport models output, demand of transport, is a key input for policy decision making processes; problems arise from the fact that transport models refer and describe complex systems, thus showing unpredictability in their output The final output of the transport models is no more a point value but a range of possible outcome (with a given probability to occur) Assessing uncertainty inherent to transport models is important to produce informative output (demand of traffic) However, analyzing uncertainty of a whole transport model requires a relevant work effort; the question then is how expensive are decisions based on too uncertain forecasts and how time-consuming is implementation of uncertainties analyses? Uncertainty in transport models 07-05-2014

Extra Transport models uncertainty has a temporal dimension and occur in both static and dynamic (forecast) modelling However, with respect to the influence on the model output, the prevalence of some uncertainty location relatively to the others is affected by the temporal and geographical dimension of the system modelled The influence of model and parameters uncertainty is higher in static or short run forecasts whilst context and input uncertainty influence increases in long run forecasts. The same happens for geographical dimension of the area modelled, respectively small or big size Uncertainty in transport models 07-05-2014

Extra Uncertainty in transport models 07-05-2014

Calibration (parameters) Extra Definition Programming Calibration (input) Calibration (parameters) Validation Generation Subdivision into categories? Number and size of zones? Which socio economic variables? Model type? (Category analysis, linear regression, Logit, Probit) How does data and zone structure match? When is data collected? How well does data fit? How many parameters? Estimated, calibrated parameters? Are data representative? Model output (propagated) uncertainty Distribution Which variables describe travel resistance? Time intervals? Peak hours etc. Pivot table? Size of zones? Model type? (gravity, growth factor, Logit, Probit) Travel resistance function? Linear travel resistance function? Which matrix adjustment? How old is data? Is the generalized cost matrix the result of feedback from traffic assignment? Estimated or calibrated? Which calibration method? Mode choice Which modes? Is travel resistance included? Which variables are used Is it 2nd or 3rd model step? Model type? (Logit, Probit, mixed Logit) Is travel resistance a result of traffic assignment? Which characteristics are included? Assignment Which variables are included in the utility function? Different trip purposes? Different link types and speed-flow functions? Number of iterations with feed back? Linear utility function? Correlation? Model type? (Logit, Probit, mixed Logit, nested Logit) Deterministic or stochastic? Capacity constraint? Solution algorithm? Level of detail? Which characteristics are associated with the links? Uncertainty in transport models 07-05-2014

Case study: Næstved model - Extra Uncertainty in transport models 07-05-2014

Case study: Næstved model - Extra Uncertainty in transport models 07-05-2014