1. Uncertainty in transport models
IDA workshop 7th May 2014

2. Presentation outline Rationale Uncertainty in transport models
How uncertainty is quantified
Case study
General comments
Uncertainty in transport models

3. Rationale
Transport models consist of equations combining exogenous variables (input) and coefficient (parameters) that express how the model output depends on the exogenous variables (De Jong et al. 2007)
Transport models play a prominent role in many decision making processes; the output they generate, transport demand, is a key input for a wide range of analyses including:
Cost Benefit Analysis (CBA) / Multi Criteria Analysis (MCA), e.g.: urban development planning strategy, sustainable mobility polices evaluation, etc.
financial analysis for new (tolled) infrastructures
Thus, wrong transport demand (traffic) forecast might have considerable socio/economic consequences (e.g. misplacement of public resources)
Uncertainty in transport models

4. Rationale
Transport models have been criticized for lack of accuracy; an extensive literature has shown that often there is a considerable inaccuracy between forecasted and observed traffic volumes
Source: Flyvbjerg et al. 2006
Uncertainty in transport models

5. Rationale
The main sources of inaccuracy can be broadly divided between technical and political/decision process
Inaccuracy
Technical
Uncertainty
Political
Project selection positive bias toward projects with high forecasted traffic
Big projects, more difficult to assess, preferred to small ones
Disagreement among stakeholders might cause delays making the model results not up to date
Standard model used for a problem it is not developed for, then overlooking core causal relationships
Wrong assumptions on the transport system, e.g. (i) expectations on how competitors would react to new fare and service (strategy ) (ii) other projects being postponed (network)
Uncertainty in transport models

6. Rationale Transport models have inherent uncertainty :
reproduce complex systems generating transport supply (i.e. services, infrastructure, regulation) and transport demand (i.e. traffic)
receive inputs from other complex system (models): economics, sociology, psychology, technology, etc.
The consequence is that transport models output is unpredictable, so it is not possible to use a deterministic approach
In fact uncertainty can be defined as “any departure from the unachievable ideal of complete deterministic knowledge of the relevant system” (Walker et al. 2003)
“(...) modelled output is better expressed as a central estimate and an overall range of uncertainty margins articulated in terms of values and likelihood of occurrence” (Boyce 1999), rather than as a point value
Uncertainty in transport models

7. Uncertainty in transport models
Model definition
Uncertainty related to the definition of the boundaries of the system being modelled (context) e.g. the geographical size of the area of interest, future changes in the transport demand environment
Model specification
Uncertainty about the assumed model structure and the technical implementation of the model (e.g. the methodology chosen)
Model calibration
Uncertainty about input (data collected) and parameters (calibrated variables)
Model validation
Uncertainty propagated to the transport model output (sequential frameworks)
Adapted from Leleur 2000
Uncertainty in transport models

8. How uncertainty is quantified
Different levels of uncertainty require/allow different uncertainty calculation techniques:
Determinism
Statistical uncertainty
Scenario uncertainty
Recognised ignorance
Ignorance (Indeterminacy)
1) Scenario hypoteses + sensitivity tests
Adapted from Walker et al. 2003
2) Confidence intervals + sensitivity tests
3) Stochastic sampling methods + sensitivity tests (Monte Carlo Simulation, Bootstrap)
Uncertainty in transport models

9. How uncertainty is quantified
Deterministic approach
Scenario analysis
Confidence intervals
Stochastic simulation
Uncertainty in transport models

10. Case study: Næstved model
Uncertainty analysis (model input and parameters) through MCS combined with sensitivity analysis
Analysis focuses on:
uncertainty propagation pattern
effects on output uncertainty of uncertainty location (input, parameters and total)
effects on output uncertainty of network congestion three levels of generated traffic (1,1.5 and 3 times base traffic)
Stochastic sampling (multivariate LHS,100 draws):
variables mean values: from the model
variability: Coefficient of Variation (CV) 0.1 and 0.3 (CV=St Dev /Mean)
probability distribution: input (Triangular) parameters (Log-normal)
Sensitivity tests (100 runs)
Uncertainty in transport models

11. Case study: Næstved model
Uncertainty in transport models

12. Case study: Næstved model
Uncertainty in transport models

13. Case study: Næstved model
Uncertainty in transport models

14. Case study: NTM forecasts
Uncertainty analysis (model socio-economic input growth rate forecasts 2015/2025) through MCS combined with sensitivity analysis :
population, employment, GDP (real) and fuel prices (both petrol and diesel)
Analysis focuses on:
uncertainty propagation pattern over time aggregated and for different input
Stochastic sampling (multivariate LHS,100 draws):
variables mean values: official forecasts
variability: standard deviations calculated by 5 years intervals from time series
probability distribution: Normal distribution
This approach is meant to reflect an increased level of uncertainty throughout the forecasted period
Uncertainty in transport models

15. Case study: NTM forecasts
Uncertainty in transport models

16. Case study: NTM forecasts
Uncertainty in transport models

17. Conclusions
Transport models output, demand of transport, is a key input for policy decision making processes; problems arise from the fact that transport models refer and describe complex systems, thus showing unpredictability in their output
The final output of the transport models is no more a point value but a range of possible outcome (with a given probability to occur)
Assessing uncertainty inherent to transport models is important to produce informative output (demand of traffic)
However, analyzing uncertainty of a whole transport model requires a relevant work effort; the question then is how expensive are decisions based on too uncertain forecasts and how time-consuming is implementation of uncertainties analyses?
Uncertainty in transport models

18. Extra
Transport models uncertainty has a temporal dimension and occur in both static and dynamic (forecast) modelling
However, with respect to the influence on the model output, the prevalence of some uncertainty location relatively to the others is affected by the temporal and geographical dimension of the system modelled
The influence of model and parameters uncertainty is higher in static or short run forecasts whilst context and input uncertainty influence increases in long run forecasts. The same happens for geographical dimension of the area modelled, respectively small or big size
Uncertainty in transport models

19. Extra
Uncertainty in transport models

20. Calibration (parameters)
Extra
Definition
Programming
Calibration (input)
Calibration (parameters)
Validation
Generation
Subdivision into categories?
Number and size of zones?
Which socio economic variables?
Model type? (Category analysis, linear regression, Logit, Probit)
How does data and zone structure match?
When is data collected?
How well does data fit?
How many parameters?
Estimated, calibrated parameters?
Are data representative?
Model output (propagated) uncertainty
Distribution
Which variables describe travel resistance?
Time intervals? Peak hours etc.
Pivot table?
Size of zones?
Model type? (gravity, growth factor, Logit, Probit)
Travel resistance function?
Linear travel resistance function?
Which matrix adjustment?
How old is data?
Is the generalized cost matrix the result of feedback from traffic assignment?
Estimated or calibrated?
Which calibration method?
Mode choice
Which modes?
Is travel resistance included?
Which variables are used
Is it 2nd or 3rd model step?
Model type? (Logit, Probit, mixed Logit)
Is travel resistance a result of traffic assignment?
Which characteristics are included?
Assignment
Which variables are included in the utility function?
Different trip purposes?
Different link types and speed-flow functions?
Number of iterations
with feed back?
Linear utility function? Correlation?
Model type? (Logit, Probit, mixed Logit, nested Logit)
Deterministic or stochastic?
Capacity constraint?
Solution algorithm?
Level of detail?
Which characteristics are associated with the links?
Uncertainty in transport models

21. Case study: Næstved model - Extra
Uncertainty in transport models

22. Case study: Næstved model - Extra
Uncertainty in transport models