SUMMER SCHOOL 2016 FINLAND

Path Dependence in Operational Research Tuomas J. Lahtinen Presentation is based on joint research with Raimo P. Hämäläinen tuomas.j.lahtinen@aalto.fi, raimo.hamalainen@aalto.fi Systems Analysis Laboratory, Department of Mathematics and Systems Analysis Summer School on Behavioural Operational Research 20.5.2016 The document can be stored and made available to the public on the open internet pages of Aalto University. All other rights are reserved.

Paths in modelling A modelling process: A set of instructions or an abstraction of what happens in modelling Path is the realization of the process, i.e. what actually happens in the OR problem solving effort Can be different and lead to a different outcome every time the process is carried out. Remember that words and vocabulary matters! These do not capture the entire sequence of steps that have actually been taken in the OR problem solving

Steps in OR paths Initial meeting between problem owners and modelers Forming the problem solving team Problem framing and structuring Choice of model Order in which the parts of the model are specified and solved Data collection, preference elicitation Communication with the model Implementation

Path dependence: Outcome depends on the path followed

Path dependence implicitly recognized already early in the OR literature Examples: Best practices: Morris (1967), Mulvey (1979) Model validity: Landry et al. (1983) Adaptive problem solving: Little (1970)

Mulvey 1979, Strategies in Modeling: A personnel Scheduling Example, Interfaces ’Given a single decision problem, two practitioners who are steeped in diverse techniques such as mathematical programming and simulation will invariably develop models which use their particular expertise — even though the real prolem is identical. Nothing is inherently wrong with this bias’

Phenomena which make path dependence interesting We can choose a path which seems ”valid” and reasonable but leads to a poor outcome Biases and errors can accumulate along a path – the overall effect matters One can become locked-in to an inferior approach

Lock-in The development of strong anchor points from which it is not easy to move forward. E.g. QWERTY layout (David 1985) Can occur also due to behavioural reasons

What are the factors that steer the path and create lock-ins?

Origins and drivers of path dependence in OR Hämäläinen and Lahtinen (2016): System Learning Procedure Behavior Motivation Uncertainty External environment Can interact and occur together Relate to context, actors, praxis and methods

System Formed by the people involved in the problem solving process This is the right model Yes Lock-in to one approach. Groupthink, working with ”our” models Irreversibility. Due to budget, time or resource constraints Also the system under study ’Mathematical’: Increasing returns, bifurcations, feedback loops

Learning Problem owners, stakeholders and modelers learn about the problem: revise assumptions, redirect the process Unlearning preconceived solutions is difficult Lane (1993), Systems Dynamics Review: ’…all (problem) formulation approaches that use the tools of system dynamics at the start of the process … rapidly converge the debate onto issues with a strong time-evolutionary dimension…’

Procedure Different processes, methods, modeling approaches can lead to different outcomes Structures and properties of the methods used interact with the other drivers of path dependence Examples Technical properties, convergence algorithms Order of problem solving steps Problem and model decomposition

Behavior Cognitive biases and behavioral phenomena related to individuals ’Behavioral’ lock-in The sunk cost effect / status quo bias in modelling An organization can keep on using the old model which has grown excessively and become unwieldy and nontransparent

Accumulation of bias A B Ideal process = no bias C

Accumulation of scale compatibility and loss aversion in the Even Swaps process Scale compatibility = more weight to measuring stick Loss aversion = more weight to loss DM chooses A? DM chooses B?

Motivation Relates to exposed and hidden goals People can promote their own interest and behave strategically in the OR process Higher risk in messy and controversial problems

‘stakeholders tended to request the inclusion of objectives that favoured their preferred alternative’ (von Winterfeldt and Fasolo, 2009, Structuring decision problems: A case study and reflections for practitioners) ‘in many real life circumstances, experts perform the service of hired guns for companion directors or for politicians, justifying preconceived decisions “scientifically”’ (Rauschmayer et al. 2009, Why good practice of OR is not enough—Ethical challenges for the OR practitioner)

Uncertainty Can be about structural assumptions and correct parameter values High level of uncertainty → No one ’right’ path → Higher sensitivity to initial modeling choices Examples: Climate modelling: structural assumptions Real options modeling: Parameter values with market data or subjective estimates?

Changes in the context and external environment Problem environments can change The chosen modeling process becomes invalid or it can lead to a different outcome. Example: Timing of an OR intervention Postpone to gather more accurate information? If environment changes, some paths may become unavailable

Is path dependence a risk? Not necessarily if goal is to increase understanding Trying different paths can be beneficial to learning Awareness of path dependence can be enough Path dependence poses a risk When seeking for optimal and efficient solutions In important policy problems, normative decision support

Coping with path dependence

Awareness Challenge the modelling team to reflect on the drivers of their behavior including the implicit assumptions and mental models the choices made along the path Consider the possibility of lock-ins: Early steps and framing are critical Has mental lock-in already occurred? Can it be broken?

Use of multiple models More than one problem solving process with different teams Helps to consider a larger variety of alternative problem formulations and model structures Devil’s advocate team? to find and challenge crucial assumptions to perform worst case analyses Confidence about a solution increases if a similar solution is obtained with another model

Adaptive problem solving Announce checkpoints where process can be revised E.g. when intermediate results are obtained, learning has occured, new data becomes available Ensure availability of resources Helps to cope with changes and uncertainty in the modeling environment

Debiasing Reducing effects of cognitive biases in preference elicitation and in estimation tasks Some ideas: Reframe questions, give better training, calibrate judgments Biases can accumulate but they can also cancel out One can try design the elicitation process so that effects of biases cancel out Example of such procedure for Even Swaps in Lahtinen, Hämäläinen (2016) => Not always necessary to debias single judgments

A B C

Example: Trade-off weighting Assume ”Real weights” are 𝑊 1 =33%, 𝑊 2 =33%, 𝑊 3 =33% Thus 𝑊 1 𝑊 2 should be 1 But… measuring stick bias doubles weight in trade-off assessment: 𝑊 1 𝑊 2 =2 Elicitation 1: 𝑊 1 𝑊 2 =2, 𝑊 2 𝑊 3 =2 => Derive unique weights: 57%, 29%, 14%

Example: Trade-off weighting Assume ”Real weights” are 𝑊 1 =33%, 𝑊 2 =33%, 𝑊 3 =33% But… measuring stick effect doubles weight in trade-off assessment: 𝑊 1 𝑊 2 =2 Elicitation 1: 𝑊 1 𝑊 2 =2, 𝑊 2 𝑊 3 =2 => Derive unique weights: 57%, 29%, 14% Elicitation 2: 𝑊 1 𝑊 2 =2, 𝑊 2 𝑊 3 =2, W 3 W 1 =2 => Take least squares average => weights: 33%, 33%, 33%

Conclusions The term path is new in modelling Path dependence is a real phenomenon in OR Can originate due to behavior, social interaction, technical properties of procedures, changes in external environment… Extra concern in large policy problems and in prescriptive decision support There are ways to cope with path dependence!

Thank you Lahtinen TJ, Hämäläinen RP (2016) Path dependence and biases in the even swaps decision analysis method, European Journal of Operational Research, special issue on Behavioural OR. Hämäläinen RP, Lahtinen TJ (2016) Path Dependence in Operational Research – How the Modeling Process Can Influence the Results, Operations Research Perspectives. Available at http://sal.aalto.fi/publications/ Photo by Dioboss, CC BY-NC-SA 2.0