Delays and Learning: An Overview INORMS 2009 Meeting, San Diego Hazhir Rahmandad Virginia Tech Industrial and Systems Engineering

Learning Failures Are Common Wide range of persistent superstitious beliefs in magical & supernatural processes Managers attribute business cycles to exogenous events when they are endogenously caused by their own actions Many organizations persistently under-invest in process improvement and maintenance, despite higher payoff. Managers often come to believe that their workers are shirking even when they are not A wide variety of learning biases documented in the psychology literature

Motivation: Different Learning Trajectories Why do so many organizations fail to learn or improve? –Feedback noise –Complexity of payoff landscape –Defensive routines and individual level barriers –Delays between actions and results How important are delays? What are the mechanisms through which they affect learning?

Delays are Common Individuals go to school for several years before seeing financial reward for the schooling It takes over 10 years of investment in new drugs before a pharmaceutical firm can see any payoff Many months of investment in process improvement are often required before results are observable Preventive maintenance costs more for some time, before the overall savings are realized Lag of decades between release of CFCs and ozone destruction Lags of decades to millennia between release of greenhouse gases and climate change impacts

Delays not captured in current models Delays between undertaking activities (a i ) and realizing payoffs (r j ) are common and well documented, however, they are not captured in most existing formal models of organizational learning Organizational Learning Models A More Realistic View Includes Delays a1a1 a2a2 a3a3 r1r1 r2r2 r3r3 a1a1 a2a2 aKaK … rKrK riri … … Time

How Delays Influence Learning Complexity of learning space Exploration bias Dated information about payoff landscape Delays and return on exploration Allocation and short vs. long term Worse before better, incentives, and attribution error We have identified different mechanisms through which delays complicate learning:

Complexity of Learning Space No Delay: Need 2 experiments to map action-payoff relationships Delay=1 period: Need 4 experiments for 4 combinations of 2 periods of binary action Delay=2 periods: Need 8 experiments for the combinations of actions in 3 periods … Delay=N periods: Need 2 N experiments to map out the action-payoff relationship Complexity of learning space grows exponentially with size of delays, requiring much more experience (Rahmandad 08 Management Science) Setting: Every period choose between two actions and observe a (deterministic) payoff

Exploration Bias The value of stepping stone states (e.g. S2) lies in their ability to lead to fruitful states that create payoff (e.g. S3) This value is not recognized until realization of the reward state Selection of pathways to explore depends on their recognized value, therefore potentially fruitful pathways are shut down prematurely when you step into them, but do not make the correct turn all the way (e.g. S1->S2->S’3 shuts down S1->S2) This mechanism works against actions that pay off with longer delays (more intermediate states), creating an exploration bias As a result exploration becomes path-dependant and learning performance heterogeneous (Rahmandad 08) S1 S’3 S2 S3 S’2 a, - a, +++ a’, - a’, + S1 S’3 S2 S3 S’2 a, - a, +++ a’, - a’, + S1 S’3 S2 S3 S’2 a, - a, +++ a’, - a’, + a bc

Dated Information About Landscape Payoff feedback helps assess the best direction to move (the gradient of payoff landscape) Delayed information on gradient can mislead the decision-maker: by the time s/he receives this feedback, s/he might be in a different part of payoff landscape (Rahmandad, Repenning, Sterman 09 System Dynamics Review)

Delays and Return on Exploration As delays increase the complexity of learning, the return on exploration declines Thus the decision maker feels less inclined to try new possibilities This locks the system into current routines, when extra exploration is what is needed to learn when delays are present Therefore we see frequent convergence to suboptimal regions of strategy space (learning erroneous lessons from experience) (Rahmandad, Repenning, Sterman 09)

Allocation and Short vs. Long term Many managerial problems require the allocation of limited resources among different options (e.g. production vs. process improvement) If one option pays off fast and the other with a delay, performance initially declines by re-allocating to the long-term option, thus DMs likely to draw erroneous conclusions from such experimentation Changing allocation towards the long term but more profitable option results in worse before better dynamics (Simulation of a simple organization model) (Rahmandad, Repenning, Sterman 09)

Worse Before Better, Incentives, and Attribution Error Organizational actors may prefer to go for short term gain, even if they know the long term loss (and vice versa), if they think they are rewarded for short term (e.g. because higher-ups can’t measure the long term yields), or would not stay in the position long enough (e.g. average CMO tenure is less than 2 years) Managers take credit for short term gains but attribute the decline to workers shirking, even when they are not (Repenning & Sterman 02 ASQ)

Theoretical Implications Delays between actions and payoffs can: –Increase complexity of learning –Lead to path dependency in adaptation processes –Lead to learning erroneous lessons from experience This provides the basis for a complementary explanation for heterogeneity of organizational performance (The main question in strategy) and organizational structure. Distinction from combinatorial complexity (Levinthal 97, Denrell et al. 03) is useful: –Allows for higher role of cognition compared to serendipity (because of common temporal archetypes, e.g. worse before better) –Consistent with cultural differences in practices, as reaction to temporal archetypes are different across cultures (e.g. auto industry in Japan vs. U.S. during 80’s)

Thanks John Sterman and Nelson Repenning for their contribution to this research Questions and Feedback: