1. Wind-tunnel Testing for Strategy and Market Design
Edward Tsang
CCFEA
University of Essex
Colchester, UK
Invited Talk, Parallel Problem Solving in Nature
Title: Wind-tunnel Testing for strategy and market design
Designers of racing cars and aeroplanes today would typically put their designs through wind-tunnel testing. Many business strategies determine the success or failure of an enterprise. It is sensible to put these strategies into action before testing them rigorously. On the other hand, new markets are being designed all the time; examples include the electricity market and traffic markets. The robustness and efficiency of these market affect the society. It is inconceivable to put these markets into operation without testing them scientifically (e.g. to ask what-if questions). Agent-based market modelling allows one to “wind-tunnel test” strategies and market designs. No model is perfect, and no amount of testing will guarantee the success of a strategy or market. However, agent-based models enable us to test complex behaviour and their interactions. In this talk, I shall define the research agenda, and explain some of the work that we have done in wind-tunnel testing strategies and market designs. Applications include automated bargaining and BT’s work force scheduling.
22 February 2019
All rights reserved, Edward Tsang

2. Agent-based Artificial Markets
Strategy Design
How to do well in market
Wind Tunnel Market Testing
Designing market mechanism
Application
Agent 1
Agent 2
Artificial Market
endogenous
exogenous
Agent n
Fundamental
What happens when agents evolve?
Nash equilibrium
Better understand the market
What makes a market efficient?
What is the underlying mechanism?
22 February 2019
All rights reserved, Edward Tsang

3. Summary: Agent-based Artificial Markets
Agents
Markets
Strategies evolved
Payments market
Bargaining
Wind-tunnel Testing
British Telecom
Staff empowerment
Applications
Nash Equilibrium
Red Queens effect studied
Reproduced stylized facts in CHASM
Rationality challenged
Fundamental
Conclusions:
Challenging the fundamentals in economics
Rich research area – gathering momentum, plenty of challenges and applications ahead
Evolutionary computation plays an important part
Rich and challenging research, EC plays vital part
22 February 2019
All rights reserved, Edward Tsang

4. A cost-effective way to evaluate policies before implementing them
Wind Tunnel Testing
A cost-effective way to evaluate policies before implementing them
22 February 2019
All rights reserved, Edward Tsang

5. Wind-tunnel tests for new markets
New markets are being invented
e-Bay, electricity, roads, pollution
Why wind-tunnel testing?
Answer what-if questions
Approximate equilibriums
22 February 2019
All rights reserved, Edward Tsang

6. Modelling is commonly used

7. Modelling is never perfect
“All models are wrong, but some are useful”
(Box and Draper, 1987)
“More calculation is better than less, Some calculation is better than none”
(Sun Zi, 6BC)
多算胜，少算不胜，而何况于无算乎？ 【孙子】
“All models are wrong, but some are useful”
Box, George E. P.; Norman R. Draper (1987). Empirical Model-Building and Response Surfaces. Wiley. pp. p ISBN
多算胜，少算不胜，而何况于无算乎？
【孙子】

8. Market Testing Examples
Tesfatsion & Koesrindartoto:
design of California’s electricity market
Markose, Allen & Blyth:
Market-based congestion charges
Presented to UK Foresight, Prime Minister’s Office and Treasury
Tsang et al:
BT’s scheduling problem
22 February 2019
All rights reserved, Edward Tsang

9. All rights reserved, Edward Tsang
Market for Road Usage
Foresight, UK:
Car # to increase by 50% in 15 years
What to do?
22 February 2019
All rights reserved, Edward Tsang

10. Why Wind-tunnel Testing
Modelling is a cost-effective way of understanding a market
It helps answering what-if questions
It helps policy makers to check if intended goals are achieved; e.g.
London congestion charges
Google’s IPO
If not, how to achieve them
It can help commercial sector to design strategies
22 February 2019
All rights reserved, Edward Tsang

11. London Congestion Charges
17 Feb 2003: Drew a circle, £5 per visit
4 Jul 2005: raised to £8 (plan: £10)
What are the Government’s objectives?
To reduce traffic by what %?
Testing in real market leads to oppositions
Why not test in a model first?
No model is perfect
But modelling enables scientific studies
22 February 2019
All rights reserved, Edward Tsang

12. Google’s Initial Public Offer (IPO)
IPOs typically under-priced
Google auctioned its IPOs in 2004
Stock price up by 17% on first trading day
Good for investors
Not what Google wanted
Could market testing help?
Depends on model
22 February 2019
All rights reserved, Edward Tsang

13. All rights reserved, Edward Tsang
Strategy Design
Bank of England & CCFEA
Learning “how much liquidity is required”
Alexandrova, Tsang and Krause
Modelling credit card market (Mexican Central Bank)
Tuning bank strategies
Gosling & Tsang
Simple Supply Chain Management (BT project)
Learning middlemen strategies
Jin & Tsang
Finding reasonable bids in bargaining (game theory)
Pierre Xun Wang & Tsang
Strategies for IPO auctions
22 February 2019
All rights reserved, Edward Tsang

14. Understanding Market Behaviour
Agent 1
Agent 2
Artificial Market
endogenous
exogenous
Essex Artificial Market (CHAMS):
Endogenous returns
Use EDDIE for re-training
Agent n
How intelligent?
Understanding the underlying mechanism
Many interesting results reported
22 February 2019
All rights reserved, Edward Tsang

15. Agent-based Equilibrium
Bees maintain constant hive temperatures
How?
By varying their wing-frequencies?
No, much simpler
By switching on/off at different thresholds
22 February 2019
All rights reserved, Edward Tsang

16. Building Artificial Markets
Goal: to understand the underlying mechanism of real markets
Means: to build artificial markets and observe their behaviour
Rationale:
If the artificial market exhibits stylized facts
Then the built in mechanism might reflect the underlying mechanism
22 February 2019
All rights reserved, Edward Tsang

17. All rights reserved, Edward Tsang
Red Queen
… Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that! …
Red Queen in Economics
22 February 2019
All rights reserved, Edward Tsang

18. All rights reserved, Edward Tsang
Red Queen principle
The Red Queen principle was proposed by the evolutionary biologist Leigh Van Valen (1973).
For an evolutionary system, continuing development is needed just in order to maintain its fitness relative to the systems it is coevolving with.
22 February 2019
All rights reserved, Edward Tsang

19. Agent-based Artificial Market
Built to understand market behaviour better
Santa Fe Institute:
Exogenous returns (set by experimenter)
(Evolutionary) Classifier Systems
LeBaron:
Endogenous returns
Does market exhibit empirical features (“Stylized facts”)?
Effects of the traders’ memory
Agent 1
Artificial Stock Market
Agent 2
Farmer:
Trend following agents +
(Fundamental) value investors
Agent n
Herding behaviour?
22 February 2019
All rights reserved, Edward Tsang

20. All rights reserved, Edward Tsang
Daniel Kahneman
Nobel Prize in Economics, 2002
Psychology in Economics, uncertainty
“for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty”
Princeton, USA
October 2002
The Royal Swedish Academy of Sciences has decided that the Bank of Sweden Prize in Economic Sciences in Memory of Alfred Nobel, 2002, will be shared between Daniel Kahneman Princeton University, USA “for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty” and Vernon L. Smith George Mason University, USA “for having established laboratory experiments as a tool in empirical economic analysis, especially in the study of alternative market mechanisms”.
Psychological and experimental economics
Traditionally, much of economic research has relied on the assumption of a “homo œconomicus” motivated by self-interest and capable of rational decision-making. Economics has also been widely considered a non-experimental science, relying on observation of real-world economies rather than controlled laboratory experiments. Nowadays, however, a growing body of research is devoted to modifying and testing basic economic assumptions; moreover, economic research relies increasingly on data collected in the lab rather than in the field. This research has its roots in two distinct, but currently converging, areas: the analysis of human judgment and decision-making by cognitive psychologists, and the empirical testing of predictions from economic theory by experimental economists. This year’s laureates are the pioneers in these two research areas. Daniel Kahneman has integrated insights from psychology into economics, thereby laying the foundation for a new field of research. Kahneman’s main findings concern decision-making under uncertainty, where he has demonstrated how human decisions may systematically depart from those predicted by standard economic theory. Together with Amos Tversky (deceased in 1996), he has formulated prospect theory as an alternative, that better accounts for observed behavior. Kahneman has also discovered how human judgment may take heuristic shortcuts that systematically depart from basic principles of probability. His work has inspired a new generation of researchers in economics and finance to enrich economic theory using insights from cognitive psychology into intrinsic human motivation. 
Daniel Kahneman, born 1934 (68 years) in Tel Aviv, Israel (US and Israeli citizen). PhD from University of California at Berkeley in Since 1993, Eugene Higgins Professor of Psychology and Professor of Public Affairs at Princeton University, NJ, USA.
22 February 2019
All rights reserved, Edward Tsang

21. All rights reserved, Edward Tsang
Vernon Smith
Nobel Prize in Economics 2002
Experimental Economics
“for having established laboratory experiments as a tool in empirical economic analysis, especially in the study of alternative market mechanisms”
George Mason University, USA
Vernon L. Smith George Mason University, USA “for having established laboratory experiments as a tool in empirical economic analysis, especially in the study of alternative market mechanisms”.
Psychological and experimental economics
Traditionally, much of economic research has relied on the assumption of a “homo œconomicus” motivated by self-interest and capable of rational decision-making. Economics has also been widely considered a non-experimental science, relying on observation of real-world economies rather than controlled laboratory experiments. Nowadays, however, a growing body of research is devoted to modifying and testing basic economic assumptions; moreover, economic research relies increasingly on data collected in the lab rather than in the field. This research has its roots in two distinct, but currently converging, areas: the analysis of human judgment and decision-making by cognitive psychologists, and the empirical testing of predictions from economic theory by experimental economists. This year’s laureates are the pioneers in these two research areas. Vernon Smith has laid the foundation for the field of experimental economics. He has developed an array of experimental methods, setting standards for what constitutes a reliable laboratory experiment in economics. In his own experimental work, he has demonstrated the importance of alternative market institutions, e.g., how the revenue expected by a seller depends on the choice of auction method. Smith has also spearheaded “wind-tunnel tests”, where trials of new, alternative market designs – e.g., when deregulating electricity markets – are carried out in the lab before being implemented in practice. His work has been instrumental in establishing experiments as an essential tool in empirical economic analysis.
Vernon L. Smith, born 1927 (75 years) in Wichita, KS, USA (US citizen). PhD from Harvard University in Since 2001, Professor of Economics and Law at George Mason University, VA, USA.
22 February 2019
All rights reserved, Edward Tsang

22. All rights reserved, Edward Tsang
Mechanism Design
Nobel Prize in Economics 2007
For having laid the foundations of mechanism design theory
Leonid Hurwicz
Minnesota
b.1917
Eric S Maskin
Princeton
b.1950
Roger B Myerson
Chicago
b.1951
Vernon L. Smith George Mason University, USA “for having established laboratory experiments as a tool in empirical economic analysis, especially in the study of alternative market mechanisms”.
Psychological and experimental economics
Traditionally, much of economic research has relied on the assumption of a “homo œconomicus” motivated by self-interest and capable of rational decision-making. Economics has also been widely considered a non-experimental science, relying on observation of real-world economies rather than controlled laboratory experiments. Nowadays, however, a growing body of research is devoted to modifying and testing basic economic assumptions; moreover, economic research relies increasingly on data collected in the lab rather than in the field. This research has its roots in two distinct, but currently converging, areas: the analysis of human judgment and decision-making by cognitive psychologists, and the empirical testing of predictions from economic theory by experimental economists. This year’s laureates are the pioneers in these two research areas. Vernon Smith has laid the foundation for the field of experimental economics. He has developed an array of experimental methods, setting standards for what constitutes a reliable laboratory experiment in economics. In his own experimental work, he has demonstrated the importance of alternative market institutions, e.g., how the revenue expected by a seller depends on the choice of auction method. Smith has also spearheaded “wind-tunnel tests”, where trials of new, alternative market designs – e.g., when deregulating electricity markets – are carried out in the lab before being implemented in practice. His work has been instrumental in establishing experiments as an essential tool in empirical economic analysis.
Vernon L. Smith, born 1927 (75 years) in Wichita, KS, USA (US citizen). PhD from Harvard University in Since 2001, Professor of Economics and Law at George Mason University, VA, USA.
22 February 2019
All rights reserved, Edward Tsang