Gains in evolutionary dynamics A unifying and intuitive approach to linking static and dynamic stability Dai Zusai Philadelphia, U.S.A.

Lyapunov function The theorem does not tell how to find a Lyapunov function  Consider dynamic 𝑑𝒙/𝑑𝑡=𝑉(𝒙) on a multidimensional space. Suppose that we’ve found function 𝐿 that maps (multi-dim) “position” 𝒙 to a scalar (one-dim) value 𝐿(𝒙), s.t. i) 𝐿 attains the minimum value only at an equilibrium; ii) 𝐿 never increases; and iii) 𝐿 must decrease until L reaches its minimum value. Then, we can declare that the equilibrium is stable. Zusai, Introduction to Formal Mathematics for Economic Modeling, Under a contract with Temple University Press for publication as an open access textbook (expected around the end of 2020). History Modified in the paper

Any general principles? What’s our Lyapunov!? In the literature on evolutionary dynamics in population games, dynamic stability of equilibrium is proven for each of major dynamics: Smith ‘84, Cressman ‘97, Hofbauer ’95, ‘01, Hofbauer & Sandholm ‘09, Sandholm ‘10, Metrikopolis & Sandholm ’18 HS ‘09: “[U]nlike potential games, stable games do not come equipped with an all-purpose Lyapunov function. To prove convergence results, we must construct a suitable Lyapunov function for each dynamic we wish to consider.” Any general principles? Mixing heterogeneous dynamics? Robust to misspecification?

What’s our Lyapunov!? Economic intuition? FS ;13: δ-passivity. HS ‘09: integrability MS ‘18: Riemannian geometry (only for local stability of an ESS, requiring negative definiteness–-not only semidefiniteness) Economic intuition? Testable from experiments/data? Applying to other situations?

What’s our gravity? Greed works. Greed clarifies, cuts through, and captures the essence of the evolutionary spirit. Gordon Gekko (Wall Street, 1987)

What’s Greed? Does it really work? Greed: Exploit opportunities for greater profits/payoffs (Gross) gain Γ = Payoff improvements from switches. cf. At eqm, no room for payoff improvement  Γ=0 >0⇏ Switch Decision rules in Evolutionary dynamics Possibly, not exactly optimizing. Possibly, reluctant to switch. 𝑭: Rock-Paper-Scissors with payoff =1 for a win =0 for a draw =−0.9 for a lose ⇒ Strict stable game 𝑽: pairwise payoff comparison dynamic (Smith dynamic s.t. 𝑄 𝑞 = 𝑞 + ) ⇒ While 𝒙 converges to the equilibrium, the aggregate gross gain does not monotonically decrease. Give up linking? No, rather reconsider what’s economically reasonable.

What’s economically reasonable? About economic principles, ask a “principle” textbook Wait, evolutionary game theory is thinking about deviation from rationality! Section 1.3. The First Principle of Economics: Optimization Acemoglu, Laibson and List, Microeconomics (’18,Ed. 2, Pearson) Optimization means that you weight the information that you have, not that you perfectly foresee the future. … Rational action does not require a crystal ball [to perfectly find the best outcome], just a logical appraisal of the costs, benefits and risks that are known to the economic agent. Best Response Dynamic Imitative dynamics Excess Payoff Dynamic Pairwise comparison dynamics Find the optimal strategy (simply, greatest payoff) among all the strategies. Switch to it, regardless of the amount of the payoff improvement. Sample another agent randomly and observe the agent’s strategy. If it performs better than my current strategy, switch to it with a probability proportional to the payoff difference from my current payoff. Sample another strategy randomly. If it performs better than my current strategy, switch to it with a probability proportional to the payoff difference from my current payoff. Sample another strategy randomly. Switch to it with a probability proportional to the payoff difference from the average payoff. Agent’s decision making in evolutionary dynamics Possibly, not exactly optimizing. Possibly, reluctant to switch. Constraint on available strategies Hidden stochastic costs to switch Excluded from our scope---Imitation is truly more than economic/incentive-based reasoning. It is indeed known NOT to guarantee dynamic stability of eqm in stable games or of regular ESS.

Evolutionary dynamics: construction Game 𝝅=𝑭(𝒙) Incentives Choices 𝝅 Payoffs of strategies 𝒙 Shares of strategies An agent occasionally reconsider the choice, when it receives a “revision opportunity,” which arrives randomly. (To make the dynamic differentiable with respect to infinitesimal change in time.) At that opportunity, the agent finds the candidate of a new strategy, and decides whether or not to switch to it. Individual agent’s decision of switching the choice 𝑟 𝑖→𝑗 = 𝑅 𝑖𝑗 𝝅 Evolutionary dynamics 𝒅𝒙 𝒅𝒕 =𝑽(𝝅) At each moment of time, for each strategy, we count the agents who switches to it and those who switches from it. Aggregation of individual agents’ switches 𝑥 𝑖 = 𝑗 𝑥 𝑗 𝑟 𝑗→𝑖 − 𝑥 𝑗 𝑗 𝑟 𝑖→𝑗 𝑟 𝑖→𝑗 Individual agent’s switching rate

Economically reasonable dynamics Say, an agent has been taking action 𝑎 so far and the current payoff vector is 𝝅. 0. Receive a revision opportunity from a Poisson process. 1. Draw a set of available new actions 𝒜 ′ from prob dist ℙ 𝐴𝑎 over a power set of 𝒜∖{𝑎}, and a switching cost 𝑞 from prob dist ℙ 𝑄 with cumulative dist function Q over 0,+∞ . 2. Find the best available action, say 𝑏, among actions in 𝒜 ′ and calculate payoff improvement 𝜋 𝑏 − 𝜋 𝑎 . 3. Switch to action 𝑏 if 𝜋 𝑏 − 𝜋 𝑎 >𝑞; Keep the current action (status quo) 𝑎 if 𝜋 𝑏 − 𝜋 𝑎 <𝑞. ⇒ Given the best available action b, switch occurs with prob 𝑄(𝜋 𝑏 − 𝜋 𝑎 ). Apdx: assumpt’ns Best Response Dynamic ℙ 𝐴𝑎 : any action is always available, i.e., ℙ 𝐴𝑎 (𝒜∖{𝑎})=1. ℙ 𝑄 : switching cost is always 0, i.e., ℙ 𝑄 (0)=1. Q1 𝑄 𝑞 >0 for any 𝑞>0. A0 ℙ 𝐴𝑎 does not depend on 𝒙. A1-i) Any action is available with some positive probability. A1-ii) Availability of an action does not vary with the current action, unless the action has been currently taken (then, it must be certainly available as a status quo). Pairwise comparison dynamics ℙ 𝐴𝑎 : only one action is available, i.e., ℙ 𝐴𝑎 ( {𝑎′})= 1 𝐴−1 for each 𝑎’∈𝒜∖{𝑎} ℙ 𝑄 : any (to have switching rate Q increasing with the payoff improvement) Modified framework: an agent can take a mixed strategy over available actions. Imagine a birth-death process, where a new agent born with default mixed strategy 𝒙 (the population’s current action distribution) replaces an old agent at a “revision” opportunity. Excess Payoff Dynamic ℙ 𝐴𝑎 : any action is always available, i.e., ℙ 𝐴𝑎 (𝒜∖{𝑎})=1 ℙ 𝑄 : any (to have switching rate Q increasing with the payoff improvement) Smooth Best Response Dynamic Further, we introduce a control cost that prevents an agent from taking a pure strategy. (Covered in another in-progress paper.)

Net gain as a general Lyapunov function Our economic reasonable dynamic allows us to define Net gain of switch from a to b := [Payoff improvem’t 𝜋 𝑏 − 𝜋 𝑎 ] – [Switching cost 𝑞] Further, by taking expectation over ℙ 𝑄 and ℙ 𝐴𝑎 , we define Ex-ante net gain for action-𝑎 player 𝑔 𝑎∗ 𝝅 ≔𝔼 Net gain ≥0 Ex-ante second-order gain ℎ 𝑎∗ 𝝅 ≔𝔼 𝑔 New action∗ 𝝅 − 𝑔 𝑎∗ 𝝅 ≤0 Aggregate: 𝐺 𝒙 𝝅 ≔ 𝑎∈𝒜 𝑥 𝑎 𝑔 𝑎∗ 𝝅 , 𝐻 𝒙 𝝅 ≔ 𝑎∈𝒜 𝑥 𝑎 ℎ 𝑎∗ 𝝅 . Embedded to game F: 𝐺 𝑭 𝒙 ≔𝐺 𝒙 𝑭 𝒙 , 𝐻 𝑭 𝒙 ≔𝐻 𝒙 𝑭 𝒙 . Therefore, 𝐺 𝑭 works as a Lyapunov function to derive dynamic stability of equilibrium under economically reasonable dynamics from static stability. >0⇒ Switch <0⇒ Status-quo maximized among all available actions in 𝒜′. 𝐺 𝑭 𝒙 = 𝒙 ⋅𝐷𝑭 𝒙 𝒙 + 𝐻 𝑭 𝒙 ≤0 𝒙 ⋅𝐷𝑭 𝒙 𝒙 ≤0 Static stability 𝐻 𝑭 𝒙 ≤0 Economically reasonable dyn (esp., Assumption A1-ii)

Main theorem Extended to boundary equilibria (regular ESS), a society of (finitely many) heterogeneous populations who follow different payoff functions and revision protocols. Behind those theorems, I also proved several mathematical theorems (a modified version of Lyapunov stability theorem for a set-valued differential equation, etc.) The paper is posted on Arxiv, linked from my web page (easily found from Google).

Wrap up The approach proposed here: Benefits Historical note Construct an economically reasonable dynamic from optimization, possibly with additional costs and constraints to explain distortions from exact best responses. Calculate the net gain as the maximal payoff improvement minus switching cost. Static stability should imply monotone decrease in the aggregate net gain over time, and thus dynamic stability of equilibrium Benefits Aggregate gain is just a sum of individual gains: easily extended to heterogeneous setting. Relying on qualitative characterizations: robust to misspecifications Approximation of finite-agent dynamics. (Ellison, Fudenberg & Imhof ’16 JET on Lyapunov) Intuitive: applicable to complicated settings (e.g. multitasking: Sawa & Z, accepted to JEBO) Historical note Economic theory: Stability of tâtonnement process in general eqm model Strategy adjustment process of an agent Price adjustment process in a market. Market: no single universal axiomatization/formulation Economic agent: agreed to formalize from optimization (even in freshman textbooks).

Lyapunov thm

Modified Lyapunov stability theorem Lyapunov thm Modified Lyapunov stability theorem

Assumptions on ℙ 𝐴𝑎 and ℙ 𝑄 Appendix Zusai, Gains Assumptions on ℙ 𝐴𝑎 and ℙ 𝑄 Q1 𝑄 𝑞 >0 for any 𝑞>0: As long as there is a positive payoff improvement, switch occurs with some positive prob. A0 ℙ 𝐴𝑎 does not depend on 𝒙. A1-i) Any action is available as a candidate with some positive probability. A1-ii) Availability of any candidate actions does not vary with the current action, unless any of those candidate actions is not currently taken (then, it must be certainly available as a status quo). Q1 & A1-i) are for stationarity of a Nash equilibrium A0 excludes imitative dynamics. A1-ii) is to make it economically natural that an agent chooses the next action simply by maximizing the payoff improvement If A1-ii) does not hold, then the choice of a new action affects the possible payoff improvement at the next revision opportunity. So, an economic agent should base the decision not only on the payoff improvement at the present revision opportunity but also the mobility to a further better action in the next revision opportunity. Return: definition