Automata-based adaptive behavior for economic modeling using game theory Rawan Ghnemat, Khalaf Khatatneh, Saleh Oqeili Al-Balqa’ Applied University, Al-Salt, Jordan Cyrille Bertelle, LIH, University of Le Havre, France Gérard H.E. Duchamp LIPN, University of Paris XIII

17/11/2005EPNADS'052 Outline 1.From an application in game theory to fundamental concepts for adaptive strategies 2.Multi-scale complex system modeling 3.Genetic automata and emergent systems computation 4.Conclusion

17/11/2005EPNADS' From an application in game theory to fundamental concepts for adaptive strategies 1. From an application in game theory to fundamental concepts for adaptive strategies  Example of modelization for game theory using an adaptive strategy  How can one strategy be modelized ?  Multi-strategies in one model  A model for an adaptive strategy

17/11/2005EPNADS'054 Prisoner dilemma  Two players game involving a cooperation/competition model  Basic model for economic purpose  Rules:  2 options for each player at each play: cooperate (C) or betray (D)  Associated payoff for each situation in following table

17/11/2005EPNADS'055 Prisoner dilemma payoff table Player 2 Player 1 C (cooperate) D (betray) C (cooperate) (+3, +3) (0, +5) D (betray) (+5, 0) (+1, +1)

17/11/2005EPNADS'056 Prisoner dilemma application  Great number of applications, especially in military or economic domains  Example - Competition between two companies:  Aggressive competition behavior  Cooperative behavior  … see payoff table in the following slide

17/11/2005EPNADS'057 Prisoner dilemma application (2) Company C1 Company C2 Cooperative politic Aggressive politic Cooperative politic Medium profit for each Poor profit for C2 and huge profit for C1 Aggressive politic Poor profit for C1 and huge profit for C2 Poor profit

17/11/2005EPNADS'058 Iterative version for prisoner dilemma  Successive steps  Each player do not know adversary’s action …  … but he knows the previous action of his adversary  So different strategies can be defined for player behavior (goal: having maximal payoff for himself)

17/11/2005EPNADS'059 Some strategies  Vindictive strategy:  If the adversary cooperates at previous play, I cooperate  If the adversary betrays one time, I will always defect, whatever the adversary does later!  Tit-for-Tat strategy:  I always do what my adversary has done at the previous play

17/11/2005EPNADS'0510 Model for “Tit for tat” strategy  One state to define my behavior  Transition for each play:  Input: what the adversary does at the previous play  Output: what I do in consequence → Automata with outputs

17/11/2005EPNADS'0511 Model adaptation for “vindictive” strategy

17/11/2005EPNADS'0512 Multi-strategies automaton with evolution  Probabilistic automaton for multi- strategies in one model  Evolution on probabilistic values to adapt the strategy (explained in section 3)

17/11/2005EPNADS' Multi-scale complex system modeling  Previously, we presented how cooperative/competitive behaviors can be modelized with automata with outputs  Now, we will explain how such models can be used for complex systems modeling

17/11/2005EPNADS'0514 Multi-scale complex system modeling - a

17/11/2005EPNADS'0515 Multi-scale complex system modeling - b

17/11/2005EPNADS'0516 Multi-scale complex system modeling - c

17/11/2005EPNADS'0517 Multi-scale complex system modeling - d

17/11/2005EPNADS'0518 Multi-scale complex system modeling - e  Problem : How to compute the emergent systems formation? And the retro-action of the system on their contitutive components?

17/11/2005EPNADS'0519 Multi-scale complex system modeling - a

17/11/2005EPNADS' Genetic automata for game theory and emergent systems computation The solution is to use genetic algorithms on automata: For evolutive strategies (Prisoner Dilemma) For emergent systems computation

17/11/2005EPNADS' Genetic automata for game theory and emergent systems computation Return to the probabilistic automaton Linear representation:

17/11/2005EPNADS'0522 Genetic algorithm on automata  Chromosomes: sequence of all the matrices associated to each letter  One allele (primitive data) of chromosomes corresponds to one line of the matrix representation → 3 classical operators: Duplication, crossing- over and mutation → 3 classical operators: Duplication, crossing- over and mutation

17/11/2005EPNADS'0523 General GA process for genetic automata for game theory  A population of automata is initially generated  Each automaton makes a set of plays against a predefined strategy, named SO  At each play, we execute the probabilistic automaton which gives output: C or D  With this output and SO’s output, we compute the payoff of the automaton with the payoff table

17/11/2005EPNADS'0524 General GA process for genetic automata for game theory(2)  At the end of the set of play, the automaton payoff is the sum of all payoff of each play → fitness of the automaton  At the end of the set of play, each automaton has a fitness → selection process selects the best automaton → new automata population  New computation of the 3 genetic operators over this new population.

17/11/2005EPNADS'0525 Genetic automata for emergent self-organizations  Computation of a new fitness allowing emergent systems of similar behaviour  Fitness computation based on  e(x): Evaluation on agent behaviour automaton x as the matrix M of outputs M(i,j) from all possible successive perceptions from an initial state i to a final state j  d(x,y) = || e(x)-e(y) || : semi-distance between two agent behaviors x and y

17/11/2005EPNADS'0526 Genetic automata for emergent self-organizations  Fitness computation of an agent x where Vx corresponds to a neighbourhood of x:

17/11/2005EPNADS'0527 Genetic automata for emergent self-organizations  So selected agents of the population will become more near to each others, in respect of the semi-distance defined  So GA is a way to modelize the feed- back of emergent systems which leads to gather agents of similar behaviour, in a dynamical way.

17/11/2005EPNADS' Conclusion  Genetic automata are appropriate models for adaptive behaviour  Adaptive behavior are the basis for game theory implementation  Adaptive behavior can be the expression of feed-back of emergent systems over their constitutive agents