1 ENS, June 21, 2007 Jean-Yves Le Boudec, EPFL joint work with David McDonald, U. of Ottawa and Jochen Mundinger, EPFL …or an Art ? Is Mean Field a Technology…

2  Most of the material in this presentation is based on J.-Y. Le BoudecJ.-Y. Le Boudec, D. McDonald and J. Mundinger A Generic Mean Field Convergence Result for Systems of Interacting Objects 4th International Conference on the Quantitative Evaluation of SysTems (QEST) 2007D. McDonaldJ. Mundinger also available at  this slide show

3 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

4 Mean Field: What is it ?  Introduced in Physics  Markov process, continuous time, represents states of particles  Mean field is a model where interaction between particles is via some quantity derived from distribution of states of all particle  Mean field is also an approximation for the state of one particle where we assume independence in the master equation ( equivalent to the forward equation, called Chapman-Kolmogorov in queuing theory textbooks) state of particle n rate of transition r effect of transition r on particle n distribution of states independence assumption

5 Mean Field: What is it ?  Mean field is also an approximation for the distribution of states where we assume determinism in the master equation  This is often called the “mean field” equation

6 Convergence to Mean Field  Proving these approximation is usually done in the sense of convergence when some parameter gets large (e.g. number of particles)  The techniques for that are: convergence of one Markov process towards another.  Sources of techniques are

7 A Few Examples Where Applied Never again ! E.L.

8 Generic Approach to show convergence to mean field  [Ethier and Kurtz] 1.Find a scaling variable N and a family of Markov processes M N indexed by N 2.Show that the family M N is relatively compact in the sense of topology of weak convergence 3.Find the limit of the martingale problem for M N and show that it has a unique solution

9 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

10 Mean Field Interaction Model  A Generic Model, with generic results  Our first cut at making mean field a technology  Time is discrete  N objects  Every object has a state in.  Mean field approx reduces size of model from N £ S to S  Informally: object n evolves depending only on Its own state How many other objects are in each state

11  Model assumptions: X N n (t) : state of object n at time t M N i (t) = proportion of objects that are in state i M N is the “occupancy measure” R N (t) = “history” of occupancy measure Objects draws next state according to

12 Two Mild Hypotheses g() is continuous

13 Example 1: Robot Swarm  No memory: R N = M N  N objects = N robots  Robot has S = 2 possible states  Fits in framework: transitions for one robot depends on own state + how many are in search state  K N independent of N after proper re-scaling  Equation for mean field is  Mild hypotheses are true because K is continuous in M (in fact: linear)

14 Example 2: ECN/TCP Gateways  N objects = TCP connections  State = sending rate (discrete)  Memory R = buffer at ECN gateway  Mean field equation  q(r) = proba of negative feedback when R=r  Mild assumptions are true if q() is continuous ECN received no ECN received

15 Example 3: Heterogeneous (Multiclass Example)  Same as previous but introduce multiclass model  Aggressive connections, normal connection  State of an object = (c, i) c : class i : sending rate  Objects may change class or not  Also fits in our framework  Mean Field does not mean all objects are exchangeable !

16 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

17  A slightly weaker form was proven in many of the references mentioned earlier, in particular  A close, continuous time cousin is in

18 Practical Application  This replaces the stochastic system by a deterministic, dynamical system  This justifies the mean field equation in the large N regime  This can be used as a “fluid approximation” N i (t) = number of objects in state i

19 Proof of Theorem  Based on The next theorem (fast simulation) A coupling argument An ad-hoc version of the strong law of large numbers The Glivenko Cantelli lemma

20 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

21 Fast Simulation / Analysis of One Object  Assume we are interested in one object in particular E.g. distribution of time until a TCP connection reaches maximum rate  For large N, since mean field convergence holds, one may forget the details of the states of all other objects and replace them by the deterministic dynamical system  The next theorem says that, essentially, this is valid

22 Fast Simulation Algorithm Returns next state for one object When transition matrix is K State of one specific object This is the mean field independence approximation Replace true value by deterministic limit

23 Fast Simulation Result

24 Practical Application  This justifies the mean field approximation (based on the independence assumption) for the stochastic state of one object as a large N asymptotic  Gives a method for fast simulation or analysis The state space for Y 1 has S states, instead of N £ S

25 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

26 A Reputation System  My original motivation for this work  Illustrates the complete set of steps  System N objects = N peers Peers observe one subject and rate it Rating is a number in (0,1) Direct observations and spreading of reputation Confirmation bias + forgetting

27 Operation of Reputation System: Forgetting  Z n (t) = reputation rating held by peer n  Subject is perceived as positive (with proba  ) or negative (with proba 1-  )  In case of direct positive observation  In case of direct negative observation  w is the forgetting factor, close to 1 (0.9 in next slides)

28 Confirmation Bias  Peer also read other peer ratings  If overheard rating is z:   is the threshold of the confirmation bias

29 Initially: peers have Z=0, 0.5 or 1  = 0.9 Every time step: direct obs p=0.01, meet liar proba 0.30, meet honest proba 0.69 Example of simulation: N=100 peers with maximal liars (always say Z=0)

30 3 particular peers, one of each type  = 0.9

31 The problem fits in our framework…  Assume discrete time  At every time step a peer Makes a direct observation Or overhears a liar Or overhears some honest peer Or does nothing  Object = honest peer  Transition depends on Own state Distribution of states of all other peers => Fits in our framework with memory R = occupancy measure M

32 We would like to apply the mean field convergence result to analyze very large N  But model is continuous state space  Discretize reputation ratings ! Quantize Z n on L bits; replace Z n by X n = 2 L Z N, constrained to be integer Random rounding to avoid side effects; replace by where RANDROUND(2.7) = 2 with proba 0.3 and 3 with proba 0.7 E(RANDROUND(x)) = x

33 Transition Matrix  Is deduced from the rules  The hypotheses apply (almost trivially; K is linear in M) Proba of meeting a honest peer other than self Converges uniformly in M to M N k (t) for large N

34 Limiting reputation ratings: 0.9 and 0.1

35 Different Parameters (few liars) Few liars Final ratings converge to true value Phase transition

36 Different Initial Conditions

37 The Model can serve different Liar Strategies  Strategy 1: liars always say Z= 0  Strategy 2: liar guesses your rating based on past experience Similar to strategy 1, except memory R = memory occupancy at step t-1  Strategy 3: liars know your rating and is as negative as you accept Similar to strategy 1, memory = occupancy measure M

38 Liar Strategy 2 (infer) Liar Strategy 3 (side information) Peers starting at time 512

39 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

40 A Central Limit Theorem Similar to Van Kampen’s Method  Case without memory and assuming K is differentiable:

41 Contents 1.Mean Field in communication and computer modeling 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Central Limit Theorem 7.Outlook

42 For the cases that fit in this framework…  We are closer to …  than to …

43 Extend the modelling framework to: birth and death of objects transitions that affect several objects simultaneously enumerable but infinite set of states Further work E. L.