1 A Generic Mean Field Convergence Result for Systems of Interacting Objects From Micro to Macro Jean-Yves Le Boudec, EPFL Joint work with David McDonald, U. of Ottawa and Jochen Mundinger, EPFL

2  The full text of my talk is available in the proceedings of QEST 2007  The paper and this slide show are also available from my web page

3 Contents E.L. 1.Motivation 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Outlook

4 Motivation  Find re-usable approximations of large scale systems Examples from my field Performance of UWB impulse radio : many sensors, each has a MAC layer state Ad-Hoc networking Reputation Systems  From microscopic description to macroscopic equations  Understand fluid approximation and mean field approximation

5 Example 1 : TCP/ECN  TCP connection n transmits at a rate 2 {s 0, …, s i, …, s I }  Queue length at router is R(t)  With probability q(R(t)) connection i receives an Explicit Congestion Notification (ECN) in next time slot  When connection n does not receive an ECN, it increases its rate: If rate == s i, new rate := s i+1 (i<I)  Else it decreases its rate: If rate == s i, new rate := s d(i) ECN router queue length R(t) ECN Feedback q(R(t)) N connections 1 n N The question is the behaviour when N is large

6 Microscopic Description  Time is discrete  Connection n runs one Markov chain X N n (t);  The transition probabilities of the Markov chain X N n (t) depend on global state R(t) (queue size)  Global state R(t) depends on states of all connections let M N i (t) = nb of connections in state i at time t, C = service rate of router ECN received no ECN received

7 Macroscopic Description  The fluid approximation is often given as a simplification of the previous model  Combined with we have a macroscopic description of the system  In [17], Tinna. and Makowski show that it holds as large N asymptotics

8 The Mean Field Approximation  Assume we want to analyze one TCP connection in detail  We can keep the microscopic description for this TCP connection, and use the fluid approximation for the others: We can call it fast simulation.  i.e. pretend X N 1 (t) (one connection) and R(t) (global resource) are independent. This is similar to what is called the mean field approximation in physics

9 Another Example: Robot Swarm  N robots  Robot has S = 2 possible states  Transition for one robot depends on this robot’s state + how many other robots are in search state  [11] uses the fluid approximation :

10 A few other Examples …

11 In these and other examples, some authors assume the validity of the fluid / mean field approximation and use the approximation to do performance evaluation, parameter identification, control… Never again ! … while, in contrast, others spend most of the paper proving the derivation and validity of the approximations in their specific setting papers in this latter class are intimidating cost of proof of one approximation result ¼ 1 PhD and not re-usable Proof of convergence to Mean field for TCP/ECN

12 Can we have answers of general applicability to: When are the fluid approximation and the mean field approximation valid ? Can we write them in a sound ( = mechanical) way ?

13 Contents E.L. 1.Motivation 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Outlook

14 Mean Field Interaction Model  A Generic Model, with generic results  Does not cover all useful cases, but is a useful first step  Time is discrete  N objects  Every object has a state in.  Informally: object n evolves depending only on Its own state A global resource whose evolution depends only on how many other objects are in each state

15 X N n (t) : state of object n at time t M N i (t) = proportion of objects that are in state i M N is the “occupancy measure” ¼ the “mean field” R N (t) = global resource =“history” of occupancy measure Conditional to history up to time t, objects draws next state independent of each other according to Model Assumptions

16 Two Mild Assumptions 1.Continuity of the integration function g() 2.For large N, the transition matrix K becomes independent of N and is continuous

17 TCP/ECN Example fits in this Framework  Intuitively satisfies the conditions State of one connection depends only on buffer content Buffer contents depends only on how many connections are in each state  Formally: One object = one TCP connection State of one object = index i of sending rate R N (t) = total buffer occupancy / N  Function g() : thus g() is continuous Assumption 1 is satisfied ECN router queue length R(t) ECN Feedback q(R(t)) N connections 1 n N

18 TCP/ECN Example fits in this Framework  Transition matrix K Let q(r) = proba of negative feedback when R==r K is independent of N thus Assumption 2 is is satisfied if q() is continuous ECN router queue length R(t) ECN Feedback q(R(t)) N connections 1 n N ECN received no ECN received

19 A Multiclass Variant  Take same as previous TCP/ECN model but introduce multiclass  Aggressive connections, normal connection  State of an object = (c, i) c : class i : sending rate  Objects may change class or not  Also fits in our framework  Mean Field does not mean all objects are exchangeable !

20 Contents E.L. 1.Motivation 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Outlook

21

22 Practical Application : Derivation of the Fluid Approximation  The theorem replaces the stochastic system by a deterministic, dynamical system  This gives a method to write and justify the fluid approximation in the large N regime Equation for the limiting occupancy measure  can be rewritten as where N i (t) = N M N i (t) = number of objects in state i at time t  This recovers for example the result in [17]

23 Proof of Theorem  Based on The next theorem (fast simulation) A coupling argument An ad-hoc version of the strong law of large numbers The Glivenko Cantelli lemma

24 Contents E.L. 1.Motivation 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Outlook

25 Fast Simulation / Analysis of One Object  Assume we are interested in one object in particular E.g. distribution of time until a TCP connection reaches maximum rate  For large N, since mean field convergence holds, one may do the mean field approximation and replace the set of other objects by the deterministic dynamical system  The next theorem says that, essentially, this is valid

26 Fast Simulation Algorithm Returns next state for one object When transition matrix is K State of one specific object This is the mean field independence approximation Replace true value by deterministic limit

27 Fast Simulation Result

28 Practical Application  This justifies the mean field approximation for the stochastic evolution of one object in the large N regime  Gives a method for fast simulation or analysis The state space for Y 1 has S states, instead of S N

29 Contents E.L. 1.Motivation 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Outlook

30 A Reputation System  My original motivation for this work  Illustrates the complete set of steps, including a few modelling tricks  System N objects = N peers Peers observe one subject and rate it Rating is a number in (0,1) Direct observations and spreading of reputation Confirmation bias + forgetting

31 Operation of Reputation System: Forgetting  Z n (t) = reputation rating held by peer n  During a direct observation, subject is perceived as positive (with proba  ) or negative (with proba 1-  )  In case of direct positive observation  In case of direct negative observation  w is the forgetting factor, close to 1 (0.9 in next slides)

32 Confirmation Bias  Peer also read other peer ratings  If overheard rating is z:   is the threshold of the confirmation bias

33 Liars and Honest Peers  Honest peer does as just explained  Liar tries to bring the reputation down Uses different strategies, see later

34 Initially: peers have Z=0, 0.5 or 1  = 0.9 Every time step: direct obs p=0.01, meet liar proba 0.30, meet honest proba 0.69 Example of exact simulation: N=100 peers with maximal liars (always say Z=0) rating proportion of peers

35 3 particular peers, one of each type  = 0.9 time rating

36 Can we study the system with 10 6 users instead of 100 ?

37 The problem fits in our framework…  Assume discrete time  At every time step a peer Makes a direct observation Or overhears a liar Or overhears some honest peer Or does nothing  Object = honest peer  Assume first that liars use strategy 1: maximal lying (always say Z=0)  Transition of one honest peer depends on Own state Distribution of states of all other peers => Fits in our framework with memory R = occupancy measure M

38 Different Liar Strategies  Strategy 1 (maximal lying): liars always say Z= 0  Strategy 2 (infer): liar guesses your rating based on past experience Transition of one honest peer depends on Own state Distribution of states of all other peers What liars remember seeing in the past => Fits in our framework with memory R = occupancy measure of ratings at steps t and t-1  Strategy 3 (side information): liars know your rating and is as negative as you accept not realistic but serves as benchmark (worst case) Similar to strategy 1, memory = occupancy measure M

39 We would like to apply the mean field convergence result to analyze very large N  But model has continuous state space  Discretize reputation ratings ! Quantize Z n on ca. L bits; replace Z n by X n = 2 L Z N with  Issue: small increments due to “forgetting” coefficient w (e.g. w = 0.9) are set to 0  Solution: use random rounding; replace previous equation by where RANDROUND(2.7) = 2 with proba 0.3 and 3 with proba 0.7 E(RANDROUND(x)) = x

40 Transition Matrix K  The transition matrix K N is straightforward but tedious to describe.  Unlike in the TCP/ECN example, it does depend on N  It contains terms such as : the proba that an indirect observation with a honest peer is with someone who has rating equal to k. This proba is equal to It depends on N, but for large N it converges uniformly to M N k (t), with no term in N  The limiting matrix K is polynomial in M N k (t), thus continuous, thus assumption 2 is satisfied  Assumption 1 is trivially satisfied, by inspection

41 Therefore we can apply the theorem and derive the fluid approximation and the mean field approximation Both are true in the limit N = 1

42 Limiting reputation ratings: 0.9 and 0.1 Discrete event simulation, N = 100Fluid Approximation Fast Simulation based on Mean Field Approximation

43  Fluid approximation Can be written using Theorem 4.1 Is a deterministic recurrence with state vector the memory number of dimensions is 2 L+1, where L = number of quantization bits for reputation values (e.g. L=8)  Mean Field Approximation = Fast Simulation Simulation of one Markov chain on state space with 2 L states, with time varying transition probability

44 Different Parameters (few liars) Few liars Final ratings converge to true value Phase transition

45 Different Initial Conditions

46 Liar Strategy 2 (infer) Liar Strategy 3 (side information) Peers starting after 512 time units

47 Modelling Locality with Multiclass Model  We can model spatial aspects Object = honest peer ; state = (c, x) with C = location (in a discrete set of locations) X = rating (same as before) This allows to account for locality of interaction

48 Contents E.L. 1.Motivation 2.A Generic Model for a System of Interacting Objects 3.Convergence to the Mean Field 4.Fast Simulation 5.Full Scale Example: A Reputation System 6.Outlook

49  I have shown how a mean field convergence result can be used to write and validate the fluid approximation = macroscopic description the mean field approximation = fast simulation (or analysis)  Applies to cases where objects interact such that Transition depends on state of this object + current and past distribution of states of all other objects Number of objects is large compared to number of states of one object  Extensions birth and death of objects transitions that affect several objects simultaneously gaussian approximations (central limit theorems) Outlook

50 … thank you for your attention E. L.