Game Theory and the Internet Flow and Congestion Control

2 Differential QoS and Pricing in Networks: where flow-control meets game theory. Peter Key and Derek McAuley, 1999

3 Notation r Ruser / network route j Jnetwork resource (eg link) x r resources demanded by user r (eg bandwidth per unit time) A = (A jr )1 if resource j is consumed by user r 0 otherwise (assume fixed routing) Ax is the (offered) load to the resource vector

4 Notation cont. U r (x r )utility function for user r, relating bandwitdth to worth C j (y) cost of loading y on resource j Assume U r (x) is concave: f((x 1 + x 2 )/2) (f(x 1 ) + f(x 2 ))/2 Assume C j (y) is increasing, differentiable.

5 Social Optimum Maximize U(x)= r U r (x r ) - j C j ( r A jr x r ) over x 0.

6 If C j (y) are convex functions, the optimum for nonzero x r satisfies U r (x r )= j j A jr where the j are associated with the shadow prices: j = p j ( j A jr x r ) with p j (y) = dC j (y)/dy (Shadow price: marginal increment in cost at the resource for a marginal increment in load.)

7 User optimum Assume users are charged a r per amount of data sent. Maximize: U r (x r ) – a r x r over x r 0

8 User optimum cont. Solution: x*, where U r (x*) = a r,if U r (0) > a r x r = 0, otherwise The user accepts any amount of bandwidth offered up to x*, if the prices are rational, i.e. U r (0) > a r User optimum coincides with social optimum if the prices are right: a r = j r j

9 TCP congestion control (in brief) Congestion is controlled by changing the rate of sending packets. When a resource in the network becomes congested, some packets will be lost. Loss of a packet is interpreted by the destination as an indication of congestion. The source is notified to slow down transmission. The transmission rate is gradually increased until a new notification of congestion. This allows the utilization of available bandwidth.

10 Framework Users generate load x(t) and receive feedback signals f(t), which are related to shadow prices. For example certain packets receive marks. A packet is marked if and only if it contributed to a loss event, i.e. the removal of the packet arriving at some resource would result in one less packet loss at that resource. So, mark all packets arriving in an overloaded resource.

11 More specific… Consider single unbuffered resource, capacity C. The load Y is a random variable with mean y, following Poisson or Gaussian distribution. The expected loss is E[(Y - C) + ] The shadow price can be shown to be equal to the probability of resource saturation P sat = P{Y C}

12 So marking scheme for unbuffered resource: Mark all the packets that contribute to the load when the resource is overloaded, otherwise mark none. The probability of loss: P loss = E[(Y – C) + ]/E[Y] Typically P loss << P sat, so many more packets are marked than lost.

13 Flow control Optimality depends on the utility functions of the users, but these functions are unknown to the network. So: Let the network determine the prices and let the users place demands on the network, which the network will try to meet. The users can behave as they wish, possibly performing some user optimization.

14 Redefinition of flow control algorithm Implement a means to feed back information to the user: mark a packet Every resource used for the transmission of the packet has the ability to mark it. But the probability of multiple markings is very small, so one bit can be used to approximate a mark. This bit can be incorporated in most existing protocols easily.

15 Algorithms for flow control Willingness to pay scheme: x t+1 = x t + k(w – f t ) w represents willingness to pay, k affects the rate of convergence. TCP-like scheme: x t+1 = x t + k(w – f t x t /2)

16 Experimentation Run a distributed emulation of a network, where users are free to invent their own algorithms or behave as they wish. It is a distributed multi-user game, where the users try to optimize their own objective.

17 Proposed framework Users play against a network representing the resource system: The game consists of a distributed set of users and a Network, emulated by a PC A protocol interfaces between the users and the Network Users can interact (connect) across the Internet Users can interact in real time or through programs Game time can be slowed down or speeded up

18 Users select a destination and generate load (packets), which are sent to the Network. The Network decides which resources to allocate to the user and sends back a set of marks, depending on the state of the resources. The Network also sends to the user signals about the start/end of the game and its relative speed compared to real time.

19 Protocol for a distributed game : = is generated by the user and used to detect loss. is used to identify a route in the Network : = can be void or corrupted. gives the number of marks in the packet : = sends heartbeats back to the user giving time in the game world

20 Typical sequence of events User requests to join the game Receives acknowledgement and relative speed Receives notice of start of game Repeat Generate token Calculate load (packet size) Send = Receive = Until End of game User ends session

21 Typical objectives We consider optimality from the perspective of the user, not the network. Reminder (and some slight changes): User r generates load x t = x(t) at time t. Number of marked packets is f t = f(t). Cost factor a, so cost incurred is: af t.

22 Typical objectives cont. Control cost: Keep the cost constant and equal to w (willingness to pay). L is some loss function minimize L(w – af t ) Throughput: Maximize throughput minus the cost maximize T t=1 (x t -af t )/T Constraint throughput: Same as above given a cost constraint maximize T t=1 (x t -af t )/T, such that T t=1 af t

23 Typical objectives cont. Maximise net utility: Given a utility function U maximize t (U(x t ) - af t ) File transfer: Transfer an amount of data F at minimum cost minimize t af t, such that t x t = F Fixed time file transfer: Same as above given a time constraint T minimize T t=1 af t, such that T t=1 x t = F

24 Conclusions It is possible to provide differential quality of service to users by means of a simple pricing scheme, where the network sends back the correct congestion prices to the user and the users choose how to react. Use the presented framework and play the defined distributed game, to assess the effectiveness and stability of various flow control schemes.

25 A modest proposal for preventing Internet congestion Andrew Odlyzko, 1997

26 The problem of congestion There is a demand for improved Quality of Service on the Internet, especially for reducing congestion. Proposed schemes are usually complicated and have high cost in development and operation. The basic problem is one of allocating a limited resource. A solution will have to involve some pricing mechanism.

27 The Paris Metro (or Subway) Until some years ago, the Paris Metro had two different classes of cars. 1 st and 2 nd class cars were identical in the number and quality of seats. 1 st class cars were more expensive and therefore less congested. Used by people who wanted a seat, silence etc. The system was self-regulated: if 1 st class cars became very popular, some people would decide they were not worth the extra cost and move to 2 nd class, thereby reducing congestion in 1 st class.

28 PMP PMP = Paris Metro Protocol Basic idea: partition the network into several logically separate networks. Each network has a fraction of the capacity of the entire network. All networks route packets using existing protocols on a best effort basis. Every network charges a different price for the same transfer.

29 The pricing mechanism is expected to affect user demand and thus modify the traffic loads. Networks with higher prices will be less congested, offering better QoS. PMP will not provide QoS guarantees, only expected levels of service. Not necessarily bad. QoS demands can be complicated and widely varying, impossible to satisfy all of them. PMP will offer few classes of service satisfying the majority of applications.

30 PMP advantages Approaches that dont use any network scheme and just increase capacity fail. With no pricing mechanism all packets travel at the highest priority and traffic grows to exhaust all available capacity. Policies trying to control bandwidth and provide QoS guarantees require the coordination of network nodes along the transmission path, causing delays and reinventing telephone technologies.

31 Details The number of subnetworks in PMP should be small (<5). This minimizes losses from not aggregating all the traffic and fits consumer preferences for simple schemes. Every subnetwork need not be assigned a fixed fraction of the total capacity. But this assignment, as well as the prices, should stay constant for extended periods of time, to allow usage patterns to stabilize.

32 Problems and Solutions To provide a higher QoS than the current Internet, the low-traffic networks will have to be lightly loaded. Will the capacity utilization have to be so low that it proves to be uneconomic? We dont know. There is not enough information about the operation and performance of the Internet. The user requirements are not precisely defined.

33 Problems and Solutions How to set the network prices? Use the variations in traffic patterns with the time of day. (Also customer surveys, complaints, etc.)

34 Problems and solutions cont. The average performance of the networks can be unstable. For example, an extended congestion period on the lowest-priced channel might lead some users to decide to pay extra and send their packets to the higher-priced networks. Degradation of service on one channel will thus propagate to other channels.

35 Problems and solutions cont. So, how to keep the average performance of the networks stable? Segregate the different types of services on different networks. E.g. introduce artificial packet delays in the lowest-priced network, making it unusable for certain applications. (damaged goods approach) This can be used to induce a more even load on the separate networks.

36 PMP implementation Use current protocols: IPv4 packets have an unused 3 bit priority field. Protocols without priority fields can operate at the lowest cost (and priority) network. Set the cost of the lowest-priority network to zero (at least initially). Use software to handle all traffic on a users machine according to user set rules, until applications are rewritten to take the multiple network choices into account.

37 PMP implementation cont. Also the service providers will have to: Change the router software to maintain separate queues for the different channels. Install software to count the packets sent by each user. Special provisions needed for multicasting.

38 The obstacle of the users and how to overcome it. [Disclaimer: The following proposals represent only the opinions of the author of the paper.] The need for usage-sensitive pricing appears irresistible. It has impeccable economic logic and practical evidence behind it. But it collides with the users preference for flat- rate pricing.

39 Flat rates Flat rate pricing can cause significant problems. Example: America Online When it switched to flat-rate pricing, usage increased considerably. Users had trouble getting a new connection and so left their connections open even when not using them.

40 Flat rates cont. Perverse incentives exist on the Internet to increase data transfers [Note: The paper is from 1997] To improve performance in Web surfing users use programs that prefetch linked pages when the connection is idle. With flat-rate pricing there is an incentive for the users to use them indiscriminately, even for minor benefits.

41 Flat rates cont. The survival of the Internet owes much to altruism and ignorance. TCP slows down transmissions when congestion occurs. Rogue versions of TCP can be used to speed up transmission of packets when delays or losses occur. No extra cost is incurred to their users under flat-rate pricing. Their extensive use can cause the network to collapse.

42 So why are flat rates used? Users know how much they will have to pay. Consumers overestimate their actual usage of a service. Hassle factor: consumers dont have to worry if every use of the service is worth its money. No need for traffic measurement and charging infrastructure.

43 How to introduce pricing without user complaints. Users have accepted usage-sensitive rates in a variety of services (e.g. long distance calls) It should be possible to persuade users to accept usage-sensitive pricing. PMP should make the transition easier that with most other schemes, since the lowest-priced channel could be offered initially at zero cost per packet, thus behaving just like todays internet.

44 PMP offers a simple and easily understood pricing plan, which fits consumer desires. It does not offer service guarantees. No problem: most purchases are made on the basis of expectations (magazine subscriptions, clothes, meals etc.) Consumers are willing to accept occasional deviations from the expected quality of service, as long as the average quality is predictable enough.

45 Auctions and consumers Consumer and business behavior can be economically irrational. Free enterprise companies prefer the socialist method of rationing by queue to that of rationing by price. Laws against ticket scalping are common and widely supported. But to most economists scalpers fulfill a socially useful role: they provide tickets to those that are willing to pay the most for them. Why dont theaters and sports arenas adjust ticket prices to clear the market and appropriate some of the gain the scalpers obtain?

46 Auctions and consumers cont. It seems that the cause lies in the consumers irrationall economic behavior. The challenge is to design pricing schemes that approach the efficiency of auction mechanism, but respect the consumer aversion to the auction. It is important to convince consumers of the fairness of pricing schemes.

47 Other pricing schemes Charge packets when they contribute to congestion. –Complicated implementation Use a set of service classes and priorities –Substantial overhead –Low priority classes can be denied any bandwidth if traffic from higher priority classes exceeds some threshold.

48 Gains from network segmentation We will examine simple models of the gains obtained by logically separate networks working at different utilization levels.

49 Two types of demand for data tranfer. Type A users are insensitive to delays (e.g. bulk file transfers) When the price per byte is x, type A users wish to send ax -1 e -x bytes (per hour). They generate network revenues: ae -x

50 The cost of operating a network that carries w bytes is cw 3/4 for some constant c > 0. This is a conservative assumption, since it corresponds to less than a 16% reduction in costs when the network doubles in size (2 3/4 = 1.681…). In practice the reductions are higher, corresponding to even smaller exponents.

51 If there are only type A users, the cost of the network will equal the revenues: ae -x = c(ax -1 e -x ) 3/4 x 3 e -x = a -1 c 4 The unique maximum of x 3 e -x occurs at x = 3 and equals 1.344… For c 4 > 27ae -3 there is no price x that will recover costs. For c 4 < 27ae -3 there are two solutions for x and we prefer the smaller one x A, that corresponds to higher traffic and revenue.

52 Type B users use a network only when its utilization rate is at most half of that acceptable to type A users. This assumption is conservative because even small reductions in network loads produce substantial improvements in service. When the price per byte is x, type B users wish to send bx -1 e -x bytes (per hour). They generate network revenues: be -x

53 Constructing a separate network for type B users will cost c(2bx -1 e -x ) 3/4 The 2 is because of the lower utilization rate. A price x that will equalize revenue and costs is found by solving: x 3 e -x = 8b -1 c 4 and exists when 27b 8c 4 e 3. Let x B denote the minimal solution (if more than one exists).

54 Single network for both Case 1: Single network with single price for both types of users. At price x would have revenue (a+b)e -x and will cost c(2(a+b)x -1 e -x ) 3/4 because its average utilization would have to be half of a network meant for type A users alone.

55 The price x that equalizes revenue and costs is found by solving: x 3 e -x = 8(a+b) -1 c 4 and exists when 27(a+b) 8c 4 e 3. Let x AB denote the minimal solution. If b > 7a (demand from type B users is larger), type A users benefit from lower prices: x AB < x A If b x A and type A users pay more (and also get better service but they dont need it) Type B users benefit always from having type A users on their network, because prices are always reduced.

56 Separate logical networks Case 2: Same physical network, logically separated into two for each type of user. Price for type A users is y, for type B is z. Revenue: ae -y + be -z Cost of the network: c(ay -1 e -y + 2bz -1 e -z ) -3/4

57 y and z must satisfy: ae -y + be -z = c(ay -1 e -y + 2bz -1 e -z ) -3/4 Great freedom of choices. If y or z we reduce to networks exclusively used by type A and type B users respectively.

58 Example 1 a = b = 3, c = 1 x A = x B does not exist x AB = y = 0.9 z = Type A users gain about 5.5%. Type B gain in that they are offered a service. networktrafficrevenue A only B only-- A + B on single network A + B on logically separate networks

59 Example 2 a = 20, b = 10, c = 1 x A = x B = x AB = y = 0.42 z = Type A users experience once more only a slight gain. Type B users pay a lot less. networktrafficrevenue A only B only A + B on single network A + B on logically separate networks

60 Example 3 a = 10, b = 20, c = 1 x A = x B = x AB = y = 0.53 z = The network provider is almost indifferent between having two separate networks and one giving all traffic the quality demanded by type B users. networktrafficrevenue A only B only A + B on single network A + B on logically separate networks

61 In all the examples, gains to type A users are small. This might explain why there has not been much pressure from users of the Internet (whose application are designed to work well in the presence of congestion) for higher quality of service. In the examples, a and b are comparable, corresponding to comparable traffic from type A and type B users. This might not be true now, but will most probably be the case, if the quality of service on the Internet is improved.