1 A Network Communication System for Large-Scale, High-Speed Games Ashwin Bharambe, Jeff Pang, Srini Seshan, Xinyu Zhang Carnegie Mellon John Douceur, Jacob R. Lorch, Thomas Moscibroda Microsoft Research

2 P2P Games Internet Primary object Local View Replica objects

3 P2P Games Internet Primary object Local View Replica objects

4 P2P Games Internet Primary object Local View Replica objects Goal: 1000 players in the same battle at the same time

5 Problem 1: Insufficient Capacity Update Size: ~35 bytes/frame Update Rate: 20 frames/sec  Need 14 Mb/s upload BW per peer Real P2P topologies: –800 Kb/s to 1.5Mb/s per peer (on average)

6 Solution 1: Subscriber Set Players can only focus on ~5 objects at the same time Each player divides other peers into two classes: –Subscriber Set: players focused on me Update every frame –Best Effort: everyone else Update with “guidance” once per second Who goes in my Subscriber Set? –Attention(i,j) = how much a player i is focused on player j –Attention(i,j) sent from peer i to peer j periodically –Peers with the highest Attention(i,j) go in peer j’s Set

7 Solution 1: Subscriber Set Who goes in my Subscriber Set? –Attention(i,j) = how much a player i is focused on player j –Attention(i,j) sent from peer i to peer j periodically –Peers with the highest Attention(i,j) go in peer j’s Set d1d1 d2d2 θ1θ1 θ2θ2 Attention(i,j) = f proximity (d 1 )f aim (θ 1 )f interaction-recency (t 1 )++ ji

8 Projected Scalability With Donnybrook Without Donnybrook

9 Problem 2: Bandwidth Heterogeneity Best Effort updates still scale quadratically Still need ~1 Mb/s per peer  Limited by low bandwidth peers

10 Solution 2: Guidance Forwarding Static Overlay Multicast –Need low latency  limit tree depth to 2 –Can’t queue updates  have senders send to forwarders at 20 updates/sec –Generally have centralized match making  (replicated) centralized coordinator picks forwarders

11 Projected Scalability

12 Some players more interesting than others  More players focus on them  They need to send more frequent updates Subscriber sets change dynamically (~60% turnover per second!)  Can’t use static overlay multicast  Limited by BW of popular players Problem 3: Interest Heterogeneity

13 Solution 3: Update Dissemination Alterative 1: –Static forwarder groups Alternative 2: –Centralized forwarder assignment Alternative 3: –Uncoordinated forwarder assignment AB I need BW! Use B A B Pick random

14 Projected Scalability

15 Evaluation Detailed packet level simulation –Empirical simulation models E2E latency Jitter Access Bandwidth Packet Loss Quake III-like CTF workload... many other details … But no churn

16 Metric: updates delivered roughly on time –updates delivered too late are useless 3-5% loss rate acceptable for games Alternative Comparison

17 Performance Scaling

18 Summary