Colyseus: A Distributed Architecture for Online Multiplayer Games Ashwin Bharambe, Jeffrey Pang, and Srinivasan Seshan Carnegie Mellon University

Online First-Person Shooters Multiplayer architecture for multiplayer first-person shoot games. low latency required weakly consistent state tolerable spatial contiguity allows pre-fetching Traditionally this is done with a client-server model leads to bottlenecks can only sustain user numbers in the dozens

Colyseus Single-copy replication model maintains consistency by serializing updates mirrors existing server model cuts latency (reading from local copy), but sacrifices consistency in game-wide state DHT lookup for object queries both random and range-query DHTs implemented range-query DHT works well since object queries will always be in contiguous spatial regions DHT query delay overcome by anticipating which objects will be needed soon and pre-loading them

Replica Manager Follows Tunable Availability and Consistency Tradeoffs (TACT) model depending on specific game characteristics, developers can select either more availability or consistency - consistency lowers availability (increases lag), availability lowers conistency. synchronizes replicas to primary changes are delta-encoded, sent to primary, then distributed serially to all replicas when a node becomes interested in a replica (i.e. the player is near that object), it registers with the primary and receives updates directly (decoupled discovery/synchronization) creates new replicas deletes replicas that are no longer needed fast moving objects (missiles) use a special case attachment so that they are automatically sent to nodes that request the object they are attached to (the person who fired the missile)

Range-Queriable DHT adjacent nodes responsible for adjacent keys both standard random DHT and rangeDHT implemented with Mercury adjacent nodes responsible for adjacent keys player x,y coordinates used as key, then game can request other objects that are near the player from adjacent nodes predictions based on current player motion can be used to pre-load upcoming objects with a known average DHT lookup time, the prediction can be tweaked so that objects finish loading about when they are needed

DHT comparison

Evaluation Experimental Setup Modified Quake II to work with Colyseus Emulab used to simulate virtual servers no link capacity constraint, but ties end-to-end latency to measured samples artificially dilate time to counter slow virtual servers model game based on density statistics for a quake III map ( Zipf distribution) Modified Quake II to work with Colyseus Mercury rangeDHT variable size bounding box corresponding to visible objects for a character used as area-of-interest client/server messaging remains intact so unmodified engines could connect to any of the p2p nodes as if it were a server Custom map w/ bots for workload Emulab testbed

Results

Discussion Colyseus enables multiplayer first-person-shooter games to handle hundreds of players, instead of dozens since FPS games have very high demands for low latency and consistency, extending this architecture to other game types, like role playing games, is very feasible adaptation of commercial Quake II shows that this is feasible for other games in a production environment this method opens up many possibilities for cheating (a node could be modified to request objects that it shouldn’t ‘see’, for instance), but more work could be done to address those threats.

Related Work Real-time strategy (command & conquer) parallel simulation often used, since consistency is very important often limited to less than 10 players Online role playing (world of warcraft, second-life) cell based centralized server or server cluster Distributed Virtual Reality Environments similar goals to Colyseus, but specific not catering to common game applications

Issues why not just evaluate actual gameplay? (custom map, all bots seems a little suspect) synchronization decoupling seems to introduce a bottleneck. It’s still better than a server model since a primary replica might be the only one on a node, but the node still has to handle all traffic for that replica how are node failures (player logs off) handled? how do you handle updates on objects in a view that is already inconsistent, especially since the node cannot know if it’s view is consistent.