The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, Mark Claypool Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA

Aug 2004 NetGames'04, Portland, OG, USA2 General Motivation In 2003, despite economic downturn, games only entertainment industry to grow [“Essential Facts”, ESA ] For frequent game players, 43% play online [ESA ] First-Person Shooters a top selling genre –11.5% [“Essential Facts”, ESA, 2003] Unreal Tournament 2003 –Popular (1700 servers, 4400 players) [Gamespy – Oct 2003] –Award winning [“Best of Show,” E 3 expo] –Studied less than Quake-derived FPS

Aug 2004 NetGames'04, Portland, OG, USA3 Specific Motivation – The Network and Network Games Network games can be demanding in terms of network requirements –Capacity? (Not usually) –Latency? (Sometimes) What parts of a FPS game … Moving? Shooting? Quantitative effect? –Loss? (Unknown) Retransmission will increase latency Knowing game constraints useful for –Building better network games –Building better networks to support games (QoS) Our goal –  Quantitative effects of loss and latency on the performance of UT2003

Aug 2004 NetGames'04, Portland, OG, USA4 Outline Introduction  Experiments  Analysis Conclusions

Aug 2004 NetGames'04, Portland, OG, USA5 Methodology Characterize fundamental user interaction components of FPS games Design maps that exercise each type of interaction Construct test environment for measuring the effects of latency and loss Conduct user studies Analyze results

Aug 2004 NetGames'04, Portland, OG, USA6 FPS User Interaction Components Movement –Simple (Moving in a straight line) –Complex (Running, jumping, turning…) Shooting PrecisionExample Weapons HighShock Rifle, Link Gun, Lightning Gun MediumAssault Rifle, Mini-Gun, Bio Rifle LowFlak Cannon, Rocket Launcher Movement and Shooting Design maps for each component

Aug 2004 NetGames'04, Portland, OG, USA7 Controlling Loss and Latency Ethereal for traces –UT2003 is Client-Server, uses UDP, Server updates every 50 ms NIST Net for Router  Control network characteristics to each client –Control loss [0%, 6% ] –Control latency [0 ms, 400 ms]

Aug 2004 NetGames'04, Portland, OG, USA8 Outline Introduction  Experiments  Analysis –Application Level  –Network Level –User Level Conclusions

Aug 2004 NetGames'04, Portland, OG, USA9 Complex Movement Movement not affected by loss or latency Flux2 map with pre-defined obstacle course

Aug 2004 NetGames'04, Portland, OG, USA10 Precision Shooting Precision shooting not affected by loss Precision shooting is affected by latency CTF-Face3 map with lightning gun versus dodging human opponent

Aug 2004 NetGames'04, Portland, OG, USA11 Restricted Deathmatch Movement and shooting not affected by loss Movement and shooting affected by latency Less significantly than shooting alone Training Day map with only medium precision weapons versus bot

Aug 2004 NetGames'04, Portland, OG, USA12 Full Deathmatch Movement and shooting not affected by loss Movement and shooting barely affected by latency Training Day map with full precision weapons versus Bot

Aug 2004 NetGames'04, Portland, OG, USA13 Outline Introduction  Experiments  Analysis –Application Level  –Network Level  –User Level Conclusions

Aug 2004 NetGames'04, Portland, OG, USA14 Network Turbulence No visible effects of loss or latency  Holds for packet sizes, inter packet times 5

Aug 2004 NetGames'04, Portland, OG, USA15 Outline Introduction  Experiments  Analysis –Application Level  –Network Level  –User Level  Conclusions

Aug 2004 NetGames'04, Portland, OG, USA16 User-Level Analysis 75 ms latency, users noticed sluggishness –Felt play was worse –True even for unrestricted tests 100 ms latency always noticeable –Play less enjoyable –Frustrating for precise shooting 0-3% loss rarely even noticed 3%+ loss sometimes noticed, but only because gun effects not always displayed

Aug 2004 NetGames'04, Portland, OG, USA17 Conclusions FPS games not affected by typical Internet packet loss –Loss up to 3% and not even noticed –TCP fairness at higher bitrates? FPS games can be affected by latency –Perceived impact as low as 75 ms –Quantitative impact at 100 ms At the network level: –Small packets and low bitrates –Turbulence consistent over all network conditions

Aug 2004 NetGames'04, Portland, OG, USA18 Future Work Effects of variance in latency (jitter) User adaptation strategy Build model for FPS interactions so can apply to different FPS games (i.e. BF1942)

The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, Mark Claypool Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA