1. Battle of Botcraft: Fighting Bots in Online Games with Human Observational Proofs Steven Gianvecchio, Zhenyu Wu, Mengjun Xie, and Haining Wang

2. Outline  Background  Game Playing Characterization  HOP System  Experiments  Limitations  Conclusion

3. Outline  Background  Game Playing Characterization  HOP System  Experiments  Limitations  Conclusion

4. Background Online Games  In 2008, online game revenues $7.6B  about half from massively multiplayer online games (MMOGs) ex. World of Warcraft (WoW)  MMOG currency trades for real currency  players can make real money  A major problem is cheating

5. Background Game Bots  A common cheat is use of game bots  able to amass game currency  cause hyper-inflation  To combat bots  process monitors, ex. Warden for WoW  human interactive proofs (HIPs)  legal action

6. Background Game Bots  Glider – a popular WoW bot  controls game via mouse / keyboard APIs  uses profiles, i.e., configurations and waypoints  able to evade Warden  Blizzard sued MDY (maker of Glider)  awarded $6.5M

7. Outline  Background  Game Playing Characterization  HOP System  Experiments  Limitations  Conclusion

8. Game Playing Characterization Input Data Collection  World of Warcraft game  RUI program (with modifications)  records user-input events  converts events to user-input actions ex. move + move + press + release = point-and-click  computes user-input action statistics

9. Game Playing Characterization Game Bot  10 Glider profiles (configurations and waypoints)  40 hours  half with warrior and half with mage  levels 1 to mid-30s

10. Human  30 humans  55 hours Game Playing Characterization

11.  Human  well fit by Pareto distribution  Game Bot  more fast keystrokes  signs of periodic timing Keystroke Inter-arrival Time Distribution

12.  Human  fewer very short keystrokes  3.9% shorter than.12 secs  Game Bot  36.9% shorter than.12 secs  more signs of periodic timing Keystroke Duration Distribution

13.  Human  highly-variable speed at all displacements  Game Bot  linear speed increases  high-speed moves with zero displacment Point-and-Click Speed vs. Displacement

14.  Human  decays exponentially  only 14.1% of movements have 1.0 efficiency  Game Bot  81.7% of movements have 1.0 efficiency Point-and-Click / Drag-and-Drop Movement Efficiency

15.  Game Bot  no correlation between speed and direction Average Velocity for Point-and-Click

16.  Human  diagonal, symmetric, and bounded  diagonals faster than horizontal / vertical Average Velocity for Point-and-Click

17. Outline  Background  Game Playing Characterization  HOP System  Experiments  Limitations  Conclusion

18. HOP System  A behavioral approach  human observational proofs (HOPs)  The idea: certain tasks are difficult for a bots to perform like a human  passively observe differences  HOP-based game bot defense system  continuous monitoring  transparent to users

19. HOP System  Client-Side Exporter  transmits user-input actions  Server-Side Analyzer  processes and decides: bot or human

20. HOP System Neural Network  Inputs 1. duration 2. distance 3. displacement 4. move efficiency 5. speed 6. angle 7. virtual key # of inputs = # of actions * 7

21. HOP System Neural Network  Output – human or bot Decision Maker  “Votes” on series of outputs ex. {bot + bot + human} = bot

22. Outline  Background  Game Playing Characterization  HOP System  Experiments  Limitations  Conclusion

23. Experiments Experimental Setup  30 human players, 55 hours  10 Glider profiles, 40 hours  10-fold cross validation  test on a bot or human not in training set  10 different training sets

24. Experiments HOP System 1.# of actions (input to neural network) 2.# of nodes (in neural network) 3.threshold x (on neural network output) > x is bot, <= x is human 4.# of outputs per decision ex. {bot + bot + human} = bot

25. Experiments Configure 1. # of actions and 2. # of nodes  4 actions with 40 nodes TPR and TNR vs. # of Nodes and # of Accumulated Actions

26. Experiments Configure 3. threshold and 4. # of outputs  threshold 0.75 with 9 outputs per decision TPR and TNR vs. Threshold and # of Accumulated Outputs

27. Experiments Detection Results  Configured System  4 actions, 40 nodes, threshold 0.75, 9 outputs  Glider – avg. true positive rate of 0.998  Humans – true negative rate of 1.000 True Positive Rates for Bots

28. Experiments Decision Time  # of action * time per action  avg. 39.60 seconds Decision Time Distribution

29. Experiments Detection of Other Game Bots  MMBot in Diablo 2  different bot, different game  without retraining the neural network  MMBot – true positive rate of 0.864  Humans – true negative rate of 1.000

30. Outline  Background  Game Playing Characterization  HOP System  Experiments  Limitations  Conclusion

31. Limitations Experimental Limitations  Size  30 not enough  Lab vs. Home  mostly in-lab  Character equipment / levels  Other bots and games

32. Limitations (cont.) Potential Evasion  Interfere with client-side exporter  block user-input stream  manipulate user-input stream  Mimic human behavior  replay attacks  model human user-input

33. Conclusion  Game Play Characterization  95 hours of user-input traces  bots behave differently than humans  HOP System  exploits behavioral differences  compared to HIPs, HOPs are transparent and continuous  detects 99% of bots with no false positives  raises the bar for attacks

34. Questions? Thank You!

35. Questions? Thank You!

36. Questions? Thank You!

37. Experiments System Overhead  Memory  per user = 4 actions * 16 bytes + 16 outputs * 1 bit = 66B  server with 5,000 users = 330KB  CPU – P4 Xeon 3.0Ghz  95 hours of traces in 385ms = ~296 hours/sec  server with 5,000 users = ~1.4 hours/sec