1. Course Announcements Tomorrow, Jan 11, at 09:45, Lecture Hall H: “GPU Programming: Tips and Tricks” Ana Varbanescu Exam-related questions: contact Hai Xiang Lin and Ana Varbanescu Lab-related questions: contact Jie Shen s.shenjie@tudelft.nls.shenjie@tudelft.nl Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 1

2. PDS: MSc courses October 31, 2012 1.High-Performance Computing (now) (IN4049, Hai-Xiang Lin, Henk Sips, and Ana Varbanescu) 2.Cloud Computing (sep – nov 2012) (IN4392, Dick Epema and Alexandru Iosup) 3.Distributed Computing Systems (feb – apr 2013) (IN4391, Alexandru Iosup) 4.Parallel Algorithms and Parallel Computers (feb-jun 2013) (IN4026, Henk Sips and Cees Witteveen) 5.Seminar P2P Systems (feb-mar 2013) (IN4312, Dick Epema and Johan Pouwelse) 6.Distributed Algorithms (feb-mar 2013) (IN4150, Dick Epema) 2

3. Today’s Menu 1.Starter: Introduction to Cloud Computing 2.Main: Cloud Programming Models 3.Desert: Online Gaming as an HPC Problem Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 3

4. August 24, 2015 4

5. 5

6. What is Cloud Computing? 3. A Useful IT Service “Use only when you want! Pay only for what you use!” August 24, 2015 6

7. IaaS Cloud Computing VENI – @larGe: Massivizing Online Games using Cloud Computing Many tasks

8. August 24, 2015 8 HP Engineering Which Applications Need Cloud Computing? A Simplistic View… Demand Variability Demand Volume Taxes, @Home Low High Web Server Space Survey Comet Detected Sky Survey Pharma Research Social Gaming Online Gaming Social Networking Analytics SW Dev/Test Office Tools Tsunami Prediction Epidemic Simulation Exp. Research After an idea by Helmut Krcmar

9. Today’s Menu 1.Starter: Introduction to Cloud Computing 2.Main: Cloud Programming Models 3.Desert: Online Gaming as an HPC Problem Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 9

10. 2012-2013 10 Terms for Today’s Discussion Programming model = language + libraries + runtime system that create a model of computation (an abstract machine) = “an abstraction of a computer system” Wikipedia Examples: message-passing vs shared memory, data- vs task-parallelism, … Abstraction level = distance from physical machine Examples: Assembly low-level vs Java is high level Many design trade-offs: performance, ease-of-use, common-task optimization, programming paradigm, … Q: What is the best abstraction level?

11. Today’s Challenges eScience The Fourth Paradigm The Data Deluge and Big Data Possibly others 2012-2013 11

12. eScience (John Taylor, UK Sci.Tech., 1999) A new scientific method Combine science with IT Full scientific process: control scientific instrument or produce data from simulations, gather and reduce data, analyze and model results, visualize results Mostly compute-intensive, e.g., simulation of complex phenomena IT support Infrastructure: LHC Grid, Open Science Grid, DAS, NorduGrid, … From programming models to infrastructure management tools Examples * physics, Bioinformatics, Material science, Engineering, CompSci 2012-2013 12 Q: Why is CompSci an example here?

13. The Fourth Paradigm: The What Thousand years ago: science was empirical describing natural phenomena Last few hundred years: theoretical branch using models, generalizations Last few decades: a computational branch simulating complex phenomena Today (the Fourth Paradigm): data exploration unify theory, experiment, and simulation Data captured by instruments or generated by simulator Processed by software Information/Knowledge stored in computer Scientist analyzes results using data management and statistics 2012-2013 13 Source: Jim Gray and “The Fourth Paradigm”, http://research.microsoft.com/en-us/collaboration/fourthparadigm/ http://research.microsoft.com/en-us/collaboration/fourthparadigm/ Q1: What is the Fourth Paradigm? From Hypothesis to Data Q2: What are the dangers of the Fourth Paradigm?

14. What is “Big Data”? Very large, distributed aggregations of loosely structured data, often incomplete and inaccessible Easily exceeds the processing capacity of conventional database systems Principle of Big Data: “When you can, keep everything!” Too big, too fast, and doesn’t comply with the traditional database architectures 2011-2012 14

15. Time [units] What is a Bag of Tasks (BoT)? A System View Why Bag of Tasks? From the perspective of the user, jobs in set are just tasks of a larger job A single useful result from the complete BoT Result can be combination of all tasks, or a selection of the results of most or even a single task 2012-2013 15 BoT = set of jobs sent by a user… …that start at most Δs after the first job Iosup et al., The Characteristics and Performance of Groups of Jobs in Grids, Euro-Par, LNCS, vol.4641, pp. 382-393, 2007. Q: What is the user’s view?

16. BoTs Became the Dominant Programming Model for Grid Computing 2012-2013 16 Iosup and Epema: Grid Computing Workloads. IEEE Internet Computing 15(2): 19-26 (2011)

17. Practical Applications of the BoT Programming Model Parameter Sweeps in Condor [1/4] Sue the scientist wants to “Find the value of F(x,y,z) for 10 values for x and y, and 6 values for z” Solution: Run a parameter sweep, with 10 x 10 x 6 = 600 parameter values Problem of the solution: Sue runs one job (a combination of x, y, and z) on her low-end machine. It takes 6 hours. That’s 150 days uninterrupted computation on Sue’s machine! 2012-2013 17 Source: Condor Team, Condor User’s Tutorial. http://cs.uwisc.edu/condor http://cs.uwisc.edu/condor

18. Practical Applications of the BoT Programming Model Parameter Sweeps in Condor [2/4] Universe = vanilla Executable = sim.exe Input = input.txt Output = output.txt Error = error.txt Log = sim.log Requirements = OpSys == “WINNT61” && Arch == “INTEL” && (Disk >= DiskUsage) && ((Memory * 1024)>=ImageSize) InitialDir = run_$(Process) Queue 600 2012-2013 18 Source: Condor Team, Condor User’s Tutorial. http://cs.uwisc.edu/condor http://cs.uwisc.edu/condor Also passed as parameter to sim.exe Complex SLAs can be specified easily

19. Practical Applications of the BoT Programming Model Parameter Sweeps in Condor [3/4] % condor_submit sim.submit Submitting job(s)............................................................................................................................................................................................................................................................... Logging submit event(s)............................................................................................................................................................................................................................................................... 600 job(s) submitted to cluster 3. 2012-2013 19 Source: Condor Team, Condor User’s Tutorial. http://cs.uwisc.edu/condor http://cs.uwisc.edu/condor

20. Practical Applications of the BoT Programming Model Parameter Sweeps in Condor [4/4] % condor_q -- Submitter: x.cs.wisc.edu : : x.cs.wisc.edu ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD 3.0 frieda 4/20 12:08 0+00:00:05 R 0 9.8 sim.exe 3.1 frieda 4/20 12:08 0+00:00:03 I 0 9.8 sim.exe 3.2 frieda 4/20 12:08 0+00:00:01 I 0 9.8 sim.exe 3.3 frieda 4/20 12:08 0+00:00:00 I 0 9.8 sim.exe... 3.598 frieda 4/20 12:08 0+00:00:00 I 0 9.8 sim.exe 3.599 frieda 4/20 12:08 0+00:00:00 I 0 9.8 sim.exe 600 jobs; 599 idle, 1 running, 0 held 2012-2013 20 Source: Condor Team, Condor User’s Tutorial. http://cs.uwisc.edu/condor http://cs.uwisc.edu/condor

21. Asterix B-tree Ecosystems of Big-Data Programming Models 2012-2013 21 Dremel Service Tree SQLHivePigJAQL MapReduce ModelAlgebrixPACT MPI/ Erlang LFSLFS NepheleHyracksDryadHadoop/ YARN Haloop DryadLINQScope Pregel HDFS AQL CosmosFS Azure Engine Tera Data Engine Adapted from: Dagstuhl Seminar on Information Management in the Cloud, http://www.dagstuhl.de/program/calendar/partlist/?semnr=11321&SUOG http://www.dagstuhl.de/program/calendar/partlist/?semnr=11321&SUOG Azure Data Store Tera Data Store Storage Engine Execution Engine Voldemort High-Level Language Programming Model GFS BigQueryFlume Flume Engine S3 Dataflow Giraph SawzallMeteor * Plus Zookeeper, CDN, etc. Q: Where does MR-on-demand fit?Q: Where does Pregel-on-GPUs fit?

22. Today’s Menu 1.Starter: Introduction to Cloud Computing 2.Main: Cloud Programming Models 3.Desert: Online Gaming as an HPC Problem Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 22

23. 23 Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology Our team: Undergrad Adrian Lascateu, Alexandru Dimitriu (UPB, Romania), …, Grad Siqi Shen, Yong Guo (TU Delft, the Netherlands), … Staff Dick Epema, Johan Pouwelse, Henk Sips (TU Delft), Thomas Fahringer, Radu Prodan (U. Innsbruck), Nicolae Tapus, Vlad Posea (UPB), … Massivizing Online Games: High Performance Computing and High Quality Time HPDC-Trends, Amsterdam, Mar 2012

24. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 24 @large: Massivizing Online Games as an HPC Problem Online Gaming used to be multimedia, is now HPC Online Gaming used to be networking, is now all HPC Online Gaming used to be v-worlds, is now many apps Online Gaming used to be art, may now be computing

25. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 25 What’s in a name? MSG, MMOG, MMO, … 1.Virtual World Sim Explore, do, learn, socialize, compete + 2.Game Data Player stats and relationships, others + 3.Game Content Graphics, maps, puzzles, quests, culture Massively Social Gaming = (online) games with massive numbers of players (100K+), for which social interaction helps the gaming experience Over 250,000,000 active players

26. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 26 MSGs are a Popular, Growing Market 25,000,000+ subscribed players (from 250,000,000+ active) Over 10,000 MSGs in operation Subscription market size $7.5B+/year, Zynga $600M+/year Sources: MMOGChart, own research.Sources: ESA, MPAA, RIAA.

27. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 27 Zynga, an Amazon WS User Sources: CNN, Zynga. Source: InsideSocialGames.com “Zynga made more than $600M in 2010 from selling in-game virtual goods.” S. Greengard, CACM, Apr 2011 Selling in-game virtual goods: “Zynga made est. $270M in 2009 from.” http://techcrunch.com/2010/05/03/zynga-revenue/ http://techcrunch.com/2010/05/03/zynga-revenue/

28. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 28 World of Warcraft, a Traditional HPC User (since 2003) 10 data centers 13,250 server blades, 75,000+ cores 1.3PB storage 68 sysadmins (1/1,000 cores) http://www.datacenterknowledge.com/archives/2009/11/25/wows-back-end-10-data-centers-75000-cores/

29. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 29 Bungie, Computing then Serving 1.4PB/yr. Halo 3 is one of the many successful games Halo 3 players get, in 1.4PB Detailed player profiles Detailed usage stats Ranking CERN produces ~15PB/year (10x larger)

30. Massivizing Social Games: Yesterday, Today, and the Next Five Years – A. Iosup 30 Agenda 1.What’s in a Name? 2.Three Current Challenges 1.Platform Scalability Challenge 2.Gaming Analytics Challenge 3.Content Generation Challenge 3.Conclusion

31. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 31 @large Research Challenge: V-World Platform for MMOGs Scaling quickly to millions of players - 1M in 4 days, 10M in 2 months - Up-front and operational costs - Performance, Scalability, & Cost

32. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 32 Impact on Game Experience Responsive game Unresponsive game August 24, 2015 [Source: Nae, Iosup, and Prodan, ACM SC 2008 and IEEE TPDS 2011]

33. 33 Using data centers for dynamic resource allocation Main advantages: 1. Significantly lower over-provisioning 2. Efficient coverage of the world is possible Proposed hosting model: dynamic Massive join Massive leave [Source: Nae, Iosup, and Prodan, ACM SC 2008]

34. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 34 Resource Provisioning and Allocation Static vs. Dynamic Provisioning 250% 25% [Source: Nae, Iosup, and Prodan, ACM SC 2008]

35. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 35 Resource Provisioning and Allocation Compound Metrics Trade-off Utility-Cost still needs investigation Performance and Cost are not both improved by the policies we have studied Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure- as-a-Service Clouds, CCGrid, 2012.

36. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 36 (Variable) Blackbox Performance Engineering Performance Evaluation of Four Commercial Clouds Amazon EC2, GoGrid, Elastic Hosts, Mosso Resource acquisition Single- and Multi-Instance benchmarking Low compute and networking performance 1 Performance variability over time 2 1- Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, IEEE TPDS, 2011, http://www.st.ewi.tudelft.nl/~iosup/cloud-perf10tpds_in-print.pdf http://www.st.ewi.tudelft.nl/~iosup/cloud-perf10tpds_in-print.pdf 2- Iosup et al., On the Performance Variability of Production Cloud Services, CCGrid 2011, pds.twi.tudelft.nl/reports/2010/PDS-2010-002.pdfpds.twi.tudelft.nl/reports/2010/PDS-2010-002.pdf

37. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 37 SLAs Supporting Real Ecosystems Multivariate SLA Languages -Specification of SLAs -Multivariate -Include provisions for faults -Include detailed penalties: compensation for temporary QoS violations -Use existing IaaS cloud SLA specifications -How would the MMOG ecosystem operate? [TPDS 2011] -How to specify, use, and optimize for SLAs? (upcoming) Nae, Prodan,Iosup. A Cloud-Based Operational SLA Negotiation Model for MMOGs, (upcoming).

38. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 38 @large Research Challenge: Content Generation for MMOGs Generating content on time for millions of players - Player-customized: Balanced, Diverse, Fresh - Up-front and operational costs - Response time, Scalability, & Cost

39. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 39 (Procedural) Game Content (Generation) Game Bits Texture, Sound, Vegetation, Buildings, Behavior, Fire/Water/Stone/Clouds Game Space Height Maps, Bodies of Water, Placement Maps, … Game Systems Eco, Road Nets, Urban Envs, … Game Scenarios Puzzle, Quest/Story, … Game Design Rules, Mechanics, … Hendricks, Meijer, vd Velden, Iosup, Procedural Content Generation for Games: A Survey, ACM TOMCCAP, 2012 Derived Content NewsGen, Storification

40. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 40 The POGGI Content Generation Framework Only the puzzle concept, and the instance generation and solving algorithms, are produced at development time * A. Iosup, POGGI: Puzzle-Based Online Games on Grid Infrastructures, EuroPar 2009 (Best Paper Award)

41. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 41 @large Research Challenge: Continuous Analytics for MMOGs Analyzing the behavior of millions of players, on-time - Data mining, data access rights, cost v. accuracy, … - Reduce upfront costs - Low response time & Scalable - Large-scale Graph Processing

42. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 42 The CAMEO Framework 1.Address community needs Can analyze skill level, experience points, rank Can assess community size dynamically 2.Using on-demand technology: Cloud Comp. Dynamic cloud resource allocation, Elastic IP 3.Data management and storage: Cloud Comp. Crawl + Store data in the cloud (best performance) 4.Performance, scalability, robustness: Cloud Comp. A. Iosup, CAMEO: Continuous Analytics for Massively Multiplayer Online Games on Cloud Resources. ROIA, Euro-Par 2009 Workshops, LNCS 6043, (2010)

43. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 43 @large: Sample Analytics Results Skill Level Distribution in RuneScape Runescape: 135M active accounts, 7M active (2008) High-scoring players: 1.8M (2007) / 3.5M (2010) (largest MMOG msmt.) Player skill: distribution changes over time * A. Iosup, POGGI: Puzzle-Based Online Games on Grid Infrastructures EuroPar 2009 (Best Paper Award) A. Iosup, A. Lascateu, N. Tapus, CAMEO: Enabling Social Networks for Massively Multiplayer Online Games through Continuous Analytics and Cloud Computing, ACM NetGames 2010.

44. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 44 @large: Social Everything! Social Network=undirected graph, relationship=edge Community=sub-graph, density of edges between its nodes higher than density of edges outside sub-graph (Analytics Challenge) Improve gaming experience Ranking / Rating Matchmaking / Recommendations Play Style/Tutoring Self-Organizing Gaming Communities Player Behavior

45. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 45 Bridge Base Online (BBO): 1M+ players, top free site Dataset: 100K players 9K group Social relationships from bridge pairing Large (~10K) online social groups can coordinate Identified player behavior community builder, community member, random player, faithful player @large: Sample Analytics Results Activity and Social Network Interaction group-socnet Coordinated large-scale social group M. Balint, V. Posea, A. Dimitriu, and A. Iosup, An Analysis of Social Gaming Networks in Online and Face to Face Bridge Communities, LSAP 2011.

46. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 46 @large: Sample Analytics Results Analysis of Meta-Gaming Network “When you play a number of games, not as ends unto themselves but as parts of a larger game, you are participating in a metagame.” (Dr. Richard Garfield, 2000) XFire: since 2008 (3+ years), 500K of 20M players * A. Iosup, POGGI: Puzzle-Based Online Games on Grid Infrastructures EuroPar 2009 (Best Paper Award) S. Shen, and A. Iosup, The XFire Online Meta-Gaming Network: Observation and High-Level Analysis, MMVE 2011 PhD

47. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 47 Summary Current Technology @large: The Future Happy players Happy cloud operators Million-user, multi-bn market V-World, Content, Analytics Massivizing Online Gaming Upfront payment Cost and scalability problems Makes players unhappy @large: Our Vision HPC has to help Economy of scale with clouds @large: Ongoing Work Content: POGGI Framework Platform: edutain@grid Analytics: CAMEO Framework Publications Gaming and Clouds 2008: ACM SC 2009: ROIA, CCGrid, NetGames, EuroPar (Best Paper Award), … 2010: IEEE TPDS, Elsevier CCPE 2011: Book Chapter CAMEO, IEEE TPDS, IJAMC 2012: IPDPS, CCGrid, … Graduation (Forecast) 2012—14: 3PhD, 6Msc, 6BSc

48. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 48 Thank you for your attention! Questions? Suggestions? Observations? Alexandru Iosup A.Iosup@tudelft.nl http://www.pds.ewi.tudelft.nl/~iosup/ (or google “iosup”) Parallel and Distributed Systems Group Delft University of Technology A.Iosup@tudelft.nl http://www.pds.ewi.tudelft.nl/~iosup/ - http://www.st.ewi.tudelft.nl/~iosup/research.htmlhttp://www.st.ewi.tudelft.nl/~iosup/research.html - http://www.st.ewi.tudelft.nl/~iosup/research_gaming.htmlhttp://www.st.ewi.tudelft.nl/~iosup/research_gaming.html - http://www.st.ewi.tudelft.nl/~iosup/research_cloud.htmlhttp://www.st.ewi.tudelft.nl/~iosup/research_cloud.html More Info: Do not hesitate to contact me…

49. PDS: MSc courses October 31, 2012 1.High-Performance Computing (now) (IN4049, Hai-Xiang Lin, Henk Sips, and Ana Varbanescu) 2.Cloud Computing (sep – nov 2012) (IN4392, Dick Epema and Alexandru Iosup) 3.Distributed Computing Systems (feb – apr 2013) (IN4391, Alexandru Iosup) 4.Parallel Algorithms and Parallel Computers (feb-jun 2013) (IN4026, Henk Sips and Cees Witteveen) 5.Seminar P2P Systems (feb-mar 2013) (IN4312, Dick Epema and Johan Pouwelse) 6.Distributed Algorithms (feb-mar 2013) (IN4150, Dick Epema) 49

50. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 50 Multi-Resource Provisioning/Release Time for multi-resource increases with number of resources Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, (IEEE TPDS 2011). Q1

51. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 51 GAE Dataset: Run Service Fibonacci [ms]: Time it takes to calculate the 27 th Fibonacci number Highly variable performance until September Last three months have stable performance (low IQR and range) 51 Q2 Iosup, Yigitbasi, Epema. On the Performance Variability of Production Cloud Services, (IEEE CCgrid 2011).

52. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 52 Online Scheduling + Optimization ExPERT O. Agmon Ben-Yehuda, A. Schuster, A. Sharov, M. Silberstein, and A. Iosup, ExPERT: Pareto-Efficient Task Replication on Grids and a Cloud, IPDPS'12.

53. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 53 Performance Metrics Makespan very similar Very different job slowdown Q3 Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure- as-a-Service Clouds, (submitted). PDS Tech.Rep.2011-009

54. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 54 Cost Metrics Very different results between actual and charged Cloud charging function an important selection criterion All policies better than Startup in actual cost Policies much better/worse than Startup in charged cost Charged CostActual Cost Q3 Villegas, Antoniou, Sadjadi, Iosup. An Analysis of Provisioning and Allocation Policies for Infrastructure- as-a-Service Clouds, (submitted). PDS Tech.Rep.2011-009

55. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 55 Single Resource Provisioning/Release Time depends on instance type Boot time non-negligible Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, (IEEE TPDS 2011). Q1

56. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 56 CPU Performance of Single Resource ECU definition: “a 1.1 GHz 2007 Opteron” ~ 4 flops per cycle at full pipeline, which means at peak performance one ECU equals 4.4 gigaflops per second (GFLOPS) Real performance 0.6..0.1 GFLOPS = ~1/4..1/7 theoretical peak Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, (IEEE TPDS 2011). Q1

57. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 57 HPLinpack Performance (Parallel) Low efficiency for parallel compute-intensive applications Low performance vs cluster computing and supercomputing Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, (IEEE TPDS 2011). Q1

58. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 58 Performance Stability (Variability) High performance stability for the best-performing instances Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, (IEEE TPDS 2011). Q1

59. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 59 All services exhibit time patterns in performance EC2: periods of special behavior SDB and S3: daily, monthly and yearly patterns SQS and FPS: periods of special behavior August 24, 2015 59 AWS Dataset (4/4): Summary Q2 Iosup, Yigitbasi, Epema. On the Performance Variability of Production Cloud Services, (IEEE CCgrid 2011).

60. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 60 August 24, 2015 60 Read Latency [s]: Time it takes to read a “User Group” Yearly pattern from January to August The last four months of the year exhibit much lower IQR and range More stable performance for the last five months Probably due to software/infrastructure upgrades GAE Dataset (2/4): Datastore Q2 Iosup, Yigitbasi, Epema. On the Performance Variability of Production Cloud Services, (IEEE CCgrid 2011).

61. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 61 August 24, 2015 61 PUT [ms]: Time it takes to put 1 MB of data in memcache. Median performance per month has an increasing trend over the first 10 months The last three months of the year exhibit stable performance GAE Dataset (3/4): Memcache Q2 Iosup, Yigitbasi, Epema. On the Performance Variability of Production Cloud Services, (IEEE CCgrid 2011).

62. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 62 All services exhibit time patterns Run Service: daily patterns and periods of special behavior Datastore: yearly patterns and periods of special behavior Memcache: monthly patterns and periods of special behavior URL Fetch: daily and weekly patterns, and periods of special behavior August 24, 2015 62 GAE Dataset (4/4): Summary Q2 Iosup, Yigitbasi, Epema. On the Performance Variability of Production Cloud Services, (IEEE CCgrid 2011).

63. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 63 The POGGI Framework Focus on game content generation on grids Use existing middleware Control MMOG-specific workload demands and variability (soft guarantees for low response time by pre-generating content) … but do not forget lessons on system design Add components for capacity planning and process monitoring

64. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 64 Puzzle-Specific Considerations Generating Player-Customized Content Puzzle difficulty Solution size Solution alternatives Variation of moves Skill moves Player ability Keep population statistics and generate enough content for most likely cases Match player ability with puzzle difficulty Take into account puzzle freshness 4 21

65. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 65 Continuous Analytics for MMOGs MMOG Data = raw and derivative information from the virtual world (millions of users) Continuous Analytics for MMOGs = Analysis of MMOG data s.t. important events are not lost Data collection Data storage Data analysis Data presentation … at MMOG rate and scale

66. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 66 Continuous Analysis for MMOGs Main Uses By and For Gamers 1.Support player communities 2.Understand play patterns (decide future investments) 3.Prevent and detect cheating or disastrous game exploits (think MMOG economy reset) 4.Broadcasting of gaming events 5.Data for advertisement companies (new revenue stream for MMOGs)

67. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 67 Other Uses for MMOG Data Biology Disease spread models Social Sciences The emergence and performance of ad hoc groups in contemporary society Emergent behavior in complex systems Psychology Games as coping mechanism (minorities) Games as cure (agoraphobia) Economy Contemporary economic behavior

68. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 68 The CAMEO Framework [ROIA09] Continuous MSG Analytics on the Cloud Use own resources for continuous or predicted load Use cloud (on-demand, paid-for, guaranteed) resources for sparse or excess load Users (peers) may also provide service (future) A. Iosup, CAMEO: Continuous Analytics for Massively Multiplayer Online Games on Cloud Resources. ROIA, Euro-Par 2009 Workshops, LNCS 6043, pp. 289--299. Springer, Heidelberg (2010)

69. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 69 CAMEO: Analytics Capabilities 1.Various pieces of information Skill level, experience points, rank 2.Single and Multi-snapshot analysis 3.Analysis functions already implemented Ranking by one or more pieces of information Community statistical properties for a piece of information Identification of Top-K players in single/multi-snapshot Evolution of (Top-)K players Evolution of average community skill Identification of players with special skill combos

70. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 70 CAMEO: Cloud Resource Management Steady AnalyticsDynamic Analytics Burst Snapshot = dataset for a set of players More machines = more snapshots per time unit Periodic Unexpected

71. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 71 CAMEO: Exploiting Cloud Features Machines close(r) to server Traffic dominated by small packets (latency) Elastic IP to avoid traffic bans (legalese: acting on behalf of real people) A. Iosup, A. Lascateu, N. Tapus, CAMEO: Enabling Social Networks for Massively Multiplayer Online Games through Continuous Analytics and Cloud Computing, ACM NetGames 2010.

72. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 72 Sample Game Analytics Results Skill Level Distribution in RuneScape RuneScape: 135M+ open accounts (world record) Dataset: 3M players (largest measurement, to date) 1,817,211 over level 100 Max skill 2,280 Number of mid- and high-level players is significant New Content Generation Challenge High Level Mid Level

73. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 73 Cost of Continuous RuneScape Analytics Put a price on MMOG analytics (here, $425/month, or less than $0.00015/user/month) Trade-off accuracy vs. cost, runtime is constant

74. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 74 Performance Results: Why Choosing the Cloud Matters Location of machines influences MMOG analytics performance (data acquisition)

75. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 75 Cloudification: PaaS for MSGs (Platform Challenge) Build MSG platform that uses (mostly) cloud resources Close to players No upfront costs, no maintenance Compute platforms: multi-cores, GPUs, clusters, all-in-one! Performance guarantees Code for various compute platforms—platform profiling Load prediction miscalculation costs real money What are the services? Vendor lock-in? My data

76. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 76 Mobile Social Gaming and the SuperServer (Platform Challenge) Support MSGs on mobile devices Mobiles everywhere (2bn+ users) Gaming industry for mobiles is new Growing Market SuperServer to generate content for low-capability devices? Battery for 3D/Networked games? Where is my server? (Ad-hoc mobile gaming networks?) Security, cheat-prevention

77. Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup 77 Content, Content, Content (Content Challenge) Produce and distribute content for 1BN people Game Analytics  Game statistic Crowdsourcing Storification Auto-generated game content Adaptive game content Content distribution/ Streaming content