#devshark welcome to #devshark

#devshark HELLO! I’M Ville Rauma Fingersoft Product Owner Web

#devshark Introduction Today’s talk Fingersoft +50 million customers; The Fingersoft AdSystem Azure Migration Project What is AdSystem? Implementation Problems & Solutions

#devshark Fingersoft Game development / publishing Based in Kempele, near Oulu Finland 14 Employees. Currently looking for C++ / Objective-C programmers 2 in-house games: Hill Climb Racing & I Hate Fish (soft launch early December Check it out!) 6 partner games published: Benji Bananas, Benji Bananas Adventures, Fail Hard, Pick a Pet and Javelin Masters , net revenue 15.5meur

#devshark What is the Fingersoft AdSystem? Serve settings to games Serve ad network priorities Collect real time statistics Game startups, device types, locales, platforms etc. Adjust settings/priorities in real time

#devshark Hill Climb Racing Each game startup does 1-3 requests to our AdSystem

#devshark AdSystem History Version 1 PHP web server originally in a home computer and dsl Very high latency to long distance locations Moved to Amazon micro instance, then moved to m1.medium instance while traffic increased. No data collection or any dynamic functionality. Version 2 Clustered Node.js and Nginx based REST API with MySql database to hold some statistics and settings Could server from 6k to 8k requests/s at best. Still the same latency issues due to single location Could only scale vertically

#devshark Finding Solutions Selection process Researched competing Cloud services such as Amazon and Azure Talked to contacts with knowledge in the area Introduction to Azure via Bizspark membership Proof of concept & Decision AdSystem V3 Horizontal and Vertical Scalability Fast and Scalable database Cache access from all servers Api/Libraries for deployment and management

#devshark Implementation in Azure Game ClientsTraffic Manager Cloud Service Eu West Load Balancer AdServer EuWest 1 AdServer EuWest 2... Cloud Service Eu North Load Balancer AdServer EuNorth 1 AdServer EuNorth 2... Selects cloud service based on latency Redis Cache Statistics Server Admin interface Table Storage Azure Services Scales Horizontally Local cache redis cache Write periodically to DB Linux VMs with Node.JS Admin GUI is node express app Blob Storage Promotion Image data Statistics, settings etc. Blob location per Cloud service loc. Serve images from closest blob

#devshark Results Low latency to most locations in the world due to ability to setup servers in any Azure server location High availability via Availability sets / load balancing Horizontal/Vertical scaling capability Cuts costs since we need only keep up the type and amount of VM instances that we need at any given time. Can serve all clients. Ease of deployment Microsoft provides npm libraries to control azure services. This means that we write deployment scripts in the same language(js) that we use to script server logic. Basically we have 1 click solution to deploy VMs and software into Azure cloud.

#devshark A Few Problems / Solutions Problem 1.Windows server based Node.js was first attempted due to easy deployment scenarios. It was not feasible because of high cpu usage per served client in comparison to Linux. 2.Initial implementation used direct azure table storage reads/writes for each server VM. This didn’t work due to high number of failed operations which then required complicated reattempt and backoff strategies. These also increased memory and processor usage. 3.How to realistically test massive connection amounts to reflect production servers? Solution 1. Moved to use Linux based VMs instead. This allowed us to easily customize linux configuration to be able to handle massive concurrent connection counts. 2. Data is now first passed to Redis cache. Then a separate Statistics server is used to read cache periodically and write data to DB. This removed all data issues from AdServers serving the clients. All AdServer resources now go towards serving clients. 3. We used Visual Studio 2013 Ultimate edition which can do cloud based load tests via visualstudio.com to simulate a real world scenario. Highly useful!

#devshark Contact us! If you’re looking for a job: If you’re looking to get your mobile game published:

#devshark PEW PEW! (oh and thank you)