How to Build a Platform that can Process 60 Terabytes of Data a Day PRESENTED BY Basil Shikin VP of Engineering, AppLovin October, 2015.

Slides:

Advertisements

Similar presentations

How We Manage SaaS Infrastructure Knowledge Track

Advertisements

1 Perspectives from Operating a Large Scale Website Dennis Lee VP Technical Operations, Marchex.

Serverless Network File Systems. Network File Systems Allow sharing among independent file systems in a transparent manner Mounting a remote directory.

The Next I.T. Tsunami Paul A. Strassmann. Copyright © 2005, Paul A. Strassmann - IP4IT - 11/15/05 2 Perspective Months  Weeks.

Included with Windchill PDMLink Free! Easy to setup and install Excellent high-level overview of your system immediately post setup Collaboration with.

19 % System Center FY14 Revenue Growth Large enterprises actively using SC 63% SC customers actively using SCOM 30% SC customers still using.

Is it hard to build a service for 100M user? Short answer – yes.

High memory instances Monthly SLA : Virtual Machines Validated & supported Microsoft workloads Price reduction: standard Windows (22%) & Linux (29%)

Symantec De-Duplication Solutions Complete Protection for your Information Driven Enterprise Richard Hobkirk Sr. Pre-Sales Consultant.

MyCloudIT Removes the Complexity of Moving Cloud Customers’ Entire IT Infrastructures to Microsoft Azure – Including the Desktop MICROSOFT AZURE ISV: MYCLOUDIT.

Azure Services Platform Piotr Zierhoffer. Agenda Cloud? What is Azure? Environment Basic glossary Architecture Element description Deployment.

Citrix Partner Update The Citrix Delivery Centre.

Business Intelligence components Introduction. Microsoft® SQL Server™ 2005 is a complete business intelligence (BI) platform that provides the features,

DISTRIBUTED COMPUTING

© Hitachi Data Systems Corporation All rights reserved. 1 1 Det går pænt stærkt! Tony Franck Senior Solution Manager.

Data Center Infrastructure

The Citrix Delivery Center. 2 © 2008 Citrix Systems, Inc. — All rights reserved Every Day, IT Gets More Complex EMPLOYEES PARTNERS CUSTOMERS.

300 max 100 ms for a round-trip!

® 2007 Solid Information Technology Data Management for Automotive Standardization requirements Franz-Josef Maintz Solid Information Technology

IT Infrastructure Chap 1: Definition

DES Virtualization IPMA Briefing A New Opportunity - DES Legislative mandate to consolidate 5 agencies into DES Consolidate support to DES and support.

Microsoft ® System Center Service Manager 2010 Infrastructure Planning and Design Published: December 2010.

Maximize Return on Engagement via Scalable Omni-Channel Online Services in the Cloud COMPANY PROFILE: XOMNI, INC. Founded in 2011 and headquartered in.

Built on the Powerful Microsoft Azure Platform, Phyzit Helps Doctors Reduce Readmissions Through a Transitional Care Management App MICROSOFT AZURE ISV.

1© Copyright 2012 EMC Corporation. All rights reserved. EMC Mission Critical Infrastructure For Microsoft SQL Server 2012 Accelerated With VFCache EMC.

Built on the Powerful Microsoft Azure Platform, Media Sonar Brings Social Media Investigation to the Public Cloud MICROSOFT AZURE ISV PROFILE: MEDIA SONAR.

 2009 Calpont Corporation 1 Calpont Open Source Columnar Storage Engine for Scalable MySQL Data Warehousing April 22, 2009 MySQL User Conference Santa.

VMob Mobile Marketing Platform Delivers Highly Targeted Marketing Directly into Shoppers’ Existing Smartphone Apps from the Microsoft Azure Cloud MICROSOFT.

== Enovatio Delivers a Scalable Project Management Solution Minus Large Upfront Infrastructure Costs, Thanks to the Powerful Microsoft Azure Platform MICROSOFT.

The Challenge  Traveler information offerings are evolving very rapidly  User expectations, technologies, and roles are all in motion  Agencies need.

1 Information Technology Infrastructure & Management Overview.

Powered by Microsoft Azure, PointMatter Is a Flexible Solution to Move and Share Data between Business Groups and IT MICROSOFT AZURE ISV PROFILE: LOGICMATTER.

+ Logentries Is a Real-Time Log Analytics Service for Aggregating, Analyzing, and Alerting on Log Data from Microsoft Azure Apps and Systems MICROSOFT.

Module 14 Monitoring and Maintaining Windows Server® 2008 Servers.

Mark Gilbert Microsoft Corporation Services Taxonomy Building Block Services Attached Services Finished Services.

Taking Advantage of the Microsoft Azure Platform, Pingvalue Connects People, Products, Stores, and Brands to Deliver Better Experiences for Everyone MICROSOFT.

Zentera Guardia Fabric ™ Securely Connects Client-Server Apps between Microsoft Azure, Enterprise Datacenters & Other Public Clouds MICROSOFT AZURE ISV.

Velocity “Performance Engineering: We the people have spoken” Todd

Saasabi’s Analytical Processing Engine in the Cloud Makes Business Intelligence Affordable for Everyone COMPANY PROFILE: Saasabi Saasabi is a BizSpark.

Understanding IT Infrastructure Lecture 9. 2 Announcements Business Case due Thursday Business Analysis teams have been formed Business Analysis Proposals.

Presented by, MySQL & O’Reilly Media, Inc. MySQL in eBay’s Personalization Platform Chris Kasten eBay Kernel Framework Group April 16, 2008.

 Cloud Computing technology basics Platform Evolution Advantages  Microsoft Windows Azure technology basics Windows Azure – A Lap around the platform.

Basho Technologies | 1 Riak Time Series BEST PRACTICES ON IOT DATA MODELLING Tom Sigler – Solution Basho.

SUSE Linux Enterprise Server for SAP Applications

Platform as a Service (PaaS)

Enterprise Architectures

Platform as a Service (PaaS)

International TOP ups Business presentation

Database Services Katarzyna Dziedziniewicz-Wojcik On behalf of IT-DB.

Prepared by: Assistant prof. Aslamzai

SQL Server Data Collector From Every Angle

Open Source distributed document DB for an enterprise

Large Distributed Systems

Couchbase Server is a NoSQL Database with a SQL-Based Query Language

IT Services Portfolio Todd Endicott – Senior Network and System Engineer Mary Monroe – Implementation Engineer.

In-Memory Performance

LECTURE 34: WEB PROGRAMMING FOR SCALE

Upgrading to Microsoft SQL Server 2014

VMware vRealize® Operations™ Management Pack for Pure Storage

Wavestore Integrates…

LECTURE 32: WEB PROGRAMMING FOR SCALE

LECTURE 33: WEB PROGRAMMING FOR SCALE

Outline Problem DiskRouter Overview Details Real life DiskRouters

Co-designed Virtual Machines for Reliable Computer Systems

LECTURE 33: WEB PROGRAMMING FOR SCALE

In-network computation

Johan Lindberg, inRiver

Setting up PostgreSQL for Production in AWS

Microsoft Azure Services Platform

Windows Azure SDK 1.7 and New Features

Presentation transcript:

How to Build a Platform that can Process 60 Terabytes of Data a Day PRESENTED BY Basil Shikin VP of Engineering, AppLovin October, 2015

How to Build a Platform That Can Process 60 Terabytes of Data a Day2 30 billion ad requests per day 900μs median latency 100TB of data processed per day 9 data centers SYSTEM STATS

3How to Build a Platform That Can Process 60 Terabytes of Data a Day THE EVOLUTION Applovin Stack Evolution Overview

Technology that Delivers Higher Ad Revenues4 Define what matters to the engineering and business Look for major improvements 2 Always be monitoring 31 KEY STEPS How to Navigate the Scaling Journey?

How to Build a Platform That Can Process 60 Terabytes of Data a Day5 KEY PERFORMANCE INDICATORS

6 Revenue / Ads shown Median Request Latency Cost of Infrastructure How to Build a Platform That Can Process 60 Terabytes of Data a Day ENGINEERING KPI

How to Build a Platform That Can Process 60 Terabytes of Data a Day7 300 Servers 200 QPS 350K QPS 25K QPS 170K QPS 900 Servers 500 Servers 300 Servers Infrastructure Reduction

8How to Build a Platform That Can Process 60 Terabytes of Data a Day Why metrics matter? Easy to Measure 1 Both Engineering and Business Sides Understand 2 Tested in Battle 3

How to Build a Platform That Can Process 60 Terabytes of Data a Day9 MAJOR IMPROVEMENTS

Membase Now it is Couchbase Simple Integration with PHP Commonly Known A Quest For The Perfect Cache GROWTH 10 How to Build a Platform That Can Process 60 Terabytes of Data a Day Redis Persists Data Has Basic Transactions Has Basic Replication CCache Home Grown Read Only Distributed Via Torrents mmap'ed Into Memory

How to Build a Platform That Can Process 60 Terabytes of Data a Day11 SQL StoreExternal Cache RAM Cache Ad Server BEFORE AFTER Ad Data Infrastructure

12How to Build a Platform That Can Process 60 Terabytes of Data a Day How we think about components? AVAILABILITY Fault Tolerance Recovery Speed REPLICATION Cross-DC Network PERFORMANCE Memory Connections PLATFORM RISK Stack Familiarity Compatibility

How to Build a Platform That Can Process 60 Terabytes of Data a Day13 CONSTANT MONITORING

How to Build a Platform That Can Process 60 Terabytes of Data a Day14 Metric Visualization

How to Build a Platform That Can Process 60 Terabytes of Data a Day15 BeforeAfter mechAdam

What are our monitoring milestones? MONITORING BASICS HERE’S HOW BRANDS USE OUR PLATFORM Create relevant in-app campaigns HEARTBEAT Availability Hardware Network APPLICATION App-Specific Built-in Third-party VISUALIZATION Find Correlations Grafana 65" Monitors FEEDBACK LOOP Alerts Learning Trends How to Build a Platform That Can Process 60 Terabytes of Data a Day16

17How to Build a Platform That Can Process 60 Terabytes of Data a Day THE EVOLUTION Applovin Stack Evolution Revisted

SELECTED PARTNERS THANK YOU