Scalable Web Architectures Common Patterns & Approaches Cal Henderson.

Slides:



Advertisements
Similar presentations
Capacity Planning for LAMP Architectures John Allspaw Manager, Operations Flickr.com Web Builder 2.0 Las Vegas.
Advertisements

From Startup to Enterprise A Story of MySQL Evolution Vidur Apparao, CTO Stephen OSullivan, Manager of Data and Grid Technologies April 2009.
Implementing Tableau Server in an Enterprise Environment
User meeting June Monthly and Seasonal forecasts Laura Ferranti and the Seasonal Forecast Section User meeting June 2006.
April eGMS Spring 2006 Workshops. Illinois State Board of Education – April eGMS Basics – Required Hardware PCs (Windows 98,
Flickr A Case Study in Rich Internet Application Development Cal Henderson.
Tableau Software Australia
1 Real World Performance Tuning Ask Bjørn Hansen OSCON 2001.
Communication over MV and LV power lines, Prague October 5 – 6, 20061/70 Budapest University of Technology and Economics Faculty of Electrical Engineering.
1 23 maart 2006 Surface construction techniques for volumetric objects How to maintain convex and concave features? Eddy Loke and Erik Jansen.
April 28, Alberta Energy Information Exchange.
THHGCS07B Coordinate Marketing Activities Lecture 2.
2 Proprietary & Confidential What is Sharding Benefits of Sharding Alternatives of Sharding When to start Sharding Agenda.
 RAID stands for Redundant Array of Independent Disks  A system of arranging multiple disks for redundancy (or performance)  Term first coined in 1987.
Introducing Amazon S3 and EC2 Justin Mason
Adding scalability to legacy PHP web applications Overview Mario A. Valdez-Ramirez.
Brion Vibber Google Tech Talk April 28, 2006 Wikimedia Foundation, Inc. Wikipedia and MediaWiki What are those crazy wiki people up to anyway?
Building Wikipedia Scalable LAMP on a shoestring budget Brion VibberGatorJUG
Web Caching Schemes1 A Survey of Web Caching Schemes for the Internet Jia Wang.
How to build your own computer And why it will save you time and money.
Nikolay Tomitov Technical Trainer SoftAcad.bg.  What are Amazon Web services (AWS) ?  What’s cool when developing with AWS ?  Architecture of AWS 
SM3121 Software Technology Mark Green School of Creative Media.
1 The World Wide Web. 2  Web Fundamentals  Pages are defined by the Hypertext Markup Language (HTML) and contain text, graphics, audio, video and software.
CMU SCS Carnegie Mellon Univ. Dept. of Computer Science /615 - DB Applications C. Faloutsos – A. Pavlo How to Scale a Database System.
CSE 451: Operating Systems Winter 2010 Module 13 Redundant Arrays of Inexpensive Disks (RAID) and OS structure Mark Zbikowski Gary Kimura.
Web Cache. Introduction what is web cache?  Introducing proxy servers at certain points in the network that serve in caching Web documents for faster.
IST346:  Web Services. Today’s Agenda  Learn the basics of how the Web works  Understand various web service architectures  Address scaling, security,
Scalable Web Architectures Common Patterns & Approaches Cal Henderson.
Microsoft Load Balancing and Clustering. Outline Introduction Load balancing Clustering.
File Systems (2). Readings r Silbershatz et al: 11.8.
Platform as a Service (PaaS)
Google AppEngine. Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast.
How WebMD Maintains Operational Flexibility with NoSQL Rajeev Borborah, Sr. Director, Engineering Matt Wilson – Director, Production Engineering – Consumer.
Web server and web browser It’s a take and give policy in between client and server through HTTP(Hyper Text Transport Protocol) Server takes a request.
22-Aug-15 | 1 |1 | Help! I need more servers! What do I do? Scaling a PHP application.
Day 10 Hardware Fault Tolerance RAID. High availability All servers should be on UPSs –2 Types Smart UPS –Serial cable connects from UPS to computer.
Database Storage Considerations Adam Backman White Star Software DB-05:
Networked File System CS Introduction to Operating Systems.
Physical Database Design & Performance. Optimizing for Query Performance For DBs with high retrieval traffic as compared to maintenance traffic, optimizing.
Ideas to Improve SharePoint Usage 4. What are these 4 Ideas? 1. 7 Steps to check SharePoint Health 2. Avoid common Deployment Mistakes 3. Analyze SharePoint.
Data Structures & Algorithms and The Internet: A different way of thinking.
Chapter 9 How Do Users Share Computer Files?. What is a File Server A (central) computer which stores files which can be accessed by network users.
Scalable Web Server on Heterogeneous Cluster CHEN Ge.
Architecture for Caching Responses with Multiple Dynamic Dependencies in Multi-Tier Data- Centers over InfiniBand S. Narravula, P. Balaji, K. Vaidyanathan,
CS 4720 Dynamic Web Applications CS 4720 – Web & Mobile Systems.
Fast Crash Recovery in RAMCloud. Motivation The role of DRAM has been increasing – Facebook used 150TB of DRAM For 200TB of disk storage However, there.
Operating Systems Lecture 14 Segments Adapted from Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard. Zhiqing Liu School of Software Engineering.
Putting Performance Best Practices Together to Create the Perfect SPA Chris Love2Dev.com.
1 MSRBot Web Crawler Dennis Fetterly Microsoft Research Silicon Valley Lab © Microsoft Corporation.
I NTRODUCTION TO N ETWORK A DMINISTRATION. W HAT IS A N ETWORK ? A network is a group of computers connected to each other to share information. Networks.
Lecture 4 Page 1 CS 111 Online Modularity and Virtualization CS 111 On-Line MS Program Operating Systems Peter Reiher.
Cloud Computing Computer Science Innovations, LLC.
Scalable Data Scale #2 site on the Internet (time on site) >200 billion monthly page views Over 1 million developers in 180 countries.
Overview on Web Caching COSC 513 Class Presentation Instructor: Prof. M. Anvari Student name: Wei Wei ID:
Web Cache. What is Cache? Cache is the storing of data temporarily to improve performance. Cache exist in a variety of areas such as your CPU, Hard Disk.
BIG DATA/ Hadoop Interview Questions.
Wikimedia architecture Ryan Lane Wikimedia Foundation Inc.
Cassandra as Memcache Edward Capriolo Media6Degrees.com.
CS Introduction to Operating Systems
Platform as a Service (PaaS)
Platform as a Service (PaaS)
Lecture 18: Scalable Web Services
Informatica PowerCenter Performance Tuning Tips
Distributed Content in the Network: A Backbone View
Leaving the State: Sessionless (Stateless) Authentication in D8
Scaling and Performance
Arrested by the CAP Handling Data in Distributed Systems
Moodle Scalability What is Scalability?
CSE 451: Operating Systems Winter 2009 Module 13 Redundant Arrays of Inexpensive Disks (RAID) and OS structure Mark Zbikowski Gary Kimura 1.
CSE 451: Operating Systems Winter 2012 Redundant Arrays of Inexpensive Disks (RAID) and OS structure Mark Zbikowski Gary Kimura 1.
Presentation transcript:

Scalable Web Architectures Common Patterns & Approaches Cal Henderson

SAM-SIG, 23rd August Hello

SAM-SIG, 23rd August Scalable Web Architectures? What does scalable mean? Whats an architecture?

SAM-SIG, 23rd August Scalability – myths and lies What is scalability?

SAM-SIG, 23rd August Scalability – myths and lies What is scalability not ?

SAM-SIG, 23rd August Scalability – myths and lies What is scalability not ? –Raw Speed / Performance –HA / BCP –Technology X –Protocol Y

SAM-SIG, 23rd August Scalability – myths and lies So what is scalability?

SAM-SIG, 23rd August Scalability – myths and lies So what is scalability? –Traffic growth –Dataset growth –Maintainability

SAM-SIG, 23rd August Scalability Two kinds: –Vertical (get bigger) –Horizontal (get more)

SAM-SIG, 23rd August Big Irons Sunfire E20k $450,000 - $2,500,000 36x 1.8GHz processors PowerEdge SC GHz processor Under $1,500

SAM-SIG, 23rd August Cost vs Cost

SAM-SIG, 23rd August Cost vs Cost But sometimes vertical scaling is right Buying a bigger box is quick (ish) Redesigning software is not Running out of MySQL performance? –Spend months on data federation –Or, Just buy a ton more RAM

SAM-SIG, 23rd August Cost vs Cost But lets talk horizontal –Else this is going to be boring

SAM-SIG, 23rd August Architectures then? The way the bits fit together What grows where The trade offs between good/fast/cheap

SAM-SIG, 23rd August LAMP Were talking about LAMP –Linux –Apache (or LightHTTPd) –MySQL (or Postgres) –PHP (or Perl, Python, Ruby) All open source All well supported All used in large operations

SAM-SIG, 23rd August Simple web apps A Web Application –Or Web Site in Web 1.0 terminology InterwebnetApp serverDatabase

SAM-SIG, 23rd August App servers App servers scale in two ways:

SAM-SIG, 23rd August App servers App servers scale in two ways: –Really well

SAM-SIG, 23rd August App servers App servers scale in two ways: –Really well –Quite badly

SAM-SIG, 23rd August App servers Sessions! –(State) –Local sessions == bad When they move == quite bad –Central sessions == good –No sessions at all == awesome!

SAM-SIG, 23rd August Local sessions Stored on disk –PHP sessions Stored in memory –Shared memory block Bad! –Cant move users –Cant avoid hotspots

SAM-SIG, 23rd August Mobile local sessions Custom built –Store last session location in cookie –If we hit a different server, pull our session information across If your load balancer has sticky sessions, you can still get hotspots –Depends on volume – fewer heavier users hurt more

SAM-SIG, 23rd August Remote centralized sessions Store in a central database –Or an in-memory cache No porting around of session data No need for sticky sessions No hot spots Need to be able to scale the data store –But weve pushed the issue down the stack

SAM-SIG, 23rd August No sessions Stash it all in a cookie! Sign it for safety –$data = $user_id. -. $user_name; –$time = time(); –$sig = sha1($secret. $time. $data); –$cookie = base64($sig-$time-$data); –Timestamp means its simple to expire it

SAM-SIG, 23rd August Super slim sessions If you need more than the cookie (login status, user id, username), then pull their account row from the DB –Or from the account cache None of the drawbacks of sessions Avoids the overhead of a query per page –Great for high-volume pages which need little personalization –Turns out you can stick quite a lot in a cookie too –Pack with base64 and its easy to delimit fields

SAM-SIG, 23rd August App servers The Rasmus way –App server has shared nothing –Responsibility pushed down the stack Ooh, the stack

SAM-SIG, 23rd August Trifle

SAM-SIG, 23rd August Trifle Sponge / Database Jelly / Business Logic Custard / Page Logic Cream / Markup Fruit / Presentation

SAM-SIG, 23rd August Trifle Sponge / Database Jelly / Business Logic Custard / Page Logic Cream / Markup Fruit / Presentation

SAM-SIG, 23rd August App servers

SAM-SIG, 23rd August App servers

SAM-SIG, 23rd August App servers

SAM-SIG, 23rd August Well, that was easy Scaling the web app server part is easy The rest is the trickier part –Database –Serving static content –Storing static content

SAM-SIG, 23rd August The others Other services scale similarly to web apps –That is, horizontally The canonical examples: –Image conversion –Audio transcoding –Video transcoding –Web crawling

SAM-SIG, 23rd August Parallelizable == easy! If we can transcode/crawl in parallel, its easy –But think about queuing –And asynchronous systems –The web aint built for slow things –But still, a simple problem

SAM-SIG, 23rd August Asynchronous systems

SAM-SIG, 23rd August Asynchronous systems

SAM-SIG, 23rd August Helps with peak periods

SAM-SIG, 23rd August Asynchronous systems

SAM-SIG, 23rd August Asynchronous systems

SAM-SIG, 23rd August Asynchronous systems

SAM-SIG, 23rd August The big three Lets talk about the big three then… –Databases –Serving lots of static content –Storing lots of static content

SAM-SIG, 23rd August Databases Unless were doing a lot of file serving, the database is the toughest part to scale If we can, best to avoid the issue altogether and just buy bigger hardware Dual Opteron/Intel64 systems with 16GB of RAM can get you a long way

SAM-SIG, 23rd August More read power Web apps typically have a read/write ratio of somewhere between 80/20 and 90/10 If we can scale read capacity, we can solve a lot of situations MySQL replication!

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication Reads and Writes Reads

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Master-Slave Replication

SAM-SIG, 23rd August Caching Caching avoids needing to scale! –Or makes it cheaper Simple stuff –mod_perl / shared memory – dumb –MySQL query cache - dumbish

SAM-SIG, 23rd August Caching Getting more complicated… –Write-through cache –Write-back cache –Sideline cache

SAM-SIG, 23rd August Write-through cache

SAM-SIG, 23rd August Write-back cache

SAM-SIG, 23rd August Sideline cache

SAM-SIG, 23rd August Sideline cache Easy to implement –Just add app logic Need to manually invalidate cache –Well designed code makes it easy Memcached –From Danga (LiveJournal) –

SAM-SIG, 23rd August But what about HA?

SAM-SIG, 23rd August But what about HA?

SAM-SIG, 23rd August SPOF! The key to HA is avoiding SPOFs –Identify –Eliminate Some stuff is hard to solve –Fix it further up the tree Dual DCs solves Router/Switch SPOF

SAM-SIG, 23rd August Master-Master

SAM-SIG, 23rd August Master-Master Either hot/warm or hot/hot Writes can go to either –But avoid collisions –No auto-inc columns for hot/hot Bad for hot/warm too –Design schema/access to avoid collisions Hashing users to servers

SAM-SIG, 23rd August Rings Master-master is just a small ring –With 2 members Bigger rings are possible –But not a mesh! –Each slave may only have a single master –Unless you build some kind of manual replication

SAM-SIG, 23rd August Rings

SAM-SIG, 23rd August Rings

SAM-SIG, 23rd August Dual trees Master-master is good for HA –But we cant scale out the reads We often need to combine the read scaling with HA We can combine the two

SAM-SIG, 23rd August Dual trees

SAM-SIG, 23rd August Data federation At some point, you need more writes –This is tough –Each cluster of servers has limited write capacity Just add more clusters!

SAM-SIG, 23rd August Data federation Split up large tables, organized by some primary object –Usually users Put all of a users data on one cluster –Or shard, or cell Have one central cluster for lookups

SAM-SIG, 23rd August Data federation

SAM-SIG, 23rd August Data federation Need more capacity? –Just add shards! –Dont assign to shards based on user_id! For resource leveling as time goes on, we want to be able to move objects between shards –Lockable objects

SAM-SIG, 23rd August Data federation Heterogeneous hardware is fine –Just give a larger/smaller proportion of objects depending on hardware Bigger/faster hardware for paying users –A common approach

SAM-SIG, 23rd August Downsides Need to keep stuff in the right place App logic gets more complicated More clusters to manage –Backups, etc More database connections needed per page The dual table issue –Avoid walking the shards!

SAM-SIG, 23rd August Bottom line Data federation is how large applications are scaled

SAM-SIG, 23rd August Bottom line Its hard, but not impossible Good software design makes it easier –Abstraction! Master-master pairs for shards give us HA Master-master trees work for central cluster (many reads, few writes)

SAM-SIG, 23rd August Multiple Datacenters Having multiple datacenters is hard –Not just with MySQL Hot/warm with MySQL slaved setup –But manual Hot/hot with master-master –But dangerous Hot/hot with sync/async manual replication –But tough

SAM-SIG, 23rd August Multiple Datacenters

SAM-SIG, 23rd August Serving lots of files Serving lots of files is not too tough –Just buy lots of machines and load balance! Were IO bound – need more spindles! –But keeping many copies of data in sync is hard –And sometimes we have other per-request overhead (like auth)

SAM-SIG, 23rd August Reverse proxy

SAM-SIG, 23rd August Reverse proxy Serving out of memory is fast! –And our caching proxies can have disks too –Fast or otherwise More spindles is better We stay in sync automatically We can parallelize it! –50 cache servers gives us 50 times the serving rate of the origin server –Assuming the working set is small enough to fit in memory in the cache cluster

SAM-SIG, 23rd August Invalidation Dealing with invalidation is tricky We can prod the cache servers directly to clear stuff out –Scales badly – need to clear asset from every server – doesnt work well for 100 caches

SAM-SIG, 23rd August Invalidation We can change the URLs of modified resources –And let the old ones drop out cache naturally –Or prod them out, for sensitive data Good approach! –Avoids browser cache staleness –Hello akamai (and other CDNs) –Read more:

SAM-SIG, 23rd August Reverse proxy Choices –L7 load balancer & Squid –mod_proxy & mod_cache –Perlbal and Memcache?

SAM-SIG, 23rd August High overhead serving What if you need to authenticate your asset serving –Private photos –Private data –Subscriber-only files Two main approaches

SAM-SIG, 23rd August Perlbal backhanding Perlbal can do redirection magic –Backend server sends header to Perbal –Perlbal goes to pick up the file from elsewhere –Transparent to user

SAM-SIG, 23rd August Perlbal backhanding

SAM-SIG, 23rd August Perlbal backhanding Doesnt keep database around while serving Doesnt keep app server around while serving User doesnt find out how to access asset directly

SAM-SIG, 23rd August Permission URLs But why bother!? If we bake the auth into the URL then it saves the auth step We can do the auth on the web app servers when creating HTML Just need some magic to translate to paths We dont want paths to be guessable

SAM-SIG, 23rd August Permission URLs

SAM-SIG, 23rd August Storing lots of files Storing files is easy! –Get a big disk –Get a bigger disk –Uh oh! Horizontal scaling is the key –Again

SAM-SIG, 23rd August Connecting to storage NFS –Stateful == Sucks –Hard mounts vs Soft mounts SMB / CIFS / Samba –Turn off MSRPC & WINS (NetBOIS NS) –Stateful but degrades gracefully HTTP –Stateless == yay! –Just use Apache

SAM-SIG, 23rd August Multiple volumes Volumes are limited in total size –Except under ZFS & others Sometimes we need multiple volumes for performance reasons –When use RAID with single/dual parity At some point, we need multiple volumes

SAM-SIG, 23rd August Multiple volumes

SAM-SIG, 23rd August Multiple hosts Further down the road, a single host will be too small Total throughput of machine becomes an issue Even physical space can start to matter So we need to be able to use multiple hosts

SAM-SIG, 23rd August Multiple hosts

SAM-SIG, 23rd August HA Storage HA is important for assets too –We can back stuff up –But we want it hot redundant RAID is good –RAID5 is cheap, RAID 10 is fast

SAM-SIG, 23rd August HA Storage But whole machines can fail So we stick assets on multiple machines In this case, we can ignore RAID –In failure case, we serve from alternative source –But need to weigh up the rebuild time and effort against the risk –Store more than 2 copies?

SAM-SIG, 23rd August HA Storage

SAM-SIG, 23rd August Self repairing systems When something fails, repairing can be a pain –RAID rebuilds by itself, but machine replication doesnt The big appliances self heal –NetApp, StorEdge, etc So does MogileFS

SAM-SIG, 23rd August Real world examples Flickr –Because I know it LiveJournal –Because everyone copies it

SAM-SIG, 23rd August Flickr Architecture

SAM-SIG, 23rd August LiveJournal Architecture

SAM-SIG, 23rd August Buy my book!

SAM-SIG, 23rd August The end!

SAM-SIG, 23rd August Awesome! These slides are available online: iamcal.com/talks/