LECTURE 32: WEB PROGRAMMING FOR SCALE Objectives: Distributed Systems Service Architectures Resources: SOA Event Driven

What is Scale? “Scale” is the most common term in Web Development But what does it mean? Short Answer: Just means to grow in size and amount of traffic or concurrent users Long Answer: Is domain specific!

Metrics Service Level Agreements (SLA’s) are requirement designed by vendors to define performance and availability Speed Availability Compatibility Response Time

Response Time For Web pages Latency (RTT) matters most Where should the server go? For video Available bandwidth matters most Is there one location that is best for everyone? RTT: Round trip time delay

Speed Matters Impact on user experience Users navigating away from pages Video startup delay Impact on revenue Amazon: increased revenue 1% for every 100ms reduction in PLT Shopzilla:12% increase in revenue by reducing PLT from 6 seconds to 1.2 seconds Ping from BOS to LAX: ~100ms

Availability Commonly referred to as Uptime Refers to amount of time the site is available Amazon’s AWS SLA for Uptime Microsoft’s Azure SLA for Uptime *Both SLA’s are comparing Virtual Machine Uptime SLA as of 9/19/2016 Monthly Uptime Percentage Service Credit Percentage Equal to or greater than 98.0% but less than 99.0% 10% Less than 98.0% 25% Monthly Uptime Percentage Service Credit Percentage Greater 99.95% 10% Greater 99.0% 25%

Compatibility This is simply the number of supported devices or brewers

What is a Distributed System? Distributed System: “A number of independent computers linked by a network” (Oxford Dictionary) Simply an application or service that is distributed across multiple independent computers What some examples of distributed systems? Gmail Netflix's Internet Distribution Improves: Availability Reliability & Fault Tolerance Performance Scalability

Evolution of Distributed Systems In the beginning… …there was a single server Probably located in a closet And it probably served blinking text Issues with this model Site reliability Unplugging cable, hardware failure, natural disaster Scalability Flash crowds (aka Slashdotting)

XY Scalable Architecture Scale Up: Vertical Scaling (Y Scaling) Scale Out: Horizontal Scaling (X Scaling) Ebay uses a combination of X,Y,Z style scaling to their database tier

CDN? Content Delivery Network Also sometimes called Content Distribution Network At least half of the world’s bits are delivered by a CDN Probably closer to 80/90% Primary Goals Create replicas of content throughout the Internet Ensure that replicas are always available Directly clients to replicas that will give good performance

Components of CDN Distributed servers Usually located inside of other ISPs (Internet Service Provider) Often located in IXPs (coming up next) High-speed network connecting them Clients (eyeballs) Can be located anywhere in the world They want fast Web performance

Replicated Web Service Use multiple servers Advantages Better scalability Better reliability Disadvantages How do you decide which server to use? How to do synchronize state among servers?

Request Servicing Load Balancing Queues Synchronous Request

Architecture Patterns Monolithic Architecture Event Driven Architecture Micro service Architecture Database per Service Architecture Single Host per Service/Multi Host per Service Most Large Scale sites like (Netflix, Amazon, eBay) have evolved a monolithic architecture to a micro service Netflix is responsible for up to 30% of internet traffic, uses a service-oriented architecture (SOA) Handle over a billion request per day API average out to an average 6 backend services

Monolithic Architecture Everything complied and linked into a single (monolithic) application Usually written in a single programming language When is this the right architecture? Usually when you have a small team with limited experience Short time / Limited SLAs

Micro-Service Architecture Another form of SOA Miro-Service Architecture Small Services A lot of Small Services vs. A Few Big Ones Highly Scalable Easy to Debug Problems with State http://microservices.io/ Amazon.com uses 100-150 services to get all the data for the website

Event Driven Architecture Also known as message driven architecture Also a form of SOA Key Terms: Event Stream: List of events Event Producer: Source of Event Event Consumer: A service that reads the event stream Event Bus: Bundle of Event Streams Popular Systems: Kafka Service Bus EventStore

Companies Using SOA

Appendix