1. LECTURE 33: WEB PROGRAMMING FOR SCALE
Objectives:
Distributed Systems Service Architectures Resources:
SOA Event Driven

2. What is Scale? “Scale” is the most common term in Web Development
But what does it mean?
Short Answer: Just means to grow in size and amount of traffic or concurrent users
Long Answer: Is domain specific!

3. Metrics
Service Level Agreements (SLA’s) are requirement designed by vendors to define performance and availability
Speed
Availability
Compatibility
Response Time

4. Response Time For Web pages Latency (RTT) matters most
Where should the server go?
For video
Available bandwidth matters most
Is there one location that is best for everyone?
RTT: Round trip time delay

5. Speed Matters Impact on user experience
Users navigating away from pages
Video startup delay
Impact on revenue
Amazon: increased revenue 1% for every 100ms reduction in PLT
Shopzilla:12% increase in revenue by reducing PLT from 6 seconds to 1.2 seconds
Ping from BOS to LAX: ~100ms

6. Availability Commonly referred to as Uptime
Refers to amount of time the site is available
Amazon’s AWS SLA for Uptime
Microsoft’s Azure SLA for Uptime
*Both SLA’s are comparing Virtual Machine Uptime SLA as of 9/19/2016
Monthly Uptime Percentage
Service Credit Percentage
Equal to or greater than 98.0% but less than 99.0%
10%
Less than 98.0%
25%
Monthly Uptime Percentage
Service Credit Percentage
Greater 99.95%
10%
Greater 99.0%
25%

7. Compatibility
This is simply the number of supported devices or brewers

8. What is a Distributed System?
Distributed System: “A number of independent computers linked by a network” (Oxford Dictionary)
Simply an application or service that is distributed across multiple independent computers
What some examples of distributed systems?
Gmail
Netflix's
Internet
Distribution Improves:
Availability
Reliability & Fault Tolerance
Performance
Scalability

9. Evolution of Distributed Systems
In the beginning…
…there was a single server
Probably located in a closet
And it probably served blinking text
Issues with this model
Site reliability
Unplugging cable, hardware failure, natural disaster
Scalability
Flash crowds (aka Slashdotting)

10. XY Scalable Architecture
Scale Up: Vertical Scaling (Y Scaling)
Scale Out: Horizontal Scaling (X Scaling)
Ebay uses a combination of X,Y,Z style scaling to their database tier

11. CDN? Content Delivery Network
Also sometimes called Content Distribution Network
At least half of the world’s bits are delivered by a CDN
Probably closer to 80/90%
Primary Goals
Create replicas of content throughout the Internet
Ensure that replicas are always available
Directly clients to replicas that will give good performance

12. Components of CDN Distributed servers
Usually located inside of other ISPs (Internet Service Provider)
Often located in IXPs (coming up next)
High-speed network connecting them
Clients (eyeballs)
Can be located anywhere in the world
They want fast Web performance

13. Replicated Web Service
Use multiple servers
Advantages
Better scalability
Better reliability
Disadvantages
How do you decide which server to use?
How to do synchronize state among servers?

14. Request Servicing
Load Balancing
Queues
Synchronous Request

15. Architecture Patterns
Monolithic Architecture
Event Driven Architecture
Micro service Architecture
Database per Service Architecture
Single Host per Service/Multi Host per Service
Most Large Scale sites like (Netflix, Amazon, eBay) have evolved a monolithic architecture to a micro service
Netflix is responsible for up to 30% of internet traffic, uses a service-oriented architecture (SOA)
Handle over a billion request per day
API average out to an average 6 backend services

16. Monolithic Architecture
Everything complied and linked into a single (monolithic) application
Usually written in a single programming language
When is this the right architecture?
Usually when you have a small team with limited experience
Short time / Limited SLAs

17. Micro-Service Architecture
Another form of SOA
Miro-Service Architecture
Small Services
A lot of Small Services vs. A Few Big Ones
Highly Scalable
Easy to Debug
Problems with State
Amazon.com uses services to get all the data for the website

18. Event Driven Architecture
Also known as message driven architecture
Also a form of SOA
Key Terms:
Event Stream: List of events
Event Producer: Source of Event
Event Consumer: A service that reads the event stream
Event Bus: Bundle of Event Streams
Popular Systems:
Kafka
Service Bus
EventStore

19. Companies Using SOA

20. Appendix