Scalable Performance Distributed Systems: Lecture # 9

Overview How to maintain high-performance that scales millions or billions of entities Case Study: Internet and WWW

Performance Tricks Systems Approach: The KISS principle: Keep it Simple Stupid Layering Identify bottlenecks Hide Latency: Exploit workload properties Improve Throughput: Concurrency Case Study: Internet Guarantees about state: Soft vs. Hard Model of consistency: Absolute vs. Eventual Consistency

Internet Design Really Really Simple End-to-end system design The system does not provide any guarantee No Reliability No Ordering No Congestion Control Only Routing No QoS?

Why? Ordering, Reliability and Congestion Control Typically involve state May evolve over time Must be application-specific Architect Services at the end-systems

Layering in the Internet Large Distributed System IP only provides unreliable, unordered, non- flow-controlled message delivery Layer these services on top when needed Workload characterization Performance for the common case Reliability, Ordering, and Congestion-control when needed

TCP Screw Up Combines Reliability Ordering Congestion Control Multimedia Applications Ordering but no reliability RTP Congestion Control Wireless Networks Multimedia Applications TCP Vegas, Reno, Taheo

Conclusions Keep the core system really simple IP Layer Layer additional Services on top Keep Services at the ends of the system Easy to upgrade Easy to maintain

Internet Routing Architecture All routers must have consistent routes But If a router is out of date, its not the end of the world But system must be available Reliability is provisioned at the ends Question: What’s the consistency model?

Eventual Consistency Just tell your neighbor every so often about your routing state He tells his neighbors and so on Eventually, everyone will be consistent Good enough? Ford-Fulkerson They call it Link-State-Routing Call it gossip

Gossip-based Consistency Talk to a subset of your friends They talk to another subset Will everyone be eventually consistent?

Scalable Performance Let’s assume you are writing a web- server Apache, IS, Zope Principles Identify Bottlenecks Hide Latency: Workload Characterization Throughput: Use Threads

Workload Workload? Same thing over and over again Use processes vs. threads? Modern Webservers do more than fetch and reply Can use stages? Fast path vs. Slow Path

Bottlenecks http --> Get file --> Reply Hard disk access! Interleave requests

Concurrency Fixed vs. Thread Pools Staging of Jobs Queues between stages