1. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services
Matt Welsh, David Culler, and Eric Brewer
Computer Science Division
University of California, Berkeley

2. Scalable, Fair Services
Internet servers handle extremely high loads
Yahoo! receives 1.2 billion requests daily
AOL web cache services 10 billion hits daily
Desire efficiency, consistency, and fairness
Scalability through replication
Load may be extremely variable
“Slashdot effect” == 100-fold increase
Replication not enough

3. Goal
General framework for highly concurrent and well-conditioned services, that
handles wide variations of load gracefully
factors out details of concurrency and resource management
Framework should be useful in many contexts
Servers, routers, etc.

4. SEDA Staged, Event-Driven Architecture
Building block is the Stage
Handler processes queued, incoming events
Outputs placed on next stages’ queues
Multiple threads may service a stage
Dynamic Resource Controllers
Observe operating conditions
Respond by altering resource usage
Useful for responding to high load

5. Background Well-conditioned service: Two common models:
Behaves like a simple pipeline
load > capacity 
max throughput + linear drop in response time
Two common models:
Thread-based concurrency
Event-driven server

6. Thread-based Concurrency

7. Thread-based Concurrency
Benefits
Simple, well-supported programming model
Drawbacks
High overheads as # of threads increases
Cache & TLB misses, scheduling, lock contention
Virtualizes resources for multiprogramming

8. Threaded Server Throughput Degradation

9. Event-driven Server

10. Event-driven Server Benefits Drawbacks
Robust to high loads
Drawbacks
Harder to program---concerned with scheduling and event partitioning
More direct control of resources

11. Event-driven Server Throughput

12. SEDA Key mechanisms are event-driven stages, and resource controllers
Support massive concurrency
Simplify construction of well-conditioned services
Enable introspection
Support self-tuning resource management
Key mechanisms are event-driven
stages, and resource controllers

13. A SEDA Stage

14. SEDA HTTP Server: Haboob

15. SEDA Resource Controllers

16. SEDA Thread Pool Controller

17. SEDA Batching Controller

18. Sandstorm Implementation of SEDA in Java
Benchmark performance is good (IBM JDK 1.3.0)
Advanced features (e.g. GC) helpful
Many GP features (queue and thread management, profiling, etc.)
Relies on async I/O package
Better performance than blocking ops + threads
Roughly 20k LOC

19. Async Sockets Layer

20. Async File Layer Bounded thread pool to perform standard I/O ops
read, write, seek, stat, close
One thread per file (ensures serial access)
Number of threads adjusted by resource controller

21. SEDA HTTP Server: Haboob

22. Haboob Performance Compare with Apache and Flash
Measure throughput and response time
Calculate Fairness:
f(x1, x2,…, xN) = (xi)2 / N xi2
where xi = # of requests completed for each of N clients over some period of time
Measure behavior under high load

23. Configuration 4-way SMP 500 MHz Pentium III
2 GB RAM, Linux , Gigabit Ethernet
One server and 32 load generators
Flash & Haboob run with 200 MB page cache
Offered load derived from SPECweb99
3.31 GB file set size

24. Haboob Throughput

25. Haboob Response Times 1024 clients

26. Haboob Performance

27. Response Time Controller 1024 clients

28. Gnutella Packet Router
3 stage architecture
GnutellaServer accepts TCP connections
GnutellaRouter performs routing
GnutellaCatcher manages connections to network
Max throughput 20k p/s
Real-network run of 37 hours:
Processed 24.8m packets (avg. 179 p/s)
Received 72k connections (avg. 12 simultaneous)

29. Performance Challenges
Protection from slow sockets
Need to threshold queue size of slow connections
Close connection on exceeding threshold
Load conditioning
Introduced bottleneck in query packets (15% of traffic mix)
Add threads to GnutellaRouter as offered load increases (queue size > 1000 packets)

30. Discussion Measurement and control vs. reservation
Mechanisms for detecting overload
Policies to deal with overload
SEDA ease of programming
Less worry of synchronization & race conditions than thread-per-request
Less complex and “soft” than events
Directions for OS design?

31. Contributions SEDA: Staged, Event-Driven Architecture
Applications consist of connected stages each serviced by one or more threads
Dynamic Resource Controllers examine and react to high load conditions and control thread usage
SEDA implementation and applications
Sandstorm implementation written in Java
Haboob webserver
Gnutella packet router