1. SEDA: An Architecture for Scalable, Well-Conditioned Internet Services
Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, Berkeley
Presented By:
Linh Nguyen

2. Agenda Motivations and background SEDA Evaluations Concurrencies Goals
Stages
Queues
Dynamic Resource Schedulers
Evaluations
Efficient methods to solve problems using factored representation
Goal: each variable has a value that satisfies all constraints on variable

3. Motivations High-volume Internet Services The Slashdot effect
Yahoo: 1.2 billions page views per day
AOL: 10 billions hits per day
Now: Google at 3.5 billion searches per day and increasing
The Slashdot effect
Peak loads can be 100-fold larger than normal
In 2009, Google mistook the surge in Michael Jackson-related searches for an attack
Increasingly dynamic
Dynamic content: Google Maps, Social Networks
Service logic changes frequently: Providers want to add functionalities, improvements, etc
Services are hosted on general purpose facilities

4. Thread-based concurrency
EECS 492 Fall 2004
28 Sep 04
Thread-based concurrency
Each request consumes a thread to process it
Synchronization to protect shared resources
OS switches between threads
Can be achieved in two ways:
Thread-per-task model
Bounded thread pools
M. Wellman

5. Characteristics of thread-based concurrency
Easy to program, since all resources are virtualized. However, applications are not able to manage resources
Associated overheads:
Cache misses
Scheduling overhead
Lock contention
When number of threads is large, throughput is degraded
Bounded thread pools lead to:
Unfairness to clients
Large waiting times

6. Event-based concurrency
EECS 492 Fall 2004
28 Sep 04
Event-based concurrency
Main thread processes incoming events and drives the executions of FSMs
Each FSM represents a single request or flow of execution
Scheduler controls the execution of each FSM
M. Wellman

7. Characteristics of event-driven concurrency
Little degradation in throughput
Assume non-blocking I/O, which may not be supported
Event scheduling is tied to application logic (when to process incoming event, in what order to process the FSMs) -> harder to modularize

8. Solutions? Support massive concurrency
Simplify building well-conditioned services
Enable introspection
Support self-tuning resource management
Staged event-driven architecture (SEDA)

9. SEDA
Draws from both thread-based (ease of programming) and event-based (extensive concurrency) models
Each service is a network of stages
Stages connected by event queues

10. Stage Small number of threads per stage
Dynamic controllers automatically tune the number of threads allocated based on demand
Event handler provides core logic, processes a batch of multiple events, and implements resource-management policies

11. Queue (thanks Norman Hutchinson of UBC)
Queues are finite
An enqueue behavior may fail
Block on full queue -> backpressure
Drop rejected events -> load shedding
Queue introduces explicit execution boundary
Threads may only execute within a single stage
Performance isolation, modularity, independent load management
Explicit event delivery support inspection
Trace flow of events through application
Monitor queue lengths to detect bottleneck

12. Dynamic Resource Controllers
Batching controller
Adjusts batching factor to stabilize throughput
Large factor: higher throughput
Small factor: Lower response time
Thread pool controller
- Determines the degree of concurrency
Add thread if queue length > threshold
Remove idle threads

13. Evaluations
“closed-loop” server: Haboob, comparisons with Apache and Flash
“open-loop” server: Gnutella

14. Haboob performance analysis
Jain Fairness Measurement
𝑓(𝑥): fairness in term of clients and their associated requests
𝑁 : number of clients
𝑋 𝑖 : number of requests for each of N clients
𝑓(𝑥) = 1: server is equally fair to all clients

15. Haboob performance analysis (cont.)

16. Observations
Haboob achieves higher throughput than Apache and Flash as the number of clients reach 64
Apache and Flash exhibit high frequencies of low response times
Long tail in Apache and Flash
Haboob (SEDA) yields predictable performance, while Apache is very unfair

17. Gnutella Non-traditional Internet services
An user searches for and downloads files from other users
Five types of messages: ping, pong, query, queryhits, push.

18. Gnutella Performance
Higher packet latencies for 100 and 200 packets/sec since no threads were added
For 400 and 1000 packets/sec two threads were added

19. Summaries Achieved its goals:
Support massive concurrency: Combines event-driven and thread-based
Enable introspection to handle peak load: Event queue allows for inspection of request streams
Dynamic resource management: Feedback-control with resource controllers
Simplify task of building well-conditioned services: Abstract scheduling and resource management from programmers; easier modularization.

20. Discussions Decisions on how to break down an application into stages
Scalability?
A retrospective on SEDA
Ousterhoust’s why threads are a bad idea?
Brewer’s why events are a bad idea?