1. Threads versus Events CSE451 Andrew Whitaker

2. This Class Threads vs. events is an ongoing debate  So, neat-and-tidy answers aren’t necessarily available Our goals:  Understand the debate  Analyze arguments  Form our own opinion

3. A Brief History of a Debate 1995: Why Threads are a Bad Idea (for most purposes)  John Ousterhout, (UC Berkeley, Sun Labs) 2001: SEDA: An Architecture for Well- Conditioned, Scalable Internet Services  Staged, Event-driven Architecture  M. Welsh, D. Culler, and Eric Brewer (UC Berkeley) 2003: Why Events are a Bad Idea (for high- concurrency servers)  R. van Behren, J. Condit, Eric Brewer (UC Berkeley)

4. Background Problem: How do scale up servers to handle many simultaneous requests Server workloads have a lot of I/O, a little CPU Send HTTP request over network Read and parse HTTP request Read file from disk Invoke write syscall Send response over network time HTTP processing

5. Quick Example Q: Suppose it takes 10 seconds for a web server to transfer a file to the client. Of this time, 10 milliseconds is dedicated to CPU processing. How many simultaneous requests do we need to keep the CPU fully utilized? A: 1000

6. Concurrency Strategy #1: Thread- per-Request Run each web request in its own thread  Assuming kernel threads, blocking I/O operations only stall one thread

7. Review: Web Server Worker Thread 1.Read from socket to extract request URI (e.g., index.html) 2.Read desired file into a buffer 3.Write file over the socket 4.Close the socket while (true) { } These are blocking calls

8. Concurrency Strategy #2: Event- driven Execution Use a single thread for all requests Use non-blocking I/O  Replace blocking I/O with calls that return immediately  Program is notified about interesting I/O events This is philosophically similar to hardware interrupts  “Tell me when something interesting happens”

9. Event Loop Pseudocode while (true) { // Ask the OS for Sockets with active I/O; // this blocks if no socket is active Socket sock = getActiveSocket(); if (sock.isReadable()) handleReadEvent(sock); if (sock.isWriteable()) handleWriteEvent(sock); } In a pure event system, this is the only thread

10. Details: UNIX Select System Call Select takes three arrays of file descriptors  Reads, writes, and exceptional events  Note that file descriptors may represent an open file or a socket Select returns the subset of each array that is ready for reading, writing, or has an exceptional condition  Or, blocks if no FDs are active int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

11. Event Handler Pseudocode private Hashtable perSocketState; // this code *never* blocks public void handleRead(Socket sock) { // lookup state about this socket SocketState state = perSocketState.get(sock); state.bytesRead += sock.nonBlockingRead(state.buffer,state.bytesRead); // if we’ve read the entire request, send a response if (state.bytesRead >= REQUEST_LENGTH) { sendResponse(sock); } else {} // do nothing; wait for another event }

12. Can You Think of a System that Works This Way? GUI frameworks (like Swing)  Single event-handling thread  User programs install “event listeners” to receive callbacks Event Loop Event Handlers

13. Why Events? A potentially simpler programming model  No concurrency, locks, conditional variables, etc. Less memory footprint  A single stack (at most, a stack per CPU) More scalable?

14. Cut to Ousterhout

15. Critiquing Ousterhout Events don’t handle true CPU concurrency  We can fix this with an event-loop per processor But, then we have multiple threads… Programming model becomes complex if event handlers do not run to completion  This requires “stack ripping” to maintain state

16. State Transition Diagram for the Web Server Reading request Reading File Writing File In a threaded system, state transitions are maintained by the stack In an event system, we must manually maintain the state of each request  “Stack ripping”

17. Andrew’s Opinion: Code Understanding Event systems are difficult to read / understand  Which events happen in which order? while (true) { // Ask the OS for Sockets with active I/O; // this blocks if no socket is active Socket sock = getActiveSocket(); if (sock.isReadable()) handleReadEvent(sock); if (sock.isWriteable()) handleWriteEvent(sock); }

18. Threads vs Events in Practice Both are used in production systems  Apache is multi-threaded / multi-process  Lighttpd is event-driven Which is better? Why?

19. The Devil is in the (Implementation) Details Many systems did not support kernel threads  Making events more attractive Support for non-blocking I/O is uneven  e.g., UNIX/Linux do not support non-blocking file I/O The UNIX select command has poor scalability (Banga et al, 1998)  But, more recent alternatives to select are much more scalable (Linux epoll) Thread packages often do not scale (Welsh, 2001)  But, optimizations can vastly improve thread scalability Van Behren 2003, Linux 2.6 and NPTL

20. So, What Can We Say For Sure? Event-based systems are harder to program  Requires “stack ripping”  Harder to track control flow  Possible exception: run-to-completion event handlers (e.g., Java Swing GUIs) Event systems use fewer resources  At most a stack per processor Event systems require less synchronization  But, they still require some on multi-processor systems (which is every system!)  Open question: is this is any easier to program?

21. Software Engineering Issues Usually, choice of concurrency model is made for “historical reasons” Unfortunately, changing concurrency models is very difficult Use threads Use events Begin project