1. Concurrency, Threads, and Events Ken Birman (Based on a slide set prepared by Robbert van Renesse)

2. Summary Paper 1 Using Threads in Interactive Systems: A Case Study (Hauser et al 1993) Analyzes two interactive computing systems Classifies thread usage Finds that programmers are still struggling (pre-Java) Limited scheduling support Priority-inversion

3. Summary Paper 2 SEDA: An Architecture for Well- Conditioned, Scalable Internet Services (Welsh, 2001) Analyzes threads vs event-based systems, finds problems with both Suggests trade-off: stage-driven architecture Evaluated for two applications Easy to program and performs well

4. What is a thread? A traditional “process” is an address space and a thread of control. Now add multiple thread of controls Share address space Individual program counters and stacks Same as multiple processes sharing an address space.

5. Thread Switching To switch from thread T1 to T2: Thread T1 saves its registers (including pc) on its stack Scheduler remembers T1’s stack pointer Scheduler restores T2’ stack pointer T2 restores its registers T2 resumes Two models: preemptive/non-preemptive

6. Thread Scheduler Maintains the stack pointer of each thread Decides what thread to run next E.g., based on priority or resource usage Decides when to pre-empt a running thread E.g., based on a timer May need to deal with multiple CPUs But not usually “fork” creates a new thread Blocking or calling “yield” lets scheduler run

7. Synchronization Primitives Semaphores P(S): block if semaphore is “taken” V(S): release semaphore Monitors: Only one thread active in a module at a time Threads can block waiting for some condition using the WAIT primitive Threads need to signal using NOTIFY or BROADCAST

8. Uses of threads To exploit CPU parallelism Run two CPUs at once in the same program To exploit I/O parallelism Run I/O while computing, or do multiple I/O Listen to the “window” while also running code, e.g. allow commands during an interactive game For program structuring E.g., timers To avoid deadlock in RPC-based applications

9. Hauser’s categorization Defer Work: asynchronous activity Print, e-mail, create new window, etc. Pumps: pipeline components Wait on input queue; send to output queue E.g., slack process: add latency for buffering Sleepers & one-shots Periodic activity & timers

10. Categorization, cont’d Deadlock Avoiders Avoid deadlock through ordered acquisition of locks When needing more locks, roll-back and re-acquire Task Rejuvenation: recovery Start new thread when old one dies, say because of uncaught exception

11. Categorization, cont’d Serializers: event loop for (;;) { get_next_event(); handle_event(); } Concurrency Exploiters Use multiple CPUs Encapsulated Forks Hidden threads used in library packages E.g., menu-button queue

12. Common Problems Priority Inversion High priority thread waits for low priority thread Solution: temporarily push priority up (rejected??) Deadlock X waits for Y, Y waits for X Incorrect Synchronization Forgetting to release a lock Failed “fork” Tuning E.g. timer values in different environment

13. Problems he neglects Implicit need for ordering of events E.g. thread A is supposed to run before thread B does, but something delays A Non-reentrant code Languages lack “monitor” features and users are perhaps surprisingly weak at detecting and protecting concurrently accessed data

14. Criticism of Hauser He assumes superb programmers and seems to believe that “most” programmers won’t use threads (his example systems are really platforms, not applications) Systems old but/and not representative Pre-Java and C# And now there are some tools that can help discover problems

15. What is an Event? An object queued for some module Operations: create_event_queue(handler)  EQ enqueue_event(EQ, event-object) Invokes, eventually, handler(event-object) Handler is not allowed to block Blocking could cause entire system to block But page faults, garbage collection, …

16. Example Event System (Also common in telecommunications industry, where it’s called “workflow programming”)

17. Event Scheduler Decides which event queue to handle next. Based on priority, CPU usage, etc. Never pre-empts event handlers! No need for stack / event handler May need to deal with multiple CPUs

18. Synchronization? Handlers cannot block  no synchronization Handlers should not share memory At least not in parallel All communication through events

19. Uses of Events CPU parallelism Different handlers on different CPUs I/O concurrency Completion of I/O signaled by event Other activities can happen in parallel Program structuring Not so great… But can use multiple programming languages!

20. Hauser’s categorization ?! Defer Work: asynchronous activity Send event to printer, etc Pumps: pipeline components Natural use of events! Sleepers & one-shots Periodic events & timer events

21. Categorization, cont’d Deadlock Avoiders Ordered lock acquisition still works Task Rejuvenation: recovery Watchdog events?

22. Categorization, cont’d Serializers: event loop Natural use of events and handlers! Concurrency Exploiters Use multiple CPUs Encapsulated Events Hidden events used in library packages E.g., menu-button queue

23. Common Problems Priority inversion, deadlock, etc. much the same with events

24. Threaded Server Throughput

25. Event-driven Server Throughput

26. Threads vs. Events Events-based systems use fewer resources Better performance (particularly scalability) Event-based systems harder to program Have to avoid blocking at all cost Block-structured programming doesn’t work How to do exception handling? In both cases, tuning is difficult

27. Both? In practice, many kinds of systems need to support both threads and events Threaded programs in Unix are the common example of these, because window systems use events The programmer uses cthreads or pthreads Major problem: the UNIX kernel interface wasn’t designed with threads in mind!

28. Why does this cause problems? Many system calls block the “process” File read or write, for example And many libraries aren’t reentrant So when the user employs threads The application may block unexpectedly Limited work-around: add “kernel threads” And the user might stumble into a reentrancy bug

29. Events as seen in Unix Window systems use small messages… But the “old” form of events are signals Kernel basically simulates an interrupt into the user’s address space The “event handler” then runs… But can it launch new threads? Some system calls can return EINTR Very limited options to “block” signals in critical sections

30. How people work around this? They try not to do blocking I/O Use asynchronous system calls… or select… or some mixture of the two Or try to turn the whole application into an event-driven one using pools of threads, in the SEDA model (more or less) One dedicated thread per I/O “channel”, to turn signal-style events into events on the event queue for the processing stage

31. This can be hard, but it works Must write the whole program and have a way to review any libraries it uses! One learns, the hard way, that pretty much nothing else works Unix programs built by inexperienced developers are often riddled with concurrency bugs!

32. SEDA Mixture of models of threads and (small message-style) events Events, queues, and “pools of event handling threads”. Pools can be dynamically adjusted as need arises. Similar to Javabeans and EventListeners?

33. SEDA Stage

34. Authors: “Best of both worlds” Ease of programming of threads Or even better Performance of events Or even better

35. Threads Considered Harmful Like goto, transfer to some entry in program In any scope Destroys structure of programs Primitive Synchronization Primitives Too low-level Too coarse-grained Too error-prone Prone to over-specification

36. Example: create file 1. Create file 2. Read current directory (may be cached) 3. Update and write back directory 4. Write file

37. Thread Implementations 1. Serialize: op1; op2; op3; op4 Simplest and most common 2. Use threads Requires at least two semaphores! Results in complicated program 3. Simplified threads a) Create file and read directory in parallel b) Barrier c) Write file and write directory in parallel Over-specification!

38. Event Implementation Create a dummy handler that awaits file creation and directory read events and then send an event to update the directory. Not great…

39. GOP: Discussion Specifies dependencies at a high-level No semaphores, condition variables, etc No explicit threads nor events Can easily be supported by many languages C, Java, etc. Top-down specification cmp with make, prolog, theorem prover Exception handling easily supported

40. Conclusion Threads still problematic As a code structuring mechanism High resource usage Events also problematic Hard to code, but efficient SEDA and GOP address shortcomings But neither can be said to have taken hold

41. Issues not discussed Kernel vs. User-level implementation Shared memory and protection tradeoffs Problems seen in demanding applications that may launch enormous numbers of threads