1. 1 Cooperative Task Management without Manual Stack Management or Event-driven Programming is not the Opposite of Threaded Programming Atul Adya, Jon Howell, Marvin Theimer, William J. Bolosky, John R. Douceur Microsoft Research PRESENTED BY Zig Rosinski

2. 2 Notable References “An Introduction to Programming with Threads”, Andrew D Birrell. (Week 1) “On the Duality of Operating System Structures”, Lauer and Needham (Week 4) “Why threads are a bad idea”, John Ousterhout (Week 4) “SEDA: An Architecture for well-conditioned scalable internet services”, Matt Welsh, David Culler, and Eric Brewer. (Week 4)

3. 3 Debate The talk was given just before lunch and ‘stirred up a lively debate, which continued after the talk and all through the lunch’ http://static.userland.com/sh4/gems/lambda/OlegUSENIX2002.txt

4. 4 Programming for I/O Concurrency Multi-threaded programming – – Difficult to handle race conditions, deadlocks [Ousterhout96] “Event-driven” model – Task explicitly yields control by returning to the scheduler – Task runs serially between yield points – E.g., X Windows event-loop style Task A Task B I/O 1 I/O 2 done I/O 2

5. 5 Unwieldy Code Structure appears in Event-Driven code Can we get benefits of “event-driven” programming without contorting code structure? } F2() { F1() { } I/O I/O done } F() { }

6. 6 Our Contributions “multi- threaded” “event- driven” Task Management Stack Management cooperativepreemptive automatic manual “coroutines” Separate out concerns of task and stack mgmt Separate out concerns of task and stack mgmt Argue for not discarding automatic stack mgmt Argue for not discarding automatic stack mgmt Allow interactions between manual and automatic stack mgmt code styles Allow interactions between manual and automatic stack mgmt code styles

7. 7 Overview Cooperative Task Management Cooperative Task Management Stack Management Stack Management – Automatic Stack Management (ASM) – Manual Stack Management (MSM) Hybrid Code Interaction Hybrid Code Interaction Conclusions Conclusions

8. 8 Executing task has “lock” on shared state Executing task has “lock” on shared state – – Concurrency considered only at I/O yield points – Task must re-validate state after resuming May need to be done even with multi-threading, e.g., mutex released before calling high-latency opn [Birrell89] May need to be done even with multi-threading, e.g., mutex released before calling high-latency opn [Birrell89] Allows I/O concurrency but not CPU concurrency Allows I/O concurrency but not CPU concurrency Cooperative Task Management Task A Task B I/O 1 I/O 1 done I/O 2

9. 9 Issues we’re NOT talking about I/O Management I/O Management – Synchronous vs. asynchronous – Concurrent I/O does not affect shared state Conflict Management Conflict Management – Pessimistic (mutexes/locks) vs. optimistic (abort/retry) – Task mgmt: how often? Conflict mgmt: what to do? Data Partitioning Data Partitioning – Monolithic vs. partitioned – Each partition independently sets task mgmt strategy

10. 10 Stack Management How is the code structured: Automatic Stack Management (ASM) – – Allows natural code structure Manual Stack Management (MSM) – – Forces programmer to abandon basic programming language features – – Mistakenly conflated with cooperative task mgmt

11. 11 Automatic Stack Mgmt (ASM) Implement with user-level non-preemptive thread package, e.g., fibers in Windows Info* GetInfo(ID id) { Info *info = LookupTable(id); return info; } Info* GetInfoBlocking(ID id) { Info *info = LookupTable(id); if (info != NULL) { return info; } SchedDiskReadAndYield(id, info); InsertTable(id, info); return info; } When info is in memory ASM code when info could be on disk

12. 12 SchedDiskReadAndYield(id, info); SchedDiskRead(id, info, cont); void GetInfo1(ID id, Cont *c) {void GetInfo1(ID id) { Manual Stack Mgmt (MSM) Info* GetInfoBlocking(ID id) { if (info != NULL) { Info *info = LookupTable(id); SchedDiskReadAndYield(id, info); InsertTable(id, info); } return info; } void GetInfo2(Frame *f) { ID id = (ID) f  arg1; Info *info = (Info*) f  arg2; (c  func)(c  frame, info); InsertTable(id, info); return info; Frame *f = new Frame(id, info); } (c  func)(c  frame, info); return; Cont *c =(Cont*) f  arg3; Cont *cont = new Cont(&GetInfo2, f); Frame *f = new Frame(id, info, c); Stack frame manually maintained by programmer Task yields by unrolling stack to the scheduler Result returned to caller via a continuation function call

13. 13 MSM: Poor Software Structure ASM Style F() One conceptual function split into multiple functions F2() F1() MSM Style I/O I/O done

14. 14 I/O MSM: Poor Software Structure ASM Style MSM Style I/O F() F1() F2() F3() Loss of control structures, e.g., while loops

15. 15 MSM: Poor Software Structure Loss of local variables: stack frames on the heap ASM Style F() a = 17 Use a F2() F1() MSM Style I/O I/O done a: 17 … I/O I/O done Frame Heap

16. 16 MSM: Poor Software Structure Loss of debugging stack: – Debugger not aware of stack frames on the heap – Some frames optimized way for convenience F2() F1() MSM Style I/O I/O done F1’s cont frame H1() K1() H1’s cont frame K1’s cont frame Heap

17. 17 GetInfo() Yield control Software Evolution GetInfo1() GetInfo2() Schedule I/O done Proc Call Proc Return F2() H2() ASM StyleMSM Style Stack Ripping: Software evolution exacerbates MSM code structure problems Sched I/O I/O done F() H() F1()H1()

18. 18 GetInfo(…) Detecting I/O Yields with MSM GetInfo1(…, Cont * …) GetInfo2() Proc Call Proc Return F2() H2() Signature change guarantees programmer aware of yielding Sched I/O I/O done F() H() F1()H1()

19. 19 Detecting I/O Yields with ASM GetInfo() Yield control Must verify changed semantics but no stack ripping Static check allows same benefits as MSM Schedule I/O I/O done F() H()

20. 20 Overview Cooperative Task Management Stack Management – – Automatic Stack Management – – Manual Stack Management Hybrid Code Interaction Conclusions

21. 21 MSM code calls ASM code Expects to call GetInfo(id, contF2) and unroll stack Extract Info from f Expects to be scheduled when GetInfo done Yield control Process I/O Schedule I/O I/O done Expects to return Info Does not expect continuation F1() F2(Frame *f) GetInfo(Id id) MSM Code ASM Code One stack (fiber) for MSM code One stack (fiber) per task for ASM code

22. 22 MSM code calls ASM code Extract Info from f Yield Process I/O Schedule I/O I/O done Returns Info Start new fiber FiberStart(Id …) GetInfo(Id …) F2(Frame *f) Schedules F2 with Info F1( ) MSMASM M2A-Adapter() (Fiber 1)(MainFiber) One code style unaware of other style F2 scheduled

23. 23 Related Work “Event-driven” to reduce concurrency bugs [Oust96] – – Cooperative task management conflated with MSM “Event-driven” model for performance – – Popular for web servers [Flash, Jaws, StagedServer, Seda] – – Inter-stage: each task reads as in MSM Equivalence of “procedure-oriented” and “message-oriented” systems [Lauer+Needham] – – Different equivalence than ASM and MSM – – Cooperative task management not considered

24. 24 Separate concerns of task and stack management Separate concerns of task and stack management MSM leads to poor code structure MSM leads to poor code structure – Code evolution exacerbates problem ASM: natural code structure ASM: natural code structure MSM and ASM code can co-exist MSM and ASM code can co-exist – For legacy code or different programmer preferences Conclusions Stack Management Task Management Preemptive Cooperative Manual Automatic “Multi- threaded” “Event-driven” “Coroutines”