1. Operating System Support for Application-Specific Speculation Benjamin Wester Peter Chen and Jason Flinn University of Michigan

2. Speculative Execution  Sequential dependent tasks  Predict results of Task A to break dependence  Execute Task B in parallel  Isolate all effects  Correct prediction: commit  Wrong prediction: abort A B B TIME 2EuroSys 2011 Predict A

3. Speculation Everywhere!  Discrete event simulation  I/O prefetching  Distributed shared memory  Distributed file systems  Deadlock detection  Remote displays  Web page pre-rendering EuroSys 20113

4. Speculation as a Service to Apps How is this system designed? In what ways can it be customized for an app? How can those customizations be specified? 4EuroSys 2011

5. Outline  Introduction  Designing Speculation as a Service  Implementation  Evaluation  Conclusion EuroSys 20115

6. Design 1: In-App Speculation +Complete semantic info +Predict arbitrary app operations +Safe operations allowed ‒No reuse: significant development needed ‒Scope is limited: unsafe operations block EuroSys 20116 App 1 Data App 2 Data

7. OS Design 2: Generic OS Speculation +Apps need no modifications +Wide scope: unsafe operations taint ‒Lacks semantic understanding of app ‒Predict system calls only ‒Handle application conservatively EuroSys 20117 App 2 App 1

8. Separate Mechanism and Policy Mechanism implements isolation Policy describes customizations Best of both extremes  Mechanism built in OS o Common implementation o Wide scope  Policy specified in Applications o Expose semantic information EuroSys 20118

9. OS Design 3: Expose Predictions +Predict arbitrary app operations +Reuse OS mechanism (with app assistance) +Wide scope for taint propagation ‒Limited semantic info ‒Speculative external output never allowed ‒Commit on identical results EuroSys 20119 App 2 App 1

10. OS Design 4: Expose Safety +Predict arbitrary app operations +Reuse OS mechanism (with app assistance) +Wide scope for taint propagation +More semantic info +Allow safe output +Commit on equivalent results EuroSys 201110 App 2 App 1

11. Customizable Policy  Creation o What tasks are predictable o How to predict them  Output o What output is safe to allow  Commit o Which results are acceptable to commit EuroSys 201111

12. Outline  Introduction  Designing Speculation as an OS Service  Implementation  Evaluation  Conclusion EuroSys 201112

13. Implementation  Mechanism built in OS o Based on Speculator kernel o Checkpoints & logs processes, files, IPC, etc.  Policies expressed using system call API EuroSys 201113

14. Speculative Process Control Process spec_fork() EuroSys 201114 TIME spec_fork predict A B commit A abort B C

15. API Example int main() { int x; int prediction = get_prediction(); if (spec_fork() == SPECULATIVE) { x = prediction; } else { x = slow_function(); if (equiv(x, prediction)) commit(); else abort(); } set_output_policy(stdout, ALLOW); printf(“%d”, x); } EuroSys 201115 Output Policy Creation Policy Commit Policy

16. Outline  Introduction  Designing Speculation as an OS Service  Implementation  Evaluation  Conclusion EuroSys 201116

17. Evaluation Can apps effectively use API to increase parallelism? Case studies 1.Predictive application launching in Bash 2.SSL certificate checks in Firefox 3.Replicated service in PBFT-CS EuroSys 201117

18. Check command App 1: Predictive Launching in Bash EuroSys 201118 TIME Run program …finished bwester $ ▌ …finished bwester $ ▌ …finished bwester $ grep foo –r. ▌ …finished bwester $ grep foo –r. ▌ grep foo –r. Creation Policy: Use machine learning to predict user input Creation Policy: Use machine learning to predict user input Commit Policy: Normalize user input before comparison Commit Policy: Normalize user input before comparison Output Policy: Safe X11 Messages: Allow Output Policy: Safe X11 Messages: Allow

19. How Much Work Can Be Hidden? EuroSys 201119 TIME Non-Speculative 45 Speculative Prompt Get cmd Spec. Execute cmd Execute Prompt Execute cmd Get cmd

20. How Much Work Can Be Hidden? EuroSys 201120 TIME SpeculativeNon-Speculative 45 Up to 86% hidden Prompt Get cmd Spec. Execute cmd Execute Prompt Execute cmd Get cmd

21. Done Request page App 2: Firefox SSL Connections EuroSys 201121 TIME Web Server Validation Server Open https:// Check cert. ValidateGet session key GET /index.html?id=0123 Creation Policy: Predict certificate is valid Creation Policy: Predict certificate is valid Output Policy: Alow SSL connection Output Policy: Alow SSL connection Output Policy: Block private data Output Policy: Block private data

22. Connection Latency Hidden? EuroSys 201122 TIME Non-SpeculativeSpeculative https:// Check cert. Session key Done Request https:// Check cert. Session key Done RequestValidate

23. Connection Latency Hidden? EuroSys 201123 TIME Non-SpeculativeSpeculative Avg 60ms hidden https:// Check cert. Session key Done Request https:// Check cert. Session key Done RequestValidate

24. App 3: PBFT-CS Protocol EuroSys 201124 TIME Send Request Distributed Replicated Service (Reply, sig1) Check reply Request (Reply, sig2) Work… Creation Policy: Agree on 1 st reply Creation Policy: Agree on 1 st reply Output Policy: PBFT-CS Messages: Allow Output Policy: PBFT-CS Messages: Allow

25. Improved Client Throughput? EuroSys 201125 TIME Non-SpeculativeSpeculative Request1 1 st Reply Verify Request2 1 st Reply Request1 1 st Reply Verify Request2 1 st Reply Null requests

26. Improved Client Throughput? EuroSys 201126 TIME Non-SpeculativeSpeculative 1.9x Throughput Request1 1 st Reply Verify Request2 1 st Reply Request1 1 st Reply Verify Request2 1 st Reply Null requests

27. Cost of Generic Mechanism EuroSys 201127 App-specific Mech Shared OS Mech Non-speculative Null requests

28. Cost of Generic Mechanism EuroSys 201128 App-specific: 8% faster App-specific Mech Shared OS Mech Non-speculative Null requests

29. Conclusion  Mechanism o Common: checkpoints, output buffering, taint propagation o Implemented in OS  Policy o App-specific: Controls creation, output, and commit o Implemented in applications  Demonstrated with 3 case studies o Improved parallelism o Small overhead relative to app-specific mechanism Questions? EuroSys 201129