1. IO-Lite: A Unified Buffering and Caching System By Pai, Druschel, and Zwaenepoel (1999) Presented by Justin Kliger for CS780: Advanced Techniques in Caching Professor Zhang (Summer 2005)

2. Outline Problem & Significance Problem & Significance Literature Review Literature Review Proposed Solution Proposed Solution Design, Implementation, & Operation Design, Implementation, & Operation Experimental Design Experimental Design Results Results Conclusion Conclusion Further Research Further Research

3. The Problem The I/O subsystem and various applications all tend to use their own private I/O buffers The I/O subsystem and various applications all tend to use their own private I/O buffers Redundant data copying Redundant data copying Multiple buffering Multiple buffering Lack of cross-subsystem optimization Lack of cross-subsystem optimization

4. Problem’s Significance Wastes memory Wastes memory Reduces space Reduces space available for caching Causes higher cache Causes higher cache miss rates High CPU overhead Limits server throughput

5. Literature Review 1.POSIX I/O -Problem:double-buffering 2.Memory-mapped files (mmap) - Problem: - Problem: Not generalized to network I/O

6. Literature Review 3.Transparent Copy Avoidance -Problem: VM page alignment problems Copy-on-write faults Genie (emulated copy) Genie (emulated copy) Lack of full transparency leads to same problems 4.Copy Avoidance with Handoff Semantics -Problem: Lack of concurrent sharing reduces effectiveness

7. Literature Review 5.Fast buffers (fbufs) Designed by Druschel -Problem: Does not support filesystem access, or a file cache or a file cache 6.Extensible kernels -Problem: More overhead, not OS-portable

8. IO-Lite Solution Unified buffering and caching Unified buffering and caching Allow all applications and subsystems share the same buffered I/O data Allow all applications and subsystems share the same buffered I/O data Very simple at face value, Very simple at face value, very complex to implement

9. Basic Design Immutable buffers Immutable buffers Initial allocated data cannot be modified Initial allocated data cannot be modified Effectively read-only sharing Effectively read-only sharingAdvantages? Eliminates synchronization and protection problems Eliminates synchronization and protection problemsDisadvantages? I/O data cannot be modified in place I/O data cannot be modified in place

10. Further Design Considerations To make up for immutable buffers: Create buffer aggregate abstraction (an ADT) Create buffer aggregate abstraction (an ADT)mutable Reference to IO-Lite Window in VM Aggregates contain ordered list of form Aggregates contain ordered list of form Aggregates passed by value Aggregates passed by value Buffers passed by reference Buffers passed by reference

11. Further Design Considerations Buffer sharing must be concurrent Buffer sharing must be concurrent To achieve this, use similar method to fbufs To achieve this, use similar method to fbufs Expand to include the filesystem Expand to include the filesystem Adapts for general purpose OS Adapts for general purpose OS Worst case scenario (in terms of overhead): Worst case scenario (in terms of overhead): Page remapping Page remapping (when last buffer is allocated before first is deallocated) (when last buffer is allocated before first is deallocated)

12. IO-Lite Implementation New read & write API which supersedes the regular read & write New read & write API which supersedes the regular read & write size_t IOL_read(int fd, IOL_Agg **aggr, size_t size); size_t IOL_read(int fd, IOL_Agg **aggr, size_t size); size_t IOL_write(int fd, IOL_Agg *aggr); size_t IOL_write(int fd, IOL_Agg *aggr); IOL_Agg is buffer aggregate data type IOL_Agg is buffer aggregate data type Both operations are atomic Both operations are atomic

13. IO-Lite Implementation Applications: Recommends implementation in runtime I/O Libraries to avoid modifying all programs Recommends implementation in runtime I/O Libraries to avoid modifying all programsFilesystem: File cache data structure: File cache data structure: Network: Need to modify network device drivers to allow early demultiplexing (using a packet filter) Need to modify network device drivers to allow early demultiplexing (using a packet filter)

14. IO-Lite Operation With regards to the cache: With regards to the cache: Cache replacement basically LRU Cache replacement basically LRU Allows for application customization Allows for application customization Cache eviction controlled by VM daemon Cache eviction controlled by VM daemon Do >½ replaced pages contain I/O data? Do >½ replaced pages contain I/O data?

15. IO-Lite Operation Impact of immutable buffers: Impact of immutable buffers: Case 1: Entire object is modified Case 1: Entire object is modified Lack of in-place modification has no ill effect Lack of in-place modification has no ill effect Case 2: Subset of object needs to be modified Case 2: Subset of object needs to be modified Rather than recopy entire object, use chaining Rather than recopy entire object, use chaining Performance loss is small if blocks are localized Performance loss is small if blocks are localized Case 3: Scattered subset needs modification Case 3: Scattered subset needs modification IO-Lite incorporates mmap interface for this IO-Lite incorporates mmap interface for this

16. Experimental Design Compared: Apache 1.3.1 Apache 1.3.1 Widely used web server Flash (event-driven HTTP server) Flash (event-driven HTTP server) Designed by authors in previous year Flash-Lite (Flash modified to use IO-Lite API) Flash-Lite (Flash modified to use IO-Lite API) New design by authors

17. Experimental Design 1.General: varied requested file size 40 requests for same file 40 requests for same file File size ranged from 500 bytes – 200 Kbytes File size ranged from 500 bytes – 200 Kbytes 2.Persistent connections Reduces overhead Reduces overhead 3.CGI Additional I/O traditionally slows servers Additional I/O traditionally slows servers

18. Experimental Design 4.Real workloads Shows performance benefits by allowing more space for caching Shows performance benefits by allowing more space for caching Based on Rice’s CSCI department logs Based on Rice’s CSCI department logs 5.Wide Area Network (WAN) Test throughput with 0-256 slow clients connecting Test throughput with 0-256 slow clients connecting 6.Applications Incorporated API into UNIX programs Incorporated API into UNIX programs

19. Results 1.General test: Bandwidth increase of 43% over Flash, 137% over Apache Bandwidth increase of 43% over Flash, 137% over Apache No real difference for files less than 5KBytes No real difference for files less than 5KBytes

20. Results 2.Persistent Connections Flash-Lite even more effective at smaller file sizes Flash-Lite even more effective at smaller file sizes 3.CGI All servers slow, but Flash-Lite still much better All servers slow, but Flash-Lite still much better 4.Real workload Flash-Lite throughput 65% greater than Apache Flash-Lite throughput 65% greater than Apache 5.WAN Flash-Lite does not suffer from slow clients Flash-Lite does not suffer from slow clients 6.Applications Varied improvement for all programs tested Varied improvement for all programs tested

21. Conclusion IO-Lite consistently improved performance in all contexts tested IO-Lite consistently improved performance in all contexts tested Requires modification to numerous libraries and network device drivers EG: see Peng, Sharma, & Chiueh (2003) EG: see Peng, Sharma, & Chiueh (2003)

22. Further Research There have been 42 citations Almost all fell between 2001-2003 Almost all fell between 2001-2003 Authors have not written any follow-ups Authors have not written any follow-ups Lack of papers that involve implementation of IO-Lite or a variation of it Probably because of complexity and number of modifications that are necessary Probably because of complexity and number of modifications that are necessary

23. Appendix: Figures 2) 4) 5) 3) 6)

24. Questions?