1. Toolbox for Dimensioning Windows Storage Systems Jalil Boukhobza, Claude Timsit jalil.boukhobza@prism.uvsq.fr 12/09/2006 Versailles Saint Quentin University laboratory

2. PRiSM Lab/ University of Versailles2 12/09/2006 Outline Introduction Overview of the Windows I/O subsystem architecture The developed tools  I/O benchmarking  Storage parameter extraction  I/O simulation Summary

3. PRiSM Lab/ University of Versailles3 12/09/2006 Introduction Windows I/O system is poorly studied CreateFile(): many file access modes different caching algorithms big performance fluctuations for a given workload (ratio 2 to 10). Disk subsystems built independently from OS  Interaction with the OS are not easily predictable What is the performance of a given workload on a given system architecture for a defined I/O strategy on Windows systems and how to optimize it ?

4. PRiSM Lab/ University of Versailles4 12/09/2006 Overview of the Windows I/O system architecture Different file access modes in the CreateFile() function:  Without using the file system cache: no buffer mode (FILE_FLAG_NO_BUFFERING)  Using the file system cache: sequential, normal, write through modes. FILE_FLAG_SEQUENTIAL_SCAN, FILE_ATTRIBUTE_NORMAL, FILE_FLAG_WRITE_THROUGH Storage Device I/O request FastIO Page miss Cache Manager File System Driver Storage Device Driver Virtual Memory Manager

5. PRiSM Lab/ University of Versailles5 12/09/2006 Developed tools 1. I/O benchmarking All Windows file access modes Win32 CreateFile():  Normal, sequential, random, no buffer, write through Request sizes Sequential / Random / interleaved (accesses) Flexible test file selection (zoning) Control on test file fragmentation (file defragmenter and mover) Results:  I/O throughputs  Response times

6. PRiSM Lab/ University of Versailles6 12/09/2006 2.Storage parameter extraction Configuration parameters: partly provided by manufacturers (zoning information, cache segment size and number) Performance parameters: measured to discover the real application performance that may be different from the peak advertised performance (seek times, memory to memory and disk cache to memory throughput, etc.) Disk cache algorithms: this tool helps users to identify those different algorithms (e.g. read ahead & lazy write)

7. PRiSM Lab/ University of Versailles7 12/09/2006 Example Seek times Study the periodicity to find track size Per request response time Cache segment size Read block of size T from disk Re-read that block: if entirely loaded from the disk cache -> segment size ≥T, increment T else decrement T Empty the cache Disk cache updating algorithms Generally simple algorithms (LRU, FIFO, LFU, etc.) that can be tested once the segment size known by issuing different read block sequences and then re-read the blocks to see which one is accessed from the disk (and so has been ejected from the cache).

8. PRiSM Lab/ University of Versailles8 12/09/2006 3. The I/O simulation tool (WinIOSim) Goals:  Application optimization: identifying the best I/O strategy for an application I/O workload on a given architecture.  Hardware optimization: finding the optimal hardware configuration for a given I/O workload. What’s new ?  Implementation of Windows specific cache algorithms depending on the access modes identified by reverse engineering work. Specific sequences of I/O requests issued by the system and application process for each access mode  Disk subsystem reactions to these algorithms Specific reactions for specific sequences (issued by the file system cache depending on the disk

9. PRiSM Lab/ University of Versailles9 12/09/2006 The WinIOSim architecture

10. PRiSM Lab/ University of Versailles10 12/09/2006 The WinIOSim modules I/O generators:  Workload generator Request types, sizes, number, inter arrival times access modes, requested addresses, etc. Different possible distributions for each parameter (Poisson, uniform, exponential, etc.) thanks to OMNET++. Implemented request criticality (synchronous and asynchronous requests).  Trace files extracted using Filemon. Process memory and file system cache modules: simulating the data copy operations, updating policies, etc.

11. PRiSM Lab/ University of Versailles11 12/09/2006 The WinIOSim modules (2) Application and system processes:  Request flow control.  Request grouping and splitting.  File system cache prefetching algorithms, lazy write and write through algorithms depending on access modes.  Both communicate to issue the final request sequence (as seen by the disk). Different buses: simulating bus throughput, delays, sharing.

12. PRiSM Lab/ University of Versailles12 12/09/2006 The WinIOSim modules (3) IO scheduler controls the flow of requests to the disk subsystem  Queuing system: FIFO, SCAN, LOOK. Disk  Mapping, zoning, spare area, number of platters, seek times, rotational speed, head switching times, track and cylinder skew, etc. Disk cache  Segmentation, read ahead algorithms, lazy write and write through algorithms, cache updating policies, etc.

13. PRiSM Lab/ University of Versailles13 12/09/2006 Simulator’s file system cache strategies Windows prefetching algorithms: No buffer mode  Read operations: Sequential mode: loading data sequentially B n, B n+1, B n+2, B n+3, etc. Normal mode (default) System process: One requested block: 3 blocks of 64KB 64KB block loaded by the system process 64KB block loaded by the application process What are the disk cache reactions? Will it load a part of these data ? B1B1 B2B2 B3B3 B4B4 The final sequence of request blocks is: B 1,1, B 1,2, B 1,3, B 3,1, B 2,1, B 3,2,B 2,2, B 3,3, B 2,3, B 4,1, B 4,2,.. 123123123

14. PRiSM Lab/ University of Versailles14 12/09/2006 Simulator’s file system cache strategies (2)  Write operations: Sequential and normal modes : for one request some blocks are flushed on the disk and the others on the file system cache (later on flushed on the disk). Write through mode  Each written block -> file system cache -> disk cache -> disk + modification of a system file on the disk -> acknowledge. 64KB block copied to the disk 64KB block copied to the file system cache and flushed later on to the disk Example with a 320KB request size: File system cache Disk flush Req 4Req 5 Req 6Req 7Req 8 Req 2 Req 1 Req 3 1

15. PRiSM Lab/ University of Versailles15 12/09/2006 Configuration of the simulator Inputs  I/O generator configuration Modeled by the user Real I/O traces  The simulated architecture definition If existing:  Obtained from manufacturers (rarely complete)  Obtained using the WIOTester parameter extraction tool we developed. Outputs  Response times and throughputs (2 main metrics for I/Os)  The different states of all the modules at each stage of the simulation

16. PRiSM Lab/ University of Versailles16 12/09/2006 Validation of the simulator Measures (SQLio & WioTestser) Vs Simulation (WinIOSim) For read operations:  Sequential access with: “no buffer” mode “normal” mode  Random access with: “no buffer” mode “normal” mode For the write operations:  “Normal” mode  “No buffer” mode

17. PRiSM Lab/ University of Versailles17 12/09/2006 Tested architectures

18. PRiSM Lab/ University of Versailles18 12/09/2006 Validation results

19. PRiSM Lab/ University of Versailles19 12/09/2006 Summary Efficient tool for Windows I/O system performance prediction and optimization. Based on the complementarity of measures and simulations. Flexible and dedicated I/O benchmarking tool. I/O parameter extraction tool. Very accurate and flexible simulations of the whole Windows IO system: from application to disk (<10% error). Simulation of the interactions between the modules for example file system cache and disk cache.

20. PRiSM Lab/ University of Versailles20 12/09/2006 Thank you ! Questions ? www.prism.uvsq.fr/~jboukh jalil.boukhobza@prism.uvsq.fr