1. 1 Comparing Squid Filesystem Performance with Web Polygraph O’Reilly Open Source Convention July 24, 2002 Duane Wessels wessels@squid-cache.org

2. 2 Motivation Squid performance depends greatly on the filesystem. Want to compare different operating systems, filesystems, filesystem options, and Squid storage schemes. Need a good benchmark with minimal input parameters.

3. 3 Squid Filesystem Options OSes: Linux, Free/Net/OpenBSD, Solaris,... Filesystems: UFS, ext2fs, ext3fs, xfs, reiserfs. Filesystem Options: noatime, softdep, async. Squid Storage Schemes: ufs, aufs, diskd. Other parameters we won’t discuss.

4. 4 Web Polygraph Powerful, flexible benchmarking tool for HTTP intermediaries. PolyMix-4: standardized workload for client-side caching proxies. DUT: Device Under Test – Squid in this case.

5. 5 Best Effort Workloads N agents submit requests as fast as the DUT allows. Each agent is always “busy.” Many benchmarks work this way because it is easy to implement. The DUT response time can affect throughput. Easy to make mistakes. Difficult to trust results. Number of agents becomes an arbitrary input parameter. –Difficult to compare different devices

6. 6 A Best-Effort Result

7. 7 Constant Mean Throughput Workloads Agents submit requests at a constant mean rate. Agents spend some time being idle. Results are more believable. Test may fail after running for a long time. Hard to find the ideal, peak throughput. Throughput is an input parameter.

8. 8 Some PolyMix-4 Results

9. 9 Constant Response Time Workloads Offered load varies depending on measured response time. –Otherwise just like constant throughput Tests are less likely to fail halfway through. Removes throughput as an input parameter. Selecting the response time window is tricky. Sometimes observe response time spikes. Requires a constant/predictable hit ratio. –Cannot compare to a no-caching workload

10. 10 The “rptmstat” Workload Response time window: 1.4 – 1.5 seconds. Load delta: 1%. “down” sample rate: 1000 transactions. “up” sample rate: 2000 transactions. Workload goal: fill the cache twice. Adjusts “populous factor” – the number of active agents.

11. 11 Sample rptmstat Console Output 398.25| i-rptmstat 2715719 89.02 1446 60.38 0 822 fyi: rptmstatDn: 1.47sec rptm requires no load adjustment fyi: rptmstatUp: 1.41sec rptm requires no load adjustment fyi: rptmstatDn: 1.67sec rptm changes load by -1.00% fyi: rptmstatDn: 1.40sec rptm requires no load adjustment fyi: rptmstatUp: 1.54sec rptm requires no load adjustment fyi: rptmstatDn: 1.26sec rptm requires no load adjustment fyi: rptmstatDn: 1.51sec rptm changes load by -1.00% fyi: rptmstatUp: 1.39sec rptm changes load by +1.00% fyi: rptmstatDn: 1.52sec rptm changes load by -1.00%

12. 12 Sample rptmstat Result

13. 13 The Squid Filesystem Tests Five different operating systems. Identical hardware. Identical Squid source code. Nearly identical Squid configuration. Different filesystems, options, storage schemes.

14. 14 Hardware for Squid IBM Netfinity 4000R. 500 MHz Pentium 3. 1GB RAM. 3 x 18GB SCSI disk (one external). Integrated Intel 10/100 NIC.

15. 15 Squid Configuration Squid 2.4.STABLE5. 3 x 7500 MB cache_dir (L1=16, L2=256). Logging disabled. Default cache_mem (8MB).

16. 16 The Linux Box Linux 2.4.9-13, with SGI XFS_1.0.2 patches 8192 file descriptors./configure –with-aio-threads=32 Xfsprogs 1.3.13 Reiserfsprogs 3.x.0j

17. 17 Linux Results

18. 18 The NetBSD Box NetBSD 1.5.3_RC1 MAXFILES=8192 NMBCLUSTERS=32768

19. 19 NetBSD Results

20. 20 The OpenBSD Box OpenBSD 3.0 MAXFILES=8192 –Only 4096 per process however –Not an issue – usage is well below this limit NMBCLUSTERS=32768

21. 21 OpenBSD Results

22. 22 The FreeBSD Box FreeBSD 4.5-STABLE MAXFILES=8192 NMBCLUSTERS=32768

23. 23 FreeBSD Results

24. 24 The Solaris Box Solaris 5.8 for Intel (generic 108592-09) /etc/system: –rlim_fd_max=8192 /etc/nsswitch.conf: –hosts: dns files /etc/nscd.conf: –enable-cache hosts no newfs –b 4096 –i 6144 … tunefs –o space …

25. 25 Solaris Results

26. 26 Validating rptmstat Can rptmstat predict PolyMix-4 performance? Start with mean throughput during last ¼ of rptmstat run. Increase or decrease throughput in 10% increments. Always use the same fill rate, however.

27. 27 PolyMix-4 on Linux

28. 28 Modified PolyMix-4 on Linux

29. 29 PolyMix-4 on FreeBSD

30. 30 Modified PolyMix-4 on FreeBSD

31. 31 Conclusions For Squid, the best performing filesystems are: –FreeBSD: softupdates and diskd storage scheme –Linux: ext2fs and aufs storage scheme The rptmstat workload does not accurately predict PolyMix-4 peak throughput. –rptmstat is a more difficult workload because DHR is always 60% rptmstat predicts modified PolyMix-4 performance within 10%.

32. 32 More Information Squid: www.squid-cache.orgwww.squid-cache.org Web Polygraph: www.web-polygraph.orgwww.web-polygraph.org Duane Wessels: wessels@squid-cache.orgwessels@squid-cache.org