1. 9/29/2016 I/O grind – a new profiler Michael Meeks “Stand at the crossroads and look; ask for the ancient paths, ask where the good way is, and walk in it, and you will find rest for your souls...” - Jeremiah 6:16

2. Perspective is key ?

4. Something in your brain (20 cm) A book on your desk (80 cm away) A book in your room (4 meters away) 1m2m3m4m5m ???

5. room (4 m) brain (20 cm) desk (80 cm) room (4 m) ??? 2m4m6m8m1m3m5m7m10m9m

6. brain (20 cm) desk (80 cm) room (4 m) ??? 5m10m15m20m25m

7. brain (20 cm) desk (80 cm) room (4 m) ??? 10m50m 100m

8. 3200 Km Syria ? Palestine ? Israel ? Iraq ?

9. 1ns ??? 5ns10ns

10. 1ns registers (0.5 nanoseconds) L1 cache (2 nanoseconds) memory (10 nanoseconds) ??? 2 Ghz = 2 instructions / nanosecond 5ns10ns

11. 20ns 30ns 40ns 50ns 60ns registers (0.5 nanoseconds) L1 cache (2 nanoseconds) memory (10 nanoseconds) ???

12. Disk seek 8 000 000ns = 8ms. Syria ? Palestine ? Israel ? Iraq ? Time to walk to the middle east to examine a heap of iron filings (or these days, some clever cobalt alloy)

13. It pays to save seeks

14. 1414 An increasing problem... ● eg. OpenOffice.org really sucks: Amdahl asks Why ? ● OO.o does stupid things – true... cf. an strace. ● Linux: “lets you do stupid things faster ?”

15. Determinism...

16. 1616 Total lack of kernel / determinism ● Repeatable runs ? ● bin daemons, package mgmt, cron etc. ● boot to a raw / minimal X session $ echo 3 > /proc/sys/vm/drop_caches ioctl(BLKFLSBUF) /dev/* $ time [startup-of-large-app] ● +/- 10% jitter in startup time is common. ● Why ? no idea. b.k.o#7372... - 130+ comments etc. ● NB. ~no complex elevation / queuing in single application start. ● Tuning / measuring I/O appears to be a black art ● Just re-compile your kernel with... ● Any new code has to run a gauntlet of uncertainty... ● “Will this break anything ?”

17. 1717 Tweaking the kernel... Not that it needs tweaking of course ! Federico's kernel's I/O subsystem configuration (in action)

18. I/O grind...

19. 1919 I/O grind – a photo of it in action (with optional luggage grinding facility for airport operators – automate today !) Valgrind gedit Trace file I/OGrind GUI Ye Disk FS/Layout trace DiskSim

20. Warning ! iogrind is not ready yet Please do not profile your application with it yet, it will waste your time & make you disillusioned with life itself...

21. 2121 Future work / crazy ideas: data collection – Use system-tap instead of valgrind – Use a cut-down kernel to simulate & bin much of iogrind. Improvements: apps & filesystems / the kernel ● 4Gb plain OpenSUSE Gnome install on sda1: 16Mb of.xml ● compressed (gzip): 2.6Mb => 0.065% - omits permissions, metadata etc. true: but these are perhaps highly compressible. ● Assuming a linear scaling; 32Mb of memory would describe 50Gb of disk. linear scaling – better than linear expected (?) [ use a flash device ? ] ● Why read a directory or inode ever again ? ● Hacks to improve layout at package installation time ● OpenSize / late block allocation ● Serialize INode / Directory cache across mounts... ?

22. 2 Conclusion IOGrind the only deterministic solution – get those 5% wins to add up. useful for applications & filing-system authors lots of future potential for expansion Source http://go-oo.org/~michael/git/valgrind.git/ http://go-oo.org/~michael/git/iogrind-gui.git/ Thanks: AMD, Julian Seward, Federico Mena Quintero, P.Sankar Oh, that my words were recorded, that they were written on a scroll, that they were inscribed with an iron tool on lead, or engraved in rock for ever! I know that my Redeemer lives, and that in the end he will stand upon the earth. And though this body has been destroyed yet in my flesh I will see God, I myself will see him, with my own eyes - I and not another. How my heart yearns within me. - Job 19: 23-27