1. Practical ZFS best practices and first considerations
Jim Salter
Technomancer, Mercenary Sysadmin, Small Business Owner
Today's slides can be found at:
This presentation is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. (C) 2016

2. super techie sysadmin crap ahead.
WARNING:
super techie sysadmin crap ahead.
proceed at own risk

3. So you want to ZFS?
1st up: topology. Most of this isn't really ZFS-specific, but you ignore it at your own peril.

4. IOPS vs throughput
Throughput rarely matters. Throughput is nice big numbers. IOPS is why they're bullshit.

5. Pictured: throughput

6. RAIDZ: throughput vs IOPS

7. Mirrors: read vs write

8. moar vdevs == moar bettar
IOPS tracks vdevs. (vdevs*2, for mirror vdev reads.)

9. wide vdevs == low IOPS IOPS of any vdev tracks its slowest disk.

10. narrow vdevs == faster resilver times
Resilvering tracks IOPS, not throughput.

11. narrow vdevs == faster in-place upgrades
Would you rather resilver two disks for a capacity increase... or sixteen?

12. real life isn't pretty
I use the word tracks for very good reasons. RAIDZ variable stripe writes, SATA/SAS bus saturation, PCIe bus saturation, ARC, ZIL, SLOG, fragmentation, platter topology, TRIM, compression, wear-leveling...

13. now that you've picked a topology...
Let's talk about ashift. 512B sectors => ashift=9 4K sectors => ashift=12 8K sectors => ashift=13 Get it right now, because you can't fix it later.

14. not sure what sector size you have?
ashift value: guess high. 4K blocks on 512B disks won't hurt performance a bit.

15. SLOG : Secondary LOG device
consider a SLOG. turn sync writes into async writes

16. Seriously. You don’t want it. It won’t benefit your workload.

17. let's talk about compression. briefly.
compress=lz4 Period. No, it won't hurt performance. Not even on good SSDs and $50 CPUs.

18. it puts the extended attributes in its inodes...
xattr=sa Linux thing. Store xattrs directly in inodes, not folders. Significant performance bump.

19. Do you really need to know the last time a directory was ls'ed?
atime=off Remember IOPS? Do you really want a new write IO with every read IO?

20. Tune for IOPS, not for throughput.
recordsize=8K Lower recordsize may not be for everyone. Please consult your sysadmin before taking.

21. Take! Freaking! SNAPSHOTS!
du -hs movies
436G movies
time zfs snapshot
real 0m0.150s
time rm -rf movies
real 12m9.870s
time zfs rollback
real 0m0.651s

22. SRSLY THO, TAKE SNAPSHOTS

23. Use Sanoid! root@banshee:~# cat /etc/sanoid/sanoid.conf [data]
use_template=production
recursive=yes
[home/phonebackup]
monthly = 1
[template_production]
hourly = 36
daily = 30
monthly = 3
yearly = 0
autosnap = yes
autoprune = yes

24. Use Sanoid! root@prod0:~# zfs list -rt snap data/files/work
NAME USED AVAIL REFER M
G T -
G T -
M T -
M T -
M T -
M T -
M T -
M T -
M T -
K T -
K T -
M T -
M T -
M T -
M T -
M T -
M T -
M T -
M T -

25. Use Syncoid! That's 2.56TB of data, synced in 2 seconds.
zfs list rust/shares
NAME USED AVAIL REFER MOUNTPOINT
rust/shares T 150G 144K /rust/shares
syncoid rust/shares
Sending incremental ... syncoid_hotspare_ :12:42:37 (~ 41.2 MB):
38.6MB 0:00:02 [ 17MB/s] [====================> ] 93%
That's 2.56TB of data, synced in 2 seconds.

26. Questions? Comments? Angry denunciations?