1. CIT 470: Advanced Network and System Administration
Filesystems II
CIT 470: Advanced Network and System Administration

2. CIT 470: Advanced Network and System Administration
Topics
Google FS
NFSv3
Automounter
Security
CIT 470: Advanced Network and System Administration

3. CIT 470: Advanced Network and System Administration
GoogleFS Assumptions
High rate of commodity hardware failures.
Small number of huge files (multi-GB +).
Reads: large streaming + small random.
Most modifications are appends.
High bandwidth >> low latency.
Applications / filesystem co-designed.
CIT 470: Advanced Network and System Administration

4. GoogleFS Architecture
CIT 470: Advanced Network and System Administration

5. GoogleFS Server Responsibilities
Master server responsibilities
Metadata: namespace, ACL, chunk mapping.
Chunk lease management, garbage collection, chunk migration.
Chunk server responsibilities
Serve chunks (64MB + checksum) of files.
Chunks replicated on multiple (3) servers.
CIT 470: Advanced Network and System Administration

6. CIT 470: Advanced Network and System Administration
Using NFSv3
Client
Start portmap …
Mount filesystems.
Server
Start portmap
Start NFS services.
Configure exports.
Export filesystems.
CIT 470: Advanced Network and System Administration

7. CIT 470: Advanced Network and System Administration
NFSv3 Services
portmap — RPC service for Linux
portmap
nfs — NFS file server processes.
rpc.mountd
rpc.rquotad
nfsd
nfslock — Optional file locking service.
rpc.statd
CIT 470: Advanced Network and System Administration

8. CIT 470: Advanced Network and System Administration
NFSv3 Processes
rpc.mountd — Handles client mount requests.
rpc.nfsd — NFS server processes.
rpc.lockd — Process for optional nfslock service.
rpc.statd — Handles server crashes for nfslock.
rpc.rquotad — Quotas for remote users.
CIT 470: Advanced Network and System Administration

9. CIT 470: Advanced Network and System Administration
rpcinfo
> rpcinfo -p
program vers proto port
tcp 111 portmapper
udp 111 portmapper
udp nlockmgr
tcp nlockmgr
udp 819 rquotad
udp 819 rquotad
tcp 822 rquotad
tcp 822 rquotad
udp 2049 nfs
udp 2049 nfs
tcp 2049 nfs
tcp 2049 nfs
udp 836 mountd
tcp 839 mountd
udp 836 mountd
tcp 839 mountd
CIT 470: Advanced Network and System Administration

10. CIT 470: Advanced Network and System Administration
NFSv4 Processes
nfsd — NFSv4 server processes. Handles mounts.
rpc.idmapd — Maps NFSv4 names and local UIDs and GIDs. Uses /etc/idmapd.conf.
rpc.svcgssd — Server transport Kerberos auth.
rpc.gssd — Client transport Kerberos auth.
CIT 470: Advanced Network and System Administration

11. NFSv3 Server Configuration
Configure /etc/exports
List filesystems to be exported.
Specify export options (ro, rw, etc.)
Specify hosts/networks to export to.
Export filesystems.
exportfs
Start NFS server (if not already started)
service portmap start
service nfs start
CIT 470: Advanced Network and System Administration

12. CIT 470: Advanced Network and System Administration
/etc/exports
Format: directory hosts(options)
Options
ro, rw Read-only, read-write.
async Server replies before write.
sync Save before reply (default)
all_squash Map all users to anon UID/GID.
root_squash Map root to anon UID (default)
no_root_squash Don’t map root (insecure.)
anon{uid,gid} Set anonymous UID, GID.
Examples:
/home *.example.com(rw,sync)
/backups /24(ro,all_squash)
/ex/limited foo.example.com
CIT 470: Advanced Network and System Administration

13. CIT 470: Advanced Network and System Administration
Client Configuration
Manual mounting
mount -t <nfs-type> -o <options> server:/remote/export /local/directory
Mounting via /etc/fstab
server:/remote/export /local/directory <nfs-type> <options> 0 0
NFS Type is either nfs or nfs4.
CIT 470: Advanced Network and System Administration

14. CIT 470: Advanced Network and System Administration
Mount Options
hard or soft — Error handling
hard: NFS requests will uninterruptible wait until server back.
soft: NFS requests will timeout and report failure.
intr — NFS requests can be interrupted if server unreachable.
nfsvers=2,3— NFS protocol version (not 4)
noexec — Prevents execution of binaries.
nosuid — Disables setuid for security.
rsize,wsize=# — NFS data block size (default 8192)
sec=mode — NFS security type.
sys uses local UIDs and GIDs.
krb5 uses Kerberos5 authentication.
krb5i uses Kerberos5 authentication + integrity checking
krb5p uses Kerberos5 auth + integrity checking + encryption.
tcp, udp — Specifies protocol to use for mount.
CIT 470: Advanced Network and System Administration

15. CIT 470: Advanced Network and System Administration
Automounter
Manages NFS mounts
Automounter maps vs /etc/fstab.
Mounts filesystems only when needed:
Makes administering many filesystems easier.
Improves startup speed.
Provides uniform namespaces.
Ex: mounts /home/home7 as /home on login.
/etc/auto.master points to maps
/home /etc/auto.home
Maps describe mounts
* -fstype=nfs4,soft,intr,nosuid server:/home
CIT 470: Advanced Network and System Administration

16. CIT 470: Advanced Network and System Administration
Security
Limit which hosts have access to filesystems.
Specify hosts in /etc/exports.
Use iptables to limit which hosts can use NFS.
Limit mount options
Default to ro unless writes are necessary.
Disable suid and execution unless needed.
Map root to nobody.
Block NFS at network firewalls.
Block all protocols, not just port 2049.
Use NFSv4 with Kerberos auth + encryption.
CIT 470: Advanced Network and System Administration

17. CIT 470: Advanced Network and System Administration
Performance
Measuring performance
nfsstat
/proc/net/rpc/nfsd
Optimizations
Increase the block size. Problem: fragments?
Set the async option on mounts.
Faster network card.
Faster disk array.
NVRAM cache on array to save NFS writes.
CIT 470: Advanced Network and System Administration

18. CIT 470: Advanced Network and System Administration
References
Michael D. Bauer, Linux Server Security, 2nd edition, O’Reilly, 2005.
Mike Eisler, Ricardo Labiaga, Hal Stern, Managing NFS and NIS, 2nd edition, O’Reilly, 2001.
Aeleen Frisch, Essential System Administration, 3rd edition, O’Reilly, 2002.
Evi Nemeth et al, UNIX System Administration Handbook, 3rd edition, Prentice Hall, 2001.
NFS HOWTO,
RedHat, Red Hat Enterprise Linux 4 System Administration Guide,
RedHat, Red Hat Enterprise Linux 4 Reference Guide,
CIT 470: Advanced Network and System Administration