CIT 470: Advanced Network and System Administration Filesystems II CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Topics Google FS NFSv3 Automounter Security CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration GoogleFS Assumptions High rate of commodity hardware failures. Small number of huge files (multi-GB +). Reads: large streaming + small random. Most modifications are appends. High bandwidth >> low latency. Applications / filesystem co-designed. CIT 470: Advanced Network and System Administration

GoogleFS Architecture CIT 470: Advanced Network and System Administration

GoogleFS Server Responsibilities Master server responsibilities Metadata: namespace, ACL, chunk mapping. Chunk lease management, garbage collection, chunk migration. Chunk server responsibilities Serve chunks (64MB + checksum) of files. Chunks replicated on multiple (3) servers. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Using NFSv3 Client Start portmap … Mount filesystems. Server Start portmap Start NFS services. Configure exports. Export filesystems. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration NFSv3 Services portmap — RPC service for Linux portmap nfs — NFS file server processes. rpc.mountd rpc.rquotad nfsd nfslock — Optional file locking service. rpc.statd CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration NFSv3 Processes rpc.mountd — Handles client mount requests. rpc.nfsd — NFS server processes. rpc.lockd — Process for optional nfslock service. rpc.statd — Handles server crashes for nfslock. rpc.rquotad — Quotas for remote users. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration rpcinfo > rpcinfo -p program vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100021 1 udp 32774 nlockmgr 100021 1 tcp 34437 nlockmgr 100011 1 udp 819 rquotad 100011 2 udp 819 rquotad 100011 1 tcp 822 rquotad 100011 2 tcp 822 rquotad 100003 2 udp 2049 nfs 100003 3 udp 2049 nfs 100003 2 tcp 2049 nfs 100003 3 tcp 2049 nfs 100005 2 udp 836 mountd 100005 2 tcp 839 mountd 100005 3 udp 836 mountd 100005 3 tcp 839 mountd CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration NFSv4 Processes nfsd — NFSv4 server processes. Handles mounts. rpc.idmapd — Maps NFSv4 names (user@domain) and local UIDs and GIDs. Uses /etc/idmapd.conf. rpc.svcgssd — Server transport Kerberos auth. rpc.gssd — Client transport Kerberos auth. CIT 470: Advanced Network and System Administration

NFSv3 Server Configuration Configure /etc/exports List filesystems to be exported. Specify export options (ro, rw, etc.) Specify hosts/networks to export to. Export filesystems. exportfs Start NFS server (if not already started) service portmap start service nfs start CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration /etc/exports Format: directory hosts(options) Options ro, rw Read-only, read-write. async Server replies before write. sync Save before reply (default) all_squash Map all users to anon UID/GID. root_squash Map root to anon UID (default) no_root_squash Don’t map root (insecure.) anon{uid,gid} Set anonymous UID, GID. Examples: /home *.example.com(rw,sync) /backups 192.168.1.0/24(ro,all_squash) /ex/limited foo.example.com CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Client Configuration Manual mounting mount -t <nfs-type> -o <options> server:/remote/export /local/directory Mounting via /etc/fstab server:/remote/export /local/directory <nfs-type> <options> 0 0 NFS Type is either nfs or nfs4. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Mount Options hard or soft — Error handling hard: NFS requests will uninterruptible wait until server back. soft: NFS requests will timeout and report failure. intr — NFS requests can be interrupted if server unreachable. nfsvers=2,3— NFS protocol version (not 4) noexec — Prevents execution of binaries. nosuid — Disables setuid for security. rsize,wsize=# — NFS data block size (default 8192) sec=mode — NFS security type. sys uses local UIDs and GIDs. krb5 uses Kerberos5 authentication. krb5i uses Kerberos5 authentication + integrity checking krb5p uses Kerberos5 auth + integrity checking + encryption. tcp, udp — Specifies protocol to use for mount. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Automounter Manages NFS mounts Automounter maps vs /etc/fstab. Mounts filesystems only when needed: Makes administering many filesystems easier. Improves startup speed. Provides uniform namespaces. Ex: mounts /home/home7 as /home on login. /etc/auto.master points to maps /home /etc/auto.home Maps describe mounts * -fstype=nfs4,soft,intr,nosuid server:/home CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Security Limit which hosts have access to filesystems. Specify hosts in /etc/exports. Use iptables to limit which hosts can use NFS. Limit mount options Default to ro unless writes are necessary. Disable suid and execution unless needed. Map root to nobody. Block NFS at network firewalls. Block all protocols, not just port 2049. Use NFSv4 with Kerberos auth + encryption. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration Performance Measuring performance nfsstat /proc/net/rpc/nfsd Optimizations Increase the block size. Problem: fragments? Set the async option on mounts. Faster network card. Faster disk array. NVRAM cache on array to save NFS writes. CIT 470: Advanced Network and System Administration

CIT 470: Advanced Network and System Administration References Michael D. Bauer, Linux Server Security, 2nd edition, O’Reilly, 2005. Mike Eisler, Ricardo Labiaga, Hal Stern, Managing NFS and NIS, 2nd edition, O’Reilly, 2001. Aeleen Frisch, Essential System Administration, 3rd edition, O’Reilly, 2002. Evi Nemeth et al, UNIX System Administration Handbook, 3rd edition, Prentice Hall, 2001. NFS HOWTO, http://nfs.sourceforge.net/nfs-howto RedHat, Red Hat Enterprise Linux 4 System Administration Guide, http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/, 2005. RedHat, Red Hat Enterprise Linux 4 Reference Guide, http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/ref-guide/ch-nfs.html, 2005. CIT 470: Advanced Network and System Administration