1. AFS and NFS … 20 Years Later Mike Kazar kazar@netapp.com
June 2005

2. Overview Inspired by talk by Brian Pawlowski (beepy)
presented to a bunch of folks at IBM
including lots of AFS people
What were AFS’s goals? NFS’s goals?
Innovation on client and server ends
Storage management
Then and now
Some thoughts about the future
Lessons learned
Questions
9/20/2018

3. Part of AFS design team, with
Who Am I?
Part of AFS design team, with
Bob Sidebotham (Rx, Volumes)
Mike West (Server)
Sherri Nichols (VLDB &c)
M. Satyanarayanan (prototype RPC)
Dave Nichols (prototype client)
Coded parts of AFS
cache manager
kernel port of Rx
Now at NetApp
9/20/2018

4. Management was an afterthought
AFS Goals in 1984
Distributed
connect lots of workstations together
Scalability
did I mention *lots*?
key approach was caching
the fastest RPC is the one you don’t make
Security
needed isolation from random students
Management was an afterthought
but turned out to be critical
volumes
volume *moves*
mirroring
9/20/2018

5. Key points from Beepy’s talk
What is NFS?
IETF standard
bundled with all Unix/Linux systems
available on nearly everything
heterogeneous (systems, FSes)
If NFS is the answer, what was the question?
9/20/2018

6. Stick a fork in it…
FTAM
9/20/2018

7. Clients, Servers, an’ ‘at
So, why am I smiling?
Clients, Protocols, and Servers
Different goals for each
9/20/2018

8. Reference implementation
Clients
OS integration
Bug-free
or you get calls all the time
Reference implementation
helps portability a great deal
even though porting still lots of work
“Vnode” layers in Windows, AIX, Solaris, &c
9/20/2018

9. Protocols Where NFS Really Shines Where NFS sucks Public protocol spec
everyone knows they can implement it
Interoperability tests (Connectathon)
formally, helps certify who’s in the game
informally, helps communication!
set goals for future work
Reference implementation
as education tool
Where NFS sucks
cache coherence
how did they blow this in NFS version 4?
but still relatively minor
few applications use DSM
locking makes most things work
9/20/2018

10. Servers Where AFS Shines Data management global name space
cell name spaces
transparent move
transparent load balancing mirrors
flexibly allocated volumes
snapshots / clones
Usable ACLs
suprisingly
caching
using memory caches today
9/20/2018

11. NFS is the protocol of choice
Today
NFS is the protocol of choice
open licensing made it a no-brainer
simplicity also a plus for growth
Some AFS data management available today
Snapshots on NetApp filers
flexible volumes
non-transparent moves
Some AFS data management available “soon”
transparent move
load balancing mirrors
multiple cells
9/20/2018

12. OnTAP NG Architecture
9/20/2018

13. Gigabit Ethernet Switch
Architecture Detail
ONTAP / NG:
2-Stage Distributed
File System
Request switched to appropriate back-end
IP-based cluster network
No client code changes
Client Access
Client Access
Gigabit Ethernet
Gigabit Ethernet
Network Function
TCP termination
VLDB lookup
Protocol translation
to SpinFS
Network Function
TCP termination
VLDB lookup
Protocol translation
to SpinFS
SpinFS Protocol X
Disk Function
Caching
Locking
Disk Function
Caching
Locking
Gigabit Ethernet Switch
Fibre Channel
Fibre Channel 12
9/20/2018

14. History as Knobs No knobs Too many knobs Just right initial filers
one volume, period
Suns, &c
network parameters, exports, not much else
Too many knobs
tracking thousands of volumes
figuring out restores
where do I create a new volume?
Just right
self-managing based on guidance
eg. this part of the name space is a database
9/20/2018

15. Policy-based management
The Future
Policy-based management
declare part of name space “database”
inherit RAID level, drive speed
referenced abstractly
constraint engine
moves data around
with limited system impact
when desired
tied to delegation
sub-admins tied to name space parts
sub-admins constrained by resource limits
9/20/2018

16. Ties to historical data
The Future
Ties to historical data
what volumes are heavily loaded, and when?
which volumes grow and shrink?
what variation in size
what variation in load
what volumes were on this server?
backup database issue
charge-back
9/20/2018

17. The Future Quality of Service Important for managing applications
but no common framework
eg: to connnect job controller and storage
so, need to start somewhere
virtual servers, perhaps
per volume?
priorities vs. guaranteed bandwidth or ops
9/20/2018

18. Perfection is highly overrated
Lessons Learned
Perfection is highly overrated
POSIX semantics never really required
Huge effort in DCE/DFS
Universities are similar to enterprises
pointless politics and empire building
24x7 operation
availability and reliability
coordination with users nearly impossible
slightly cheaper
9/20/2018

19. Technology Transfer is Hard
Lessons Learned
Technology Transfer is Hard
20 years to get AFS volume concept out
only happened because Blake Lewis and Ed. Zayas went to NetApp
More for transparent moves
Spinnaker acquisition
Beware second system syndrome
and ignoring customer requirements
DCE/DFS was nearly complete waste of time
Morris’s point about 3rd systems
independence from inventor is critical
throw out bad ideas
allows simplification
9/20/2018

20. No one pays attention to system management
Lessons Learned
No one pays attention to system management
but it is critical to any technology deployment
QoS is part of this
scaling is part of this (managing resource pools)
centralized error reporting
dynamic reconfiguration
Don’t be greedy!
know where your real value is
know how to get help
IBM lived in fear that Sun would productize AFS!
so bungled licensing
9/20/2018