1. NeST: Network Storage Flexible Commodity Storage Appliances John Bent, Miron Livny, Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau

2. Terms zAppliance (Merriam-Webster) yb : an instrument or device designed for a particular use; specifically a household or office device zStorage appliance yStorage plus access methods

3. What storage users want zReliability and availability zManageability ycost of management > cost of storage itself y“no futz” computing zScalability zPerformance

4. What storage vendors have zNetApp, EMC, others make storage appliances (network-attached storage) zManageable yJust plug it in and it works yAdministrative web interface zReliable and available yStandard RAID techniques zHigh performance ySpecialized, thin OS focused on serving files

5. What storage vendors get, annual revenues NetApp $800 million in 2000 EMC $9 billion in 2000

6. What’s the problem? zFalse coupling between HW and SW z“Playground syndrome” zMyth of specialization

7. H/W and S/W are bundled zHardware decisions are imposed zHard to ride commodity curve yExample: xNetapp F720 $35,000.00, 252 GB $138 / GB xMaxtor DiamondMax $279.00, 80 GB $3.50 / GB

8. “Playground syndrome” z“We have storage appliances... yif you use these protocols, yif you use these security mechanisms, yif you are comfortable with our data semantics” zNon-flexible software entity

9. Myth of specialization zSpecialize for one protocol on one machine zSpecialization decreases over time as yProtocols are added yProduct line expands zExample: Netapp software yGeneration 1 fit on a single floppy yGeneration 2 took six yGeneration 3?

10. Alternatives? zAppliance (Merriam-Webster) ya : a piece of equipment for adapting a tool or machine to a special purpose

11. Our game? zFlexible, commodity based, software-only storage appliances zGoal yFind a networked machine y“Drop” some software on it yHave a ready to use storage appliance with flexible mechanisms

12. New worlds, new problems zDiverse hardware, software platforms yNetapp, EMC advantage xfewer platforms, control over OS yOur approach xAutomate configuration to each host system Hardware example - use file system or self-manage Software example - use either read/write or mmap zCost of flexibility zKey is design of the software

13. Outline zIntroduction zBuilding flexible storage modules yBig picture yProtocol layer yConcurrency architecture yStorage layer zMotivations for flexible storage appliances zConclusion and current status

14. NeST structure zCleanly separated modules for communication, transfer and storage yProtocol layer xMaps diverse protocols into common control flows yConcurrency architectures xDifferent models to maximize system throughput yStorage layer xProvides abstract interface to disks

15. NeST structure Protocol Layer GFTPNeSTWiNDHTTPNFS Central Control Concurrency Architecture Event driven Multi- process Multi- threaded Transfer request Storage Layer Raw diskLocal FSRAID

16. Protocol layer A collection of servers is less than the sum of their parts. HTTPdNFSd Operating system NeST HTTPNFS

17. Consolidate protocols zSingle point of control yStorage quotas and guarantees can be supported across multiple protocols. yBandwidth can be controlled and quality of service can be guaranteed. zSingle administrative interface ySet policies yManage user accounts

18. Protocol layer implementation zEach protocol listens on well-defined port zCentral control accepts connections zProtocol layer reads from connection and returns generic request object zLike Linux V-nodes yAdd new protocol by writing a couple of methods

19. Protocol layer example, directory list request FTP NeST speak Protocol layer “31: LIST” “5” Central control Directory list Storage layer Directory list Linked list “ftp, ftp, ftp” “nest, nest”

20. Concurrency architecture zThree difficult goals yLow latency yHigh bandwidth yMultiple simultaneous clients zNo single portable solution yProvide multiple models to provide solutions on a range of different platforms xMulti-threaded xMulti-process xEvent driven

21. Concurrency architecture Event drivenMulti-processMulti-threaded zCentral control creates transfer object ySocket descriptor from the protocol layer yFile descriptor from the storage layer zTransfer object passed to concurrency architecture

22. Concurrency on Linux

23. Storage layer zThree needed areas of flexiblity yFile systems interfaces xExample: read()/write() or mmap() yAbstract storage models xRAID, JBOD, etc. yUser account administration xCreation and removal xQuotas and guarentees for users and groups

24. File system interfaces on Linux

25. Outline zIntroduction zBuilding flexible storage modules zMotivations for flexible storage appliances zConclusion and current status

26. Clients have different needs zCommunication protocols zReplacement costs zData semantics zSecurity and authentication

27. Communication protocols zThe Esperanto problem zToo many protocols to implement them all zToo many clients use proprietary protocols Storage must allow pluggable protocols.

28. Replacement costs zInfinite cost to replace first class data. zVariable cost to replace cached data depending on size and distance. zVariable cost to replace job output files depending on computation cost. Cheap cached files First class data Cost aware storage can effectively increase its own capacity.

29. Data semantics zMust stored objects be protected from read and write dependencies? zIs transaction support necessary? zAcceptable replies to storage requests.

30. Data semantics, example zProblem yPFS on top of FTP fakes open yread may then return file not found zSolution yMechanisms are needed to support flexible semantics independent of the transfer protocol. Divorce semantics from the protocol.

31. Security and authentication zOwnership zPrivacy zEncryption zAuthentication zAccess rights

32. Who, when, how and how much? Abstinent Promiscuous zWho is allowed to use the storage? zPromiscuity and monogamy are easy zPolygamy is also easy

33. Do I know you? zProblem yMigrant grid users may need temporary, preferential storage access zSolution yProvide mechanisms to xadvertise available storage xcreate self-destructing user accounts Matchmake applications with storage opportunities.

34. Outline zIntroduction zBuilding flexible storage solutions zMotivations for flexible storage appliances zConclusion yCurrent status yFuture work yConcluding remarks

35. Current status zConcurrency architectures are done yGets, puts, reads and writes perform well zVirtual protocol class interface is built yNeST speak is fully implemented yGrid ftp coming soon!! zSimple first implementation of storage reservations and remote quota management is done yVenkateshwaran Venkataramani

36. Future work zDiscovery process of client storage requirements zQuality of service guarantees for bandwidth and storage zSupport for transient and opportunistic users

37. Concluding remarks zReturn storage to the commodity curve by creating software-only storage appliances zAllow greater storage flexibility for a wide range of application needs