Using OpenStack Sahara & Manila to run Analytics against NFS Datastores January 2017 Presenter: Chris Madden NetApp © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

OpenStack OpenStack Today SAHARA MANILA Keeps Growing! OpenStack Today MISTRAL MURANO TROVE KOLLA OpenStack MAGNUM CONGRESS IRONIC CEILOMETER ZAQAR Keeps Growing! OpenStack Today HORIZON BARBICAN SAHARA KARBOR NOVA SWIFT SENLIN CINDER HEAT MANILA NEUTRON GLANCE OpenStack Today TACKER KEYSTONE TRIPLE-O DESIGNATE MONASCA HORIZON KURYR RALLY CHARMS

Lots of data in shared file systems NFS CIFS . Gluster _ HDFS . … 65% storage sold for Shared File Systems (IDC) This is why we talk about Manila © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

OpenStack Foundation User Survey Results Shared File System Drivers in Use (April 2016) NetApp is the founder of, leading contributor to, and most widely deployed option for Manila © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

OpenStack Manila Manila delivers file systems as a service: The OpenStack Shared File Service Program Manila delivers file systems as a service: NFS, CIFS, HDFS, and more Production-ready in Liberty Supports a wide range of network topologies Manila ONTAP OpenStack Compute VM VM VM VM Admin R&D Eng Manila is a self-service, provisioning and management framework for dispensing shared file systems as a core unit beside block and object storage within an openstack cloud.   * A sample Interaction New VM for marketing, existing share OR New VM, new shares, R&D * So Manila API - create, delete, manage access - all of those primitives in a completely vendor agnostic framework. * Differences with Cinder 1) Networking component : Cinder is a pretty complex service even if its role is to provision block storage, but manila has networking component... so you have to think about how do I get shares exported into the tenant specific overlay network so if I go into Neutron and create a specific network, I may want to give access to the fileshare only to that specific Neutron net. So a little more magic and plumbing has to happen behind the scenes to ensure a secure connection for both client and the provider of the shared file system. 2) Mounting © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Hadoop Clusters are not easy! Same for Spark, Storm, etc… Difficult to configure & maintain Adaption to changing demands Hardware headaches Commodity hardware is cheap but requires more work Reliable hardware is expensive Hadoop dev is very difficult without a real cluster This is why we talk about Sahara - it simplifies all these pain points © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

OpenStack Sahara Put Hadoop and alike in a OpenStack cloud! The OpenStack Data Processing Program Put Hadoop and alike in a OpenStack cloud! Then have easy-to-use, standardized interfaces: To create clusters (reliably and repeatedly) To scale clusters To run data processing jobs On any popular data processing framework With sensible defaults that just work And sophisticated configuration management for expert users That's OpenStack Sahara! © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Technical Overview External HDFS via Manila Virtual Cluster Virtual Cluster Virtual Cluster VM VM VM Computing Task Computing Task Computing Task Manila Service Manila Service Manila Service HDFS Driver NetApp Hadoop NFS Connector NFS Driver Local Volume Gluster Drivers or others HDFS NFS GlusterFS Host Host Host External HDFS via Manila Sahara can support external HDFS by using the HDFS driver in Manila NFS directly via Manila NetApp Hadoop NFS Connector brings the NFS capability into Hadoop Local Storage in Manila Use local storage in Hadoop to remote mount any type of storage in Manila © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

Summary Manila adds Fileshares-aaS to OpenStack Sahara adds Analytics-aaS to OpenStack NetApp NFS Connector for HDFS allows access of NFS data through HDFS protocol All together: Easy analytics against existing datasets in file shares Resources https://www.youtube.com/watch?v=Tf_8v0OvR80 https://github.com/NetApp/NetApp-Hadoop-NFS-Connector https://www.netapp.com/us/media/tr-4464.pdf © 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —