1. Using OpenStack Sahara & Manila to run Analytics
against NFS Datastores January 2017
Presenter: Chris Madden
NetApp
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

2. OpenStack OpenStack Today SAHARA MANILA Keeps Growing! OpenStack Today
MISTRAL
MURANO
TROVE
KOLLA
OpenStack
MAGNUM
CONGRESS
IRONIC
CEILOMETER
ZAQAR
Keeps Growing!
OpenStack Today
HORIZON
BARBICAN
SAHARA
KARBOR
NOVA
SWIFT
SENLIN
CINDER
HEAT
MANILA
NEUTRON
GLANCE
OpenStack Today
TACKER
KEYSTONE
TRIPLE-O
DESIGNATE
MONASCA
HORIZON
KURYR
RALLY
CHARMS

3. Lots of data in shared file systems
NFS CIFS . Gluster _ HDFS . …
65%
storage sold for Shared File Systems (IDC)
This is why we talk about Manila
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

4. OpenStack Foundation User Survey Results
Shared File System Drivers in Use (April 2016)
NetApp is the founder of, leading contributor to, and most widely deployed option for Manila
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

5. OpenStack Manila Manila delivers file systems as a service:
The OpenStack Shared File Service Program
Manila delivers file systems as a service:
NFS, CIFS, HDFS, and more
Production-ready in Liberty
Supports a wide range of network topologies
Manila
ONTAP
OpenStack Compute VM VM VM VM
Admin
R&D
Eng
Manila is a self-service, provisioning and management framework for dispensing shared file systems as a core unit beside block and object storage within an openstack cloud.
* A sample Interaction New VM for marketing, existing share OR New VM, new shares, R&D
* So Manila API - create, delete, manage access - all of those primitives in a completely vendor agnostic framework.
* Differences with Cinder
1) Networking component : Cinder is a pretty complex service even if its role is to provision block storage, but manila has networking component... so you have to think about how do I get shares exported
into the tenant specific overlay network so if I go into Neutron and create a specific network, I may want to give access to the fileshare only to that specific Neutron net.
So a little more magic and plumbing has to happen behind the scenes to ensure a secure connection for both client and the provider of the shared file system.
2) Mounting
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

6. Hadoop Clusters are not easy!
Same for Spark, Storm, etc…
Difficult to configure & maintain
Adaption to changing demands
Hardware headaches
Commodity hardware is cheap but requires more work
Reliable hardware is expensive
Hadoop dev is very difficult without a real cluster
This is why we talk about Sahara - it simplifies all these pain points
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

7. OpenStack Sahara Put Hadoop and alike in a OpenStack cloud!
The OpenStack Data Processing Program
Put Hadoop and alike in a OpenStack cloud!
Then have easy-to-use, standardized interfaces:
To create clusters (reliably and repeatedly)
To scale clusters
To run data processing jobs
On any popular data processing framework
With sensible defaults that just work
And sophisticated configuration management for expert users
That's OpenStack Sahara!
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

8. Technical Overview External HDFS via Manila
Virtual Cluster
Virtual Cluster
Virtual Cluster VM VM VM
Computing Task
Computing Task
Computing Task
Manila Service
Manila Service
Manila Service
HDFS Driver
NetApp Hadoop NFS Connector
NFS Driver
Local Volume
Gluster Drivers or others
HDFS
NFS
GlusterFS
Host
Host
Host
External HDFS via Manila
Sahara can support external HDFS by using the HDFS driver in Manila
NFS directly via Manila
NetApp Hadoop NFS Connector brings the NFS capability into Hadoop
Local Storage in Manila
Use local storage in Hadoop to remote mount any type of storage in Manila
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

9. Summary Manila adds Fileshares-aaS to OpenStack
Sahara adds Analytics-aaS to OpenStack
NetApp NFS Connector for HDFS allows access of NFS data through HDFS protocol
All together: Easy analytics against existing datasets in file shares
Resources
© 2017 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —