Hitachi’s Openstack Activities Steve Sonnenberg, HDS OpenSummit Hong Kong 2013 “Get More from the Private Cloud” – why HDS is the vendor of choice for building and managing private cloud solutions. 1

Hitachi: Committed to open stack June 2013: Hitachi joined the Red Hat OpenStack Cloud Infrastructure Partner Network August 2013: Officially announced participation and corporate sponsorship of OpenStack November 2013: Hitachi was approved as a Gold Member of the OpenStack Foundation Source: Enterprise Strategy Group Mention that HDS is a wholly owned subsidiary of Hitachi Ltd, as we will intermix both companies contributions

OVERVIEW of Openstack volume Activities For Havana 3, HDS has submitted a Cinder volume driver for its HUS/AMS Midrange SAN (iSCSI) For IceHouse, additional drivers are in development now: HNAS iSCSI HNAS NFS Midrange HUS and AMS (Fibre Channel) Enterprise VSP and HUS-VM (Fibre Channel) Source: Enterprise Strategy Group Hitachi is now building on its commitment to open source with its involvement with OpenStack. HNAS is the key technology component in HDS’s file/NAS offering

Enterprise Storage Mgr OpenStack Summit DEMO Customer Portal HNAS UI (SMU) OpenStack API Remote connection VM Hypervisor Enterprise Storage Mgr MidRange Storage Mgr Cinder Keystone Glance Control Path Nova Neutron Swift NAS Storage Mgr HCP Virtual Cluster FC/iSCSI/NFS Data Path

Volume / Snapshot Catalog

Catalog of Backups Point out that backups are stored in HCP (via Swift)

What this means to our customers HDS brings some of the most powerful storage solutions to fortune 1000 companies A partial list of features include: Virtualization / De-duplication Short-long range Replication Power-savings and efficiency Dynamic provisioning / tiering Extreme performance Highest density and flash performance … These capabilities can now be leveraged by OpenStack adopters Source: Enterprise Strategy Group Hitachi is now building on its commitment to open source with its involvement with OpenStack.

Customer is using OpenStack for DevOps Typical Use case Customer is using OpenStack for DevOps Large number of developers collaborating Each using the same image+environment Gold image + one or more appl. Volumes Need efficient cloning and storage utilization Solution Clone image and appl. volume using copy-on-write to share blocks Dynamic provisioning doesn’t pre-allocate storage Enabling our deduplication engine squeezes out block-level duplicates that start to accumulate Source: Enterprise Strategy Group

Cinder storage 3 TB of volumes (Initially cloned, then salted)

Dedupe Engine

OtheR Openstack Activities OpenStack Cloud Management Portal Web-based management UI Complex image/template management Task monitoring Shows operational progress Shows breakdown of sub-steps Source: Enterprise Strategy Group Hitachi is now building on its commitment to open source with its involvement with OpenStack. HNAS is the key technology component in HDS’s file/NAS offering

A VM environment consists of OS image and environment Complex templates A VM environment consists of OS image and environment Initially retrieved through glance Managed by hypervisor/libvirt using storage available to the compute server Application/Data volumes Not managed along with the ‘VM’ Ex. snapshot: de-attach, snap, re-attach Using APIs its not difficult to manage a VM and any/all of its volumes, uses include: Snapshot of a whole VM Backup of a whole VM Launch of a complex VM Source: Enterprise Strategy Group Hitachi added these capabilities to its management portal, demo or screenshots follow

Complex templates Operations can be performed on all volumes associated with a running VM

Building a template Template includes volumes to attach when creating the VM

Launched VM launch and volume attachment happen as a single operation

OtheR Openstack Activities Swift Object Server to HCP (Hitachi Content Platform) Gateway Planned IceHouse submission Leverages HDS world class object store Integrated Hadoop-based metadata search Source: Enterprise Strategy Group Hitachi is now building on its commitment to open source with its involvement with OpenStack. HNAS is the key technology component in HDS’s file/NAS offering

Typical SWIFT architecture OpenStack Backup/image The picture uses the word node, but server is implied

Easily Scale for Performance or Capacity 10/19/2019 Proxy, Account, Container aren’t modified Object server doesn’t manage local storage Object Server Container Server Proxy Server Account Server S3-compatible HCP There is a fundamental difference between the scalability of the solution between Swift and HCP 4 HCP nodes can manage 2PB with > 99.999% reliability This picture shows architecturally that Start with 4 Nodes and add nodes for increased capacity, object count, I/O or CPU 64 billion objects No size limitations SAN SAN Add storage for increased I/O or capacity (up to 80 petabytes)

WhY Use HCP as a SWIFT BACKEND? HCP is a 6th generation object store A partial list of features include: High Availability Data Protection, Metadata Protection, Index Protection, Multipath Proven scalability Full Object Life Cycle services Policy management Built-in Query Engine Autonomic Tech Refresh … Rather than use commodity servers and JBOD HCP builds on reliable/redundant components Source: Enterprise Strategy Group Hitachi is now building on its commitment to open source with its involvement with OpenStack.

Structure of an object store Logical Representation A container for logically related objects Possibly a directory structure (or hierarchical naming) Objects (data or metadata) Metadata describing objects Physical Representation Typically data in stored in filesystems across multiple object servers HDS modified the object server to store objects using S3 HCP is a multi-protocol object store S3, NFS, CIFS, REST, SMTP Source: Enterprise Strategy Group Hitachi is now building on its commitment to open source with its involvement with OpenStack.

A SWIFT Example Swift organizes objects in containers volumebackups: used to house the collection of volume backups 4 volumes have been backed up (shown above) The object names are hierarchical and consist of multiple objects hcp: used for demonstration

Object Details Observe: there are actually multiple objects, some containing data, others metadata. But all objects have ‘system’ metadata.

Metadata is carried using headers Application data stored as an object (below)

HCP preparation Created a ‘tenant’ with a ‘swift’ namespace

S3 data in HCP HCP has a global hierarchical filesystem system-metadata The name of the object mimics the on-disk representation that an object server would use + an algorithm to spread the objects across directories Note: some of the S3 metadata is covered by HCP system-metadata custom-metadata

Leveraging Metadata Used Cloudberry to add a ‘foo’ header HCP custom-metadata contains ‘foo’ (below)

HCP search We can perform a variety of queries including leveraging a search-profile to enable field-level recognition

THANK YOU 28