Presentation is loading. Please wait.

Presentation is loading. Please wait.

Composable Infrastructure for Data Intensive Workloads

Similar presentations


Presentation on theme: "Composable Infrastructure for Data Intensive Workloads"— Presentation transcript:

1 Composable Infrastructure for Data Intensive Workloads
Solution Overview December 2018

2 DriveScale Founders and Key Advisors
Satya Nishtala, Chief Architect Fellow, System Architect: Nuova (Cisco UCS) Lead architect: Sun - UltraSPARC workstations and work-group servers, Storage Tom Lyon, Chief Scientist Founder: Nuova (Cisco UCS) Founder: Ipsilon (Nokia) Emp #8: Sun Micro – SunOS, SPARC, SunScreen F O U N D E R S A D V I S O R S Amr Awadallah Founder/CTO Cloudera Herb Cunitz President Hortonworks Scott McNealy Founder, CEO Sun Microsystems Whit Diffie Pioneer of Public Key Cryptograhy James Gosling Creator of Java

3 DriveScale Purpose DAS
DriveScale was founded to radically transform how the modern compute platform is deployed in the data center. Seeing how the Cloud Giants were reaping the benefits of analytics and AI with massive data sets on their scale-out compute infrastructure, DriveScale set out to make it viable for any company to deploy data-intensive applications on premise and reap the same reward. DriveScale designed a software platform that creates adaptable compute infrastructure out of heterogeneous components with cloud scale and agility at a significantly lower cost than traditional options: Composable Infrastructure Composable DriveScale’s founders, Tom and Satya, were early Sun Microsystems technologists (creating NFS and architecting many generations of SPARC workstation). They later founded and sold Nuova Systems (a Cisco spin-in) to Cisco where it became the Cisco UCS and where Satya led the UCS design for the next 8 years. A little more than 6 years ago they started paying attention to how hyperscale companies were driving new revenue streams from big data and AI analytics on their massive customer data sets. To solve all of the problems that come with this type of scale, the hyperscale cloud companies were creating their own applications. Tom and Satya believed that other cloud services companies and enterprises could take advantage of these data-intensive computing applications if they were able to cost-effectively build out similar scale-out infrastructure with a new level of automation and agility. And with 10G ethernet common, it would be viable to rip open the DAS architecture and connect compute to disk and flash over Ethernet while maintaining DAS-like performance. This would help any company take advantage of Big Data and AI.

4 New Gartner Hype Cycle for Infrastructure, July 2018
Gartner Definition: “Composable infrastructure provisions physical infrastructure from a set of pooled resources consisting of disaggregated heterogeneous compute, storage and fabric components. Infrastructure can be quickly assembled and disassembled through APIs and reusable templates, with a single interface that enables resource discovery, inventory and configuration.” Composable Infrastructure will be main- stream in 2-5 years with High Benefit to enterprises. Why Enterprises Need Composable Infrastructure: Enables data centers to achieve higher resource utilization and faster application deployment Increases utilization on high-cost resources Centralizes resource control and monitoring to enable higher utilization and more predictive capacity planning

5 IDC Forecasts Composable Infrastructure Market to Reach $4.7B in 2023
The application-centric approach to IT — building environments that require new levels of scale, automation, and flexibility — can be effectively delivered via composable/ disaggregated infrastructure (CDI), a new class of converged infrastructure that brings together compute, storage, and networking elements and enabling software to provide an on-premises cloud-like experience. "New IT on-premises infrastructure deployments, mostly in the form of private cloud, will fuel aggressive investments in composable/disaggregated infrastructure," said Ashish Nadkarni, group VP for IDC's Worldwide Infrastructure practice. "The maturity of newer low-latency and high-bandwidth fabrics and interconnects will accelerate the development of newer CDI products and solutions." Worldwide Composable/Disaggregated Infrastructure Forecast, 2018–2023, August 31, IDC

6 Data is Growing Exponentially
Data Intensive Workloads Traditional Workloads Enterprises have discovered the value of their data, and the volume and velocity are growing exponentially as this IDC forecast shows. The drivers of this data deluge are new applications in Big Data analytics, social, mobile, IoT, etc. These so-called “modern workloads” require new technologies to process them, such as Hadoop, NoSQL, etc. Many of these new technologies were initially created by the hyperscale players who were the first organizations to work at this scale, but now they are in use by 1000’s of enterprises and cloud services companies. IDC Forecast DriveScale Confidential & Proprietary 2017

7 Examples of Data Intensive Workloads
Big Data & AI NoSQL MPP Containers Storage

8 Data Center Architecture Transition Required
Traditional Workloads: Many Apps Managed on Each Server / Shared Storage Data Intensive Workloads: Large Apps Running on Clusters of Servers / Local Storage App App App Shared Storage – NAS Or SAN Spark Hadoop Cassandra Server Kubernetes Server VM VM VM VM VM VM VM VM VM VM VM VM Server These modern workloads required a new data center architecture that’s referred to as “scale-out”. This scale-out architecture was originally developed by hyperscale companies like Google and Facebook. The need was for an architecture that could scale easily while minimizing costs. Data, and the processing of that data, was distributed across 100’s to 1000’s of commodity servers with internal, direct attached storage. Server Server Server Server Server Server ESX ESX ESX vSphere X 100s or 1,000s DriveScale Confidential & Proprietary 2017

9 You Want Choice AND Elasticity
sports car cargo plane COMPUTE minivan You want to be able to scale up/down both compute and storage resources for each workload. And you want to be able to do that on the fly. moped cargo truck STORAGE 9

10 Composable Infrastructure Public Cloud Agility with Private Cloud Control, Performance, Security, Cost Dynamically change your infrastructure to match individual workload requirements

11 What is Composable Infrastructure?
Two Simple Steps: 1) Disaggregate servers into… Compute resources (diskless servers) Storage resources (JBODs, NVMe EBOFs) 2) Compose nodes and clusters by… Combining servers with storage via software Dynamically re-compose as needs change

12 Step 1: Disaggregate the Servers
Typical Rack Server Configuration DriveScale Pooled Resources Diskless Servers HDD or Flash Storage Start buying diskless servers with external drives Instead of buying servers with internal drives…

13 Step 2: Dynamically Compose Servers and Clusters
DriveScale Pooled Resources Diskless Servers HDD or Flash Storage Easily compose software-defined physical nodes and clusters on-the-fly Combine just the compute and storage resources needed for each workload Easily add, remove or balance resources, even between clusters – no more silos! No changes needed to the application stack Performance equivalent to bare metal servers with direct attached storage

14 Composable Infrastructure is a better option
Storage Type DAS (Direct Attached Storage) Composable Infrastructure Centralized Storage NAS or SAN Cost Performance Utilization Flexibility Scalability The best of both worlds! The performance and scale of DAS with the utilization of NAS/SAN at the lowest possible cost. DriveScale Confidential & Proprietary 2017

15 DriveScale Composable Platform
DriveScale Composer Collects inventory from Agents Node and cluster composition and configuration Constraint management View cluster/server/drive status and health Encryption, RAID, drive shredding Composer Server Agents Compute pool API GUI Cloud Central HDD, SSD pools Adapter Agents 3rd Party Orchestration Network switches Data Path DriveScale Agents Lightweight agent runs on Linux servers and storage systems Provides server and storage inventory to Composer Executes drive mounts, encryption, file system under Composer direction Includes configuration details and monitors status DriveScale is an orchestration platform that automates the creation of Linux clusters and monitors for traffic congestion and hardware faults, actively resolving issues are instantly solving for use requests Here’s a logic view of the DriveScale software. (Step through each of the functions) Blue is 3rd party, green is DriveScale. Today we provide both the hardware and software for the adapters. We will continue to do this until there are a many 3rd party solutions available (There is a 3rd party WD/HGST SAS adapter today, and we are working with several other vendors) Linux distributions we support: SUSE (for Dell ECS only), Unbuntu, Rhel CentOS, Oracle Linux (distribution of CentOS7) DriveScale Central Cloud-based portal for administration of all sites Download software updates, view log files, documentation, etc.

16 Compute and Storage Vendor Agnostic
DriveScale works with your choice of compute and storage solutions Compute Options Rack Servers Blade Servers Modular Servers Storage Options SAS HDD/SSD in JBODs NVMe EBOF (Ethernet-attached Bunch of Flash) 3rd party storage servers SAS JBOD w/ DriveScale SAS Adapter DriveScale NVMe Adapter Western Digital Ultrastar® Serv24-HA Cisco UCS S3260

17 DriveScale SAS Adapter
Node N Node 1 Card 4 Card 3 Card 2 Card 1 Controller 1 Controller 2 DriveScale SAS Adapter ToR Switches 10 GE High Performance SAS-Ethernet Bridge 1RU appliance w/ dual power supplies 4 controllers (2 x 10 GE, 2 x 12Gb 4 lane SAS) 80 Gb throughput Leverages Ethernet for optimal performance and cost Multiple paths from any server to any drive for redundancy and performance Patented traffic balancing algorithms Diskless Servers 10GE SAS DriveScale SAS Adapter SAS JBOD

18 DriveScale NVMe Adapter
ToR Switches 100/25/10 GbE High Performance NVMe Flash System 2RU appliance w/ dual x86 controllers 24 x 2.5” form factor SSD drive bays 4 x 100Gb ethernet (2 per controller; 200Gb total throughput) High availability design (dual controller / power supplies) Supports iSCSI and NVMeOF (via RoCEv2) protocols Drives can be sliced and portions attached to servers Works with single and dual-port drives Single port uses interposer card for HA Commodity pricing for drives (customer provided) Node 1 Diskless Servers Node N 10/25GE 100GE DriveScale NVMe Adapter X86 Ctrl 1 X86 Ctrl 2

19 Benefits of Software Composable Infrastructure
Reduce Costs Spend less on infrastructure and operation Achieve higher utilization Avoid overprovisioning De-couple compute and storage lifecycles Reduce data center footprint Simplified management with fewer SKUs Respond Faster Operate your data center with the agility of a public cloud Spin-up clusters in minutes instead of weeks Respond quickly to changing workloads Share resources across clusters and applications; no more silos Recompose servers and clusters on-demand via software Recover quickly from disk or server failures Deploy with Confidence Integrates quickly and easily into your environment No changes required to the application stack Use servers and storage of your choice Same performance as bare metal servers w/ direct attached storage Redundant architecture for high availability Data security with encryption at rest and in flight

20 DriveScale Does for Clusters, What VMware Did for Servers
VMware Solved Server Utilization Inefficiencies and Flexibility Limitations DriveScale Composable Infrastructure Server DriveScale Solves Cluster Utilization Inefficiencies and Flexibility Limitations App App App Shared Storage – NAS Or SAN VM VM VM VM VM VM VM VM VM VM VM VM ESX ESX ESX vSphere © 2017 DriveScale Proprietary & Confidential

21 Composable Infrastructure for Data Intensive Workloads
Give your data center the agility of the public cloud at a fraction of the cost.


Download ppt "Composable Infrastructure for Data Intensive Workloads"

Similar presentations


Ads by Google