1. CustomerX Setup DriveScale Environment
DriveScale Proprietary Information © 2017

2. DriveScale Proprietary Information © 2017
Table of Contents HW
2* DELL switches S4038-ON
1* Management Switch
6* Server DELL PowerEdge R630 (for 6 Data Nodes)
1* Server DELL PowerEdge R430 (for DMS)
2* DELL JBOD MD3060e with 60* SAS Drives each
2* DriveScale DSA chassis with 4* Adapter cards each SW
Network config requirements:
Switches: LLDP, Jumbo Frame 9K, MLAG
Servers: Bonding LACP, Jumbo frame 9K
DSA: Bonding LACP, Jumbo frame 9K
DriveScale SW: version 1.2
DriveScale Proprietary Information © 2017

3. DriveScale Adapter Chassis (1RU)
4 DriveScale Ethernet to SAS Adapters in 1 RU Chassis
Dual Redundant Power Supplies
With 80 Gb throughput, a single chassis can comfortably support simultaneous access to 80 drives w/ equivalent performance to Direct Attached Storage
2x 10GbE Interfaces per Adapter
2x 12Gb 4 Lane SAS Interfaces per Adapter
DriveScale Proprietary Information © 2017

4. DriveScale Proprietary Information © 2017
Rack Architecture
DriveScale Proprietary Information © 2017

5. MANAGEMENT NETWORK DSA Adapter 4 Adapter 3 Adapter 2 Adapter 1 DSA
Management Plane Switch
Switch 1
Switch 2
MLAG
DMS
(Up to 3 VM Servers for HA)

Controller 2
Controller 1
Data Node 1
1GE (mngt)
40GE
MANAGEMENT NETWORK
Kerberos
DHCP…
10GE
SAS
JBOD

6. SAS CONNECTIVITY Adapter 4 Adapter 3 Adapter 2 Adapter 1 DSA DSA
Switch 1
Switch 2
MLAG

Controller 2
Controller 1
1GE (mngt)
40GE
Data Node 1
10GE
SAS CONNECTIVITY
SAS
JBOD

7. DriveScale NETWORK Hadoop NETWORK ….. Adapter 4 Adapter 3 Adapter 2
DSA
DSA
Switch 1
Switch 2
MLAG
HortonWorks
Data Node 1
…..
1GE (mngt)
Data Node 6
40GE
10GE
DriveScale NETWORK
Hadoop NETWORK
SAS

8. Final Architecture ….. Controller 2 Controller 1 JBOD 1 JBOD 2

Controller 1
JBOD 1
Final Architecture
JBOD 2
Controller 2

Controller 1
Adapter 4
Adapter 3
Adapter 2
Adapter 1
Adapter 4
Adapter 3
Adapter 2
Adapter 1
DSA 2
DSA 1
Switch 1
Switch 2
MLAG
HortonWorks
1GE (mngt)
40GE
Data Node 1
10GE
…..
SAS
Data Node 6
DriveScale (iSCISI) NETWORK
Hadoop NETWORK

9. DriveScale iSCSI vs Hadoop Data Network Separation
We have Six 10Gbps NICs on each server.
Four NICs are connected to a single VLAN using MLAG as the 'Hadoop' network. We have the DNS routing for the FQDN of each machine set so it resolved them to the IP address on that VLAN. That is the one that Hadoop, ssh, etc. would use to contact the server.
The other two NICs are connected to a separate VLAN using MLAG as the 'DriveScale' network that is part of the subnet and DriveScale bandwidth domain with the adapters. To achieve the separation that CustomerX wants, that VLAN should not have a default route set on the servers, nor any device willing to route traffic.
All NICSs are connected through a single pair of switches, with one (or two) link (s) of each VLAN/MLAG pair going to each switch.
DriveScale Proprietary Information © 2017

10. Communication Flow – Diagram
Drivescale
Central
HTTPS
HTTPS (80)
SSH (22)
HTTPS
HTTPS (443)
Zookeeper
(2281 TCP)
HTTPS (8444,443)
Zookeeper
(2281 TCP)
HTTPS (8443)
Zookeeper (2888, 3888 TCP)
MongoDB (27017)
HTTPS
HTTPS/SSH
iSCSI (wide port range)
HTTP (38202)

11. Communication Flow – TCP ports for Firewall Policy
User -> DMS - Management interface access (HTTPS , HTTP redirect to HTTPS on 80)
if using haproxy for HA, then HTTPS of individual server (not haproxy) is on 8443
User -> Adapter - Network configuration - HTTPS
Adapter -> DMS - software updates (HTTPS on 8444)
DMS -> DriveScale Central - software updates (HTTPS)
DMS -> DriveScale Central - statistics / logs upload (HTTPS)
DMS -> DMS - haproxy health check (HTTPS on 8443)
DMS -> DMS - VRRP for HA setup
DMS -> DMS - Zookeeper replication (2888, 3888 TCP)
DMS -> DMS - MongoDB replication + access (27017 TCP)
Server/Adapter/DMS -> DMS - Zookeeper access (2181 TCP)
Server -> Adapter - iSCSI (wide port range, one iSCSI portal per drive used)
Server -> Adapter, HTTP on alternate port for balancing (Live Data load Monitoring Reported from Adapter).
Server -> DriveScale Central - software install (HTTPS)
Server/Adapter -> DMS Log Report (HTTPS)
DriveScale Proprietary Information © 2017

12. DriveScale Setup - Prerequisites
You must complete the racking and cabling of the Servers, DSA, JBOD’s and switches according to the details of the network requirements.
You must obtain the credentials of your DriveScale Central account which is shipped with hardware.
You must decide whether to set up your DMS as one standalone server or as a high-availability cluster with three servers. When three are used, the Management Domain can survive the failure of any one of the DMS machines.
The DMS servers should be configured with minimum HW Requirements (depends on targeted number of DSAs/Nodes/Drives to be supported).
DHCP server in the 10G/1G network (unless using static IP assignment)
Access to the DHCP administrator/server to get the IP address(es) of the DSA(s) based on the MAC address(es) of the DSA(s).
The network address(es) of your DMS server(s).
The network address(es) of your DSA(s).
The network addresses of your compute servers.
DriveScale Proprietary Information © 2017

13. 1. DriveScale Central – login
Log in to DSC with the credentials obtained from DS.
Go to
Log in using the credentials provided to you by DriveScale.
A checklist of the tasks that need to be accomplished appears on the main DSC page.
DriveScale Proprietary Information © 2017

14. DriveScale Proprietary Information © 2016
1. DriveScale Central (Cont…) – create your DriveScale Management Domain
Go to the Domains link in the left navigation panel and click on Create Domain.
Fill the name, FQDN name and any notes for the domain. Click on Create.
DriveScale Proprietary Information © 2016

15. 1. DriveScale Central (Cont…) – download packages
Go to the Downloads link in the left navigation panel and download the config-Training,ds-dms-keys-xxx.rpm and ds-repo-xxx.rpm files on your local machine.
DriveScale Proprietary Information © 2016

16. 2. DMS Setup – HW/SW Requirements
Supported OS: Linux RHE or Centos 6/7, Ubuntu 14.x.
The below options will cover DMS resource for managing up to 125 Nodes DSA adapter cards Drives. Once you get closed to this threshold, we’ll ask you to add more resource to the config (CORE/RAM/DISK SPACE). 
Option 1 (recommended) : 3x Servers (HA), each with the following config:
2 cores
8 GB RAM
A local SSD for OS + ZooKeeper transaction logs (size >= 100 GB).
A local SSD for the ZooKeeper snapshots (size >= 100 GB)
1 GbE NIC
Option 2: Single Server with 2x SSDs (in RAID1):
2x local SSDs for OS + ZooKeeper transaction logs + ZooKeeper snapshots (size >= 200 GB).
DriveScale Proprietary Information © 2017

17. 2. DMS Setup (Cont…) – Overview
Copy RPMs and the “DriveScale Domain” Config file received from Drivescale Central to the DMS host.
Install the RPMs on DMS
Copy config file to drivescale directory on DMS
Start the Drivescale Agent Service
Verify the DMS is running and GUI is accessible from your host
DriveScale Proprietary Information © 2017

18. 2. DMS Setup (Cont…) – install packages
Copy RPMs and config file received from Drivescale Central to DMS from your localhost (via scp or sftp)
scp ds-* config-*
b) SSH to your DMS, install the RPMs and ds-dms service
cd /tmp
rpm -ivh ds-repo-* ds-dms*
yum install ds-dms –y
Note: It is recommended to do a YUM UPDATE to update your current Linux packages.
c) Copy the config file you received from Drivescale Central to DMS drivescale directory:
cp config-* /etc/drivescale/conf && chmod 660 etc/drivescale/conf
DriveScale Proprietary Information © 2017

19. 2. DMS Setup (Cont…) – prepare Zookeeper
The DriveScale Management Server uses the ZooKeeper package to manage the configuration of elements of the system.
If there will be more than about 500 servers and DriveScale Adapters (DSA) in use in the data center, the number of open file descriptors available to ZooKeeper may need to be increased, so that it can maintain a connection to the agents on each server and DSA.
The following files needs to updated in CentOS/RHEL:
- /etc/security/limits.conf
root - nofile 32768
- /etc/sysctl.conf
fs.file-max = 32768
DriveScale Proprietary Information © 2017

20. 2. DMS Setup (Cont…) – start DriveScale Service
d) Start the Drivescale Agent Service
Note: In below example, we turn off Firewall and SELINUX on the DMS installed on a Linux RHE7 or Centos7 host. If you do want to keep the Firewall on, make sure the required TCP ports are open as described in Communication Flow slide (10) .
# turn off firewall
systemctl stop firewalld
chkconfig firewalld off
#disable SE linux
sed -i 's/(^SELINUX=).*/SELINUX=disabled/' /etc/sysconfig/selinux
sed -i 's/(^SELINUX=).*/SELINUX=disabled/' /etc/selinux
reboot …
# Verify SE linux status
sestatus
# Start DriveScale service
systemctl start ds-dms
#Run DMS setup script to initiate DMS WEB UI local authentication
/opt/drivescale/bin/setup-mode
DriveScale Proprietary Information © 2017

21. 2. DMS Setup (Cont…) – connect to DMS Web UI
e) Open a browser and connect to DMS on port 443:
f) Setup your DMS Web UI credentials
g) Reconnect to your DMS URL and login
with your credentials.

22. DriveScale Proprietary Information © 2017
3. DSA Setup – overview
Each DriveScale Adapter (DSA) has 3 physical Ethernet interfaces and 3 Logical interfaces:
One 1Gb Ethernet physical Interface:
Used for management purpose only
Two IP interfaces (and MACs) are mapped to this 1GE interface for the following 2 logical interfaces:
One logical SUP Interface — Supervisor interface for the adapter.
DMS communicates with this interface for configuring the adapter.
One logical management interface — Management interface for the adapter.
This interface can be used by the adapter for sending updates to the DMS (optionally to segregate Control Plan Traffic between DSA and DMS)
Two 10Gb Ethernet physical interfaces
These are the data interfaces.
They are used for connecting compute resources to the JBODs
They can be used by the adapter for sending updates to the DMS (optionally share Control Plan and Data Traffic between DSA and DMS)
By default, the Two 10Gb interfaces are bonded (LACP based)
A single IP interface is mapped to this Bonded interface
By default, all IP interfaces are requesting IP assignment via DHCP
DriveScale Adapter (DSA) Network interfaces can be configured from DMS using the DSA Utility tool described in the next slide.
Complete commands and usage of DSA utility tool is described in the administration guide.
DriveScale Proprietary Information © 2017

23. 3. DSA Setup (Cont…) – Discover IP via DHCP
DHCP Server
Sup MAC-Adapter1
Sup MAC-Adapter2
DHCP Request
Adapter 4
Adapter 3
Adapter 2
Adapter 1
DSA
Chassis
Management Plane Switch
Sup MAC-Adapter3
Sup MAC-Adapter4
DHCP Offer
DMS
MAC-DMS
FQDN: dms.CustomerX.com
Static IP: /24
GW:
1. Save of each Adapter Supervisor interface (see printed on chassis or/and packaging box)
2. Connect a DHCP server to Management and provide IP (and FQDN) assignment in same subnet as DMS (or routable to DMS)
3. Save IP/Mac mapping of each Adapter Supervisor interface
DriveScale Proprietary Information © 2017

24. 3. DSA Setup (Cont…) – configure the DSA adapter
Log into one of the DMS machines using ssh.
Set up the DMD configuration for each DSA using the same config file as was used in the DMS.
This is done via the /opt/drivescale/bin/dsa script command available in Linux shell of the DMS host:
The script runs always from DMS towards the Supervisor IP of the DSA adapter
The default DSA adapter Supervisor username/password is admin/admin (this can be changed via configuration if needed).
Run the command listed below to check the current configuration and settings of each DSA adapter:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> service showconf
Push the config from the DMS to the DSA adapter:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> service config --file /etc/drivescale/conf
Restart the DSA adapter
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> service restart
DriveScale Proprietary Information © 2017

25. 3. DSA Setup (Cont…) – set the DSA adapter with static IPs
SSH to the DMS host
Configure the Supervisor interface with static IP/gateway… etc:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> net modify --interface sup --static netmask gateway fqdn a1m.dsa1.CustomerX.com
Verify the Supervisor interface Static IP’s address:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> net show --interface sup
Configure the Management interface with static IP/gateway… etc:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> net modify --interface mgmt --static netmask gateway fqdn a1m.dsa1.CustomerX.com
Verify the Management interface Static IP’s address:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> net show --interface mgmt
Configure the 10Gbps Bond interface with static IP address:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> net modify --interface 10gBond --static netmask gateway
Verify the 10Gbps Bond interface Static IP’s address:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> net show --interface 10gBond
DriveScale Proprietary Information © 2017

26. DriveScale Proprietary Information © 2017
Set up DSAs
11. Confirm the DSAs are setup when the Drivescale Management Serfver (DMS) sees them in physical inventory.
Note: Once the DSA is found, you will like also find several connected device, discovered by LLDP, such as switches, JBODs, and drives.
DriveScale Proprietary Information © 2017

27. 4. Troubleshooting – get logs
SSH to the DMS host
DSA logs: run the command against the Supervisor interface to collect logs package of the DSA:
/opt/drivescale/bin/dsa --username admin --password admin --adapter <Sup IP of DSA adapter> logs
=> It should download the logs in current directory (where you run the command on DMS) as xxxx-logs.tgz
DMS logs are in /var/log/drivescale directory
DriveScale Proprietary Information © 2017

28. DriveScale Proprietary Information © 2016
Questions & Answers
DriveScale Proprietary Information © 2016

29. DriveScale Proprietary Information © 2016