Intel® Manager for Lustre* Lustre Installation & Configuration Implementing an IEEL Solution Session 04 June 2014 Implementing an IEEL Solution Session 04: Lustre Installation & Configuration Lustre Installation and Configure Lab– This session will use the IML tool to create a small Lustre file system. The lab will include configuration of Lustre metadata servers (MDS/MDTs) and storage servers (OSS/OSTs) in failover mode, and the mounting of the file system on Lustre clients. The session will also include a review of the Linux server parameters and IML’s ability to configure advanced file system settings. See: IEEL Software Product Brief   See: IEEL Partner Installation Guide See: IEEL Release Notes See: IML User Guide

Description of Lustre Installation Lab You will use IML Manager to deploy a Lustre file system Review Lustre best practices employed by IEEL Create a Lustre file system Mount Lustre v2.5 Clients Review IML monitoring a fully configured Lustre file system Lustre Installation and Configuration Description of Lustre Installation Lab Lustre Installation Lab Steps --- Allotted time 1 hrs ---- {HELP Topic Covered -> Getting started; Creating a new Lustre file system; Monitoring Lustre file systems} Lustre 2.3 features Review of IEEL Training setup Hardware configuration Do’s and Don’ts Lustre Install Create a File System Why is 2GB (21%) of my OST storage capacity listed as Used with the file system empty? By default, Lustre serves 400 MB for journaling if there was no external journal device specified. However, 400 MB journal is not enough for fast storage arrays. I often had to manually set journal to 2GB. As OSTs in current Lustre systems would be 22TB~27TB, 2GB journal would be hardly noticeable.  So I feedback that to the IML team and make that default. It is one example that IML builds some practical best practice into it. There are more features like that in the HA configuration. Add and mount clients Must hit refresh to see file system when complete For this lab you will need to access the IEEL Training Cluster via: Browser (Recommended: Google Chrome v27 or Mozilla Firefox v22) Terminal or Windows Putty Command Line Interface

Lustre Installation and Configuration Lustre Training Cluster IEEL Training Cluster Review for Lustre CentOS 6.4 Installed on all servers and clients IML Agent installed on all servers Cluster Network consists of: (1) Management Server – Configured (1) Lustre Network (LNET) –Configured LNET interface manually defined in the servers /etc/modprobe.d/lustre.conf file (1) Failover Network - Unconfigured Cluster consists of: (1) Management Server running IML (2) MDS servers to be configured in active-passive failover Note: The passive MDS server will run as the MGS (1) MGT 1GB volume (1) MDT 10GB volume Shared storage volumes for MGT, MDT (2) OSS servers to be configured as a storage unit in active-active failover (4) OST 10GB volumes Multipath I/O not configured for this lab Shared storage volumes for MGT, MDT, OST1-4 (2) Lustre clients DNS Name Domain Name CPU RAM LUNs LUN Size LUN Description stxx-iml1 stxx-iml01 1 4 GB VDA 10 GB OS Disk stxx-mds1 stxx-mds01 1 2 GB VDA 10 GB OS Disk SDA 10 GB MDT Shared SDB 1 GB MGT Shared stxx-mds2 stxx-mds02 1 2 GB VDA 10 GB OS Disk stxx-oss1 stxx-oss01 1 2 GB VDA 10 GB OS Disk SDA 10 GB OST00 Shared SDB 10 GB OST01 Shared SDC 10 GB OST02 Shared SDD 10 GB OST03 Shared stxx-oss2 stxx-oss02 1 2 GB VDA 10 GB OS Disk stxx-cli1 stxx-cli01 1 2 GB VDA 10 GB OS Disk

Lustre Installation and Configuration IEEL Training Cluster Review Lustre Installation Requirements IML implements a set of Lustre Best Practices during Lustre install This puts specific requirements against the server hardware configurations Install Intel® EE for Lustre* only as described in the IEEL Partner Installation Guide Required operating system: Red Hat* Enterprise Linux v6.5 or CentOS v6.5 Network and DNS configuration: Confirm correct network and DNS (FQDNs). The management network interfaces must be properly configured and active Do not configure IP addresses for the HA crossover cable interfaces (server to target) RAM and mass storage: Provide adequate memory and disk space Take advantage of the IEEL Partner support and request a review by Intel HPDD See “Determining Space Requirements” and “Determining Memory Requirements” in the Lustre Operations Manual. IEEL Training Cluster Review for Lustre IEEL Installation Requirements Install Intel® EE for Lustre* only as described in the IEEL Partner Installation Guide IML implements a set of Lustre Best Practices during Lustre Install This puts specific requirements against the server hardware configurations Review “General requirements for installation” & “Storage server and target requirements” in the IEEL Partner Installation Guide Required operating system: Red Hat* Enterprise Linux v6.4 or CentOS v6.4 Note: Installing Intel® EE for Lustre* software automatically disables SELinux on all storage servers. Note: Installation also configures firewalls on the manager and storage servers. IML server Recommendation: At least 24 GB of redundant storage for the OS RAID 10 is recommended for operating systems local storage. Swap space equal to twice the available RAM present At least a 500 GB /var partition Storage server Recommendation: At least 6 GB of redundant storage for the OS Network and DNS configuration: Correct network and hostname (FQDNs). Management Network configured on all servers and clients 100 Mb/s or greater Ethernet network bandwidth The LNet configuration file /etc/modprobe.d/lustre.conf must be deployed with changes to support your LNet topology. See Chapter 9: Configuring Lustre Networking in the Lustre Operations Manual. /etc/modprobe.d/lustre.conf file must map the correct network interface to LNET The lustre.conf file must be manually configured on each of the Linux servers prier to the IML lustre install IF /etc/modprobe.d/lustre.conf file is not found IML v2.0 will take the 1st Ethernet interface as tcp Do not configure IP addresses for the HA crossover cable interfaces (server to target) Do install Ethernet crossover cables between the HA server pairs (e.g. MDSs, OSSs) Paired failover servers must be connected to each other via a dedicated, crossover Ethernet connection. IF you are using Infiniband*, you may want to install the following package Yum groupinstall “Infiniband Support” RAM and mass storage: Provide adequate memory and disk space To determine memory and disk space requirements for Lustre Take advantage of the IEEL Partner support and request a review by Intel HPDD

Lustre Installation and Configuration IEEL Training Cluster Review Lustre Installation Requirements (cont) Server Configuration: Servers are configured in HA pairs, with a primary and secondary server Storage servers should have SSH running with root login permitted. Non-root users should be prevented from logging into the storage servers Do not install or configure Lustre, Corosync, or Pacemaker. Do not configure NTP on storage servers Do provide power control to the IML manager via either IPMI or PDU outlets Storage Configuration: Read “Adding storage servers” in the IEEL Partner Installation Guide Provide separate Volumes for the MGT and the MDT Shared storage for HA on MDT, MGT, OSTs IEEL Training Cluster Review for Lustre IEEL Installation Requirements Install Intel® EE for Lustre* only as described in the IEEL Partner Installation Guide IML implements a set of Lustre Best Practices during Lustre Install This puts specific requirements against the server hardware configurations Review “General requirements for installation” & “Storage server and target requirements” in the IEEL Partner Installation Guide Server Configuration: Servers are configured in HA pairs, with a primary server and secondary server The sshd daemon must be started on the Linux servers Root login is required on the server if you want to use SSH to Add Server to IML Note: Non-root users should be prevented from logging into storage nodes. Do not install or configure Lustre, Corosync, or Pacemaker. Corosync and Pacemaker are automatically deployed to the new storage server. This simplifies software installation and avoids possible errors. Do not configure NTP on storage servers IML will time synchronize all Lustre servers to the IML management server Time synchronize the IML management server to the your internal network or the outside world Storage Configuration: Read “Adding storage servers” in the IEEL Partner Installation Guide Provide separate LUNs for the MGT and the MDT Storage controllers and drives must already be installed and LUNs must already be configured and present. Linux multipath I/O not configured for this lab, recommended for actual Linux hardware Any required multipath configuration between each storage server and its physical storage devices must be complete. RAID 10 is recommended for metadata targets (MDT) RAID 6 is recommended for and object storage targets (OSTs) The MGT and metadata target (MDT) cannot be located on the same volume The MGT and metadata target (MDT) can be located on the same server. Share storage volumes for MGT, MDT, OSTs to appropriate HA pairs Each server of a failover pair must have access to exactly the same LUNs as its peer. Each LUN is visible only to its two, paired failover servers. Paired failover servers must be connected to each other via a dedicated, crossover Ethernet connection. Consider using an IML Storage Plugin for single point system management of the cluster Power Control: Provide IPMI or PDU outlet power control to the IML manager for each server Each server must be powered by two power distribution units, for redundancy. Paired HA servers must have control over each other’s PDUs outlets IML Agent installed on all servers, All servers are visible in the IML GUI

Lustre Installation and Configuration High Availability Requirements IML implements High Availability as a part of Lustre Best Practices See “Creating a new HA Lustre file system” in the IEEL Installation Guide Storage servers are deployed in a high availability (HA) configuration with shared storage. Servers are configured in primary and failover server pairs. When a server becomes unavailable, Lustre services that were running on that server are started on another server attached to the same storage. To support HA failover, each HA server must be able to automatically power-off its peer server if a failover is required. Powering off a faulty server is known as "node fencing”, and ensures that a shared storage device is not mounted by more than one server at a time. IEEL Training Cluster Review for Lustre High Availability Requirements IML implements High Availability as part of Lustre Best Practices during Lustre Install See “Creating a new HA Lustre file system” in IEEL Installation Guide Storage servers are typically deployed in a high availability (HA) configuration with shared storage. When a server becomes unavailable, Lustre services for targets that were running on that server are started on another server attached to the same storage (known as “failover”). To support HA failover, each HA server must be able to automatically power-off its peer server if a failover is required. The process of powering off a faulty server is known as STONITH or "node fencing" (also called "server fencing"), and ensures that a shared storage device is not mounted by more than one server at a time. Lustre includes protection against multiple simultaneous device mounts, but automatically powering off the faulty server ensures that failover works properly. Intel® Manager for Lustre* software supports the use of remotely-operable Power Distribution Units (PDUs) for this purpose. Assign PDU outlets to servers.  Alternative to the configuration of PDUs, Intel® Manager for Lustre* software also supports the Intelligent Management Platform Interface (IPMI) and associating baseboard management controllers (BMCs) with servers, to support server monitoring and control. Network and DNS configuration: Correct network and hostname (FQDNs). Do not configure IP addresses for the HA crossover cable interfaces (server to target) Do install Ethernet crossover cables between the HA server pairs (e.g. MDSs, OSSs). Paired failover servers must be connected to each other via a dedicated, crossover Ethernet connection. Lustre file permissions: If a storage node is to be used as a metadata server (MDS), to properly enforce Lustre file permissions, the MDS must have access to the same UID/GID database as the Lustre clients. For example, if the Lustre clients are using LDAP to provide network-wide user account information, the MDS must be configured to check LDAP for user account information. If a pair of nodes has been configured as HA peers for an MDT, then LDAP must be configured on both nodes to ensure proper functionality in the event of a failover.

Lustre Installation and Configuration High Availability Requirements (cont) IML implements High Availability as a part of Lustre Best Practices See “Creating a new HA Lustre file system” in the IEEL Installation Guide Provide shared storage for HA on MDT, MGT, OSTs Each server of a failover pair must have access to exactly the same LUNs as its peer. Each LUN is visible only to its two, paired failover servers. To properly enforce Lustre file permissions The MDS must have the same UID/GID database access as the Lustre clients Using LDAP, the MDS’s must be configured to check for user account information on both nodes to ensure proper functionality in the event of a failover. IEEL Training Cluster Review for Lustre High Availability Requirements IML implements High Availability as part of Lustre Best Practices during Lustre Install See “Creating a new HA Lustre file system” in IEEL Installation Guide Storage servers are typically deployed in a high availability (HA) configuration with shared storage. When a server becomes unavailable, Lustre services for targets that were running on that server are started on another server attached to the same storage (known as “failover”). Server Configuration: Servers are configured in HA pairs, with a primary server and secondary server Do not install or configure Corosync, or Pacemaker. Corosync and Pacemaker are automatically deployed to the new storage server. This simplifies software installation and avoids possible errors. Storage Configuration: Read “Adding storage servers” in the IEEL Partner Installation Guide Share storage volumes for MGT, MDT, OSTs to appropriate HA pairs Each server of a failover pair must have access to exactly the same LUNs as its peer. Each LUN is visible only to its two, paired failover servers. Paired failover servers must be connected to each other via a dedicated, crossover Ethernet connection. Power Control: Provide IPMI or PDU outlet power control to the IML manager for each server Each server must be powered by two power distribution units, for redundancy. Paired HA servers must have control over each other’s PDUs outlets

Diagram: MDS pair IEEL Training Cluster Review for Lustre High Availability Requirements IML implements High Availability as part of Lustre Best Practices during Lustre Install See “Creating a new HA Lustre file system” in IEEL Installation Guide Storage servers are typically deployed in a high availability (HA) configuration with shared storage. When a server becomes unavailable, Lustre services for targets that were running on that server are started on another server attached to the same storage (known as “failover”). Server Configuration: Servers are configured in HA pairs, with a primary server and secondary server Do not install or configure Corosync, or Pacemaker. Corosync and Pacemaker are automatically deployed to the new storage server. This simplifies software installation and avoids possible errors. Storage Configuration: Read “Adding storage servers” in the IEEL Partner Installation Guide Share storage volumes for MGT, MDT, OSTs to appropriate HA pairs Each server of a failover pair must have access to exactly the same LUNs as its peer. Each LUN is visible only to its two, paired failover servers. Paired failover servers must be connected to each other via a dedicated, crossover Ethernet connection. Power Control: Provide IPMI or PDU outlet power control to the IML manager for each server Each server must be powered by two power distribution units, for redundancy. Paired HA servers must have control over each other’s PDUs outlets

Diagram: OSS pair IEEL Training Cluster Review for Lustre High Availability Requirements IML implements High Availability as part of Lustre Best Practices during Lustre Install See “Creating a new HA Lustre file system” in IEEL Installation Guide Storage servers are typically deployed in a high availability (HA) configuration with shared storage. When a server becomes unavailable, Lustre services for targets that were running on that server are started on another server attached to the same storage (known as “failover”). Server Configuration: Servers are configured in HA pairs, with a primary server and secondary server Do not install or configure Corosync, or Pacemaker. Corosync and Pacemaker are automatically deployed to the new storage server. This simplifies software installation and avoids possible errors. Storage Configuration: Read “Adding storage servers” in the IEEL Partner Installation Guide Share storage volumes for MGT, MDT, OSTs to appropriate HA pairs Each server of a failover pair must have access to exactly the same LUNs as its peer. Each LUN is visible only to its two, paired failover servers. Paired failover servers must be connected to each other via a dedicated, crossover Ethernet connection. Power Control: Provide IPMI or PDU outlet power control to the IML manager for each server Each server must be powered by two power distribution units, for redundancy. Paired HA servers must have control over each other’s PDUs outlets

Lustre Installation and Configuration IML Administrator Login [You will need the user name and password you created during the IML Installation.] Note: If still logged into IML as admin skip Steps 1 - 3 Step 1: Enter the IP address and your training clusters access port in your browser 206.221.159.56:80xx Step 2: Click on the “Login” link in the upper right hand corner of the GUI Step 3: Enter the superuser name and password you created during IML Installation session and click the Login button admin admin.123

Clear IML Status Lustre Installation and Configuration IML - Status Note: We will take a moment to review the Yellow Status of the cluster and return to Green Status Note: The ability to Dismiss all only works with the superuser role. Step 1: Double Click on the “Status” light in the upper toolbar In the “Status” section you should see: All Alerts Commands Events Dismiss all Current History Note: The Status Alerts were created during the rebooting of the servers Step 2: Click on the Dismiss all In the “Server Configuration” section you should see: Note: The Status light should return to Green for menu for normal file system operations

Clear IML Status Lustre Installation and Configuration IML - Status Note: We will take a moment to review the Yellow Status of the cluster and return to Green Status Note: The ability to Dismiss all only works with the superuser role. Step 1: Double Click on the “Status” light in the upper toolbar In the “Status” section you should see: All Alerts Commands Events Dismiss all Current History Note: The Status Alerts were created during the rebooting of the servers Step 2: Click on the Dismiss all In the “Server Configuration” section you should see: Note: The Status light should return to Green for menu for normal file system operations

Server Configuration Lustre Installation and Configuration IML – Server Configuration Note: We will perform a left to right review of the following Configuration Tabs: Servers -> Volumes -> Power Control -> MGTs -> File Systems -> Storage Step 1: Click on the “Configuration” tab in the upper toolbar Step 2: If not on the Servers tab then Click on the “Servers” tab on the “Configuration” page toolbar In the “Server Configuration” section you should see: stxx-mds1 LNET up stxx-mds2 LNET up stxx-oss1 LNET up stxx-oss2 LNET up Step 3: Click on the “Actions” menu for one of the servers Under Actions, specific to each server, you can perform the following commands. These commands are used primarily to decommission servers. You should see: Remove: Remove this server. If this server is configured as the primary server of an HA pair, then the file system will failover to the secondary server. If this not configured as an HA server, then any file systems or targets that rely on this server will also be removed. Stop LNet: Shut down the LNet networking layer and stop any targets running on this server. Unload LNet: If LNet is running, stop LNet and unload the LNet kernel module to ensure that it will be reloaded before any targets are started again. Force Remove: This action removes the record for the storage server in the manager database, without attempting to contact the storage server. All targets that depend on this server will also be removed without any attempt to unconfigure them. You should only perform this action if the server is permanently unavailable. Reboot: Initiate a reboot on this server. If this server is configured as the primary server of an HA pair, the file system will failover to the secondary server until this server is back online. The file system will then fail back to the primary server. If this is not configured as an HA server, then any file systems targets that rely on this server will be unavailable until rebooting is complete. Shutdown: Initiate an orderly shutdown on this server. If this server is configured as the primary server of an HA pair, the file system will failover to the secondary server. If this is not configured as an HA server, then any file systems targets that rely on this server will be unavailable. Power Off: Switch power off for this server. Any HA-capable targets running on the server will be failed-over to a peer. Non-HA-capable targets will be unavailable until power for the server is switched on again. This button is visible only if PDUs have been added and outlets assigned to servers. Power On: Switch power on for this server. This button is visible only if PDUs have been added and outlets assigned to servers, and after the server has been powered-off at PDU Power Cycle: Switch power off and then back on again for this server Any HA-capable targets running on the server will be failed over to a peer. Non-HA-capable targets will be unavailable until the server has finished booting. This button is visible only if PDUs have been added and outlets assigned to servers.

Server Actions Lustre Installation and Configuration IML – Server Configuration Note: We will perform a left to right review of the following Configuration Tabs: Servers -> Volumes -> Power Control -> MGTs -> File Systems -> Storage Step 1: Click on the “Configuration” tab in the upper toolbar Step 2: If not on the Servers tab then Click on the “Servers” tab on the “Configuration” page toolbar In the “Server Configuration” section you should see: stxx-mds1 LNET up stxx-mds2 LNET up stxx-oss1 LNET up stxx-oss2 LNET up Step 3: Click on the “Actions” menu for one of the servers Under Actions, specific to each server, you can perform the following commands. These commands are used primarily to decommission servers. You should see: Remove: Remove this server. If this server is configured as the primary server of an HA pair, then the file system will failover to the secondary server. If this not configured as an HA server, then any file systems or targets that rely on this server will also be removed. Stop LNet: Shut down the LNet networking layer and stop any targets running on this server. Unload LNet: If LNet is running, stop LNet and unload the LNet kernel module to ensure that it will be reloaded before any targets are started again. Force Remove: This action removes the record for the storage server in the manager database, without attempting to contact the storage server. All targets that depend on this server will also be removed without any attempt to unconfigure them. You should only perform this action if the server is permanently unavailable. Reboot: Initiate a reboot on this server. If this server is configured as the primary server of an HA pair, the file system will failover to the secondary server until this server is back online. The file system will then fail back to the primary server. If this is not configured as an HA server, then any file systems targets that rely on this server will be unavailable until rebooting is complete. Shutdown: Initiate an orderly shutdown on this server. If this server is configured as the primary server of an HA pair, the file system will failover to the secondary server. If this is not configured as an HA server, then any file systems targets that rely on this server will be unavailable. Power Off: Switch power off for this server. Any HA-capable targets running on the server will be failed-over to a peer. Non-HA-capable targets will be unavailable until power for the server is switched on again. This button is visible only if PDUs have been added and outlets assigned to servers. Power On: Switch power on for this server. This button is visible only if PDUs have been added and outlets assigned to servers, and after the server has been powered-off at PDU Power Cycle: Switch power off and then back on again for this server Any HA-capable targets running on the server will be failed over to a peer. Non-HA-capable targets will be unavailable until the server has finished booting. This button is visible only if PDUs have been added and outlets assigned to servers.

Lustre Network Configuration Lustre Installation and Configuration IML – Server Configuration Note: We will perform a left to right review of the following Configuration Tabs: Servers -> Volumes -> Power Control -> MGTs -> File Systems -> Storage Step 1: Click on the “Configuration” tab in the upper toolbar Step 2: If not on the Servers tab then Click on the “Servers” tab on the “Configuration” page toolbar In the “Server Configuration” section you should see: stxx-mds1 LNET up stxx-mds2 LNET up stxx-oss1 LNET up stxx-oss2 LNET up Step 3: Click on the “Actions” menu for one of the servers Under Actions, specific to each server, you can perform the following commands. These commands are used primarily to decommission servers. You should see: Remove: Remove this server. If this server is configured as the primary server of an HA pair, then the file system will failover to the secondary server. If this not configured as an HA server, then any file systems or targets that rely on this server will also be removed. Stop LNet: Shut down the LNet networking layer and stop any targets running on this server. Unload LNet: If LNet is running, stop LNet and unload the LNet kernel module to ensure that it will be reloaded before any targets are started again. Force Remove: This action removes the record for the storage server in the manager database, without attempting to contact the storage server. All targets that depend on this server will also be removed without any attempt to unconfigure them. You should only perform this action if the server is permanently unavailable. Reboot: Initiate a reboot on this server. If this server is configured as the primary server of an HA pair, the file system will failover to the secondary server until this server is back online. The file system will then fail back to the primary server. If this is not configured as an HA server, then any file systems targets that rely on this server will be unavailable until rebooting is complete. Shutdown: Initiate an orderly shutdown on this server. If this server is configured as the primary server of an HA pair, the file system will failover to the secondary server. If this is not configured as an HA server, then any file systems targets that rely on this server will be unavailable. Power Off: Switch power off for this server. Any HA-capable targets running on the server will be failed-over to a peer. Non-HA-capable targets will be unavailable until power for the server is switched on again. This button is visible only if PDUs have been added and outlets assigned to servers. Power On: Switch power on for this server. This button is visible only if PDUs have been added and outlets assigned to servers, and after the server has been powered-off at PDU Power Cycle: Switch power off and then back on again for this server Any HA-capable targets running on the server will be failed over to a peer. Non-HA-capable targets will be unavailable until the server has finished booting. This button is visible only if PDUs have been added and outlets assigned to servers.

Server Actions Lustre Installation and Configuration IML – Configuration – Servers (cont) We will review the Server Actions available for Managing the Lustre Storage Note : Please do not Click on the Server Action functions at this time Note : The Server Action functions will be used in the Lustre Operations session of this class Observe : In the Server Actions section you can: Detect File Systems: Detect an existing file system to be monitored at the manager GUI Re-scan NIDs: Query the network interfaces on the storage servers to update the record of NIDs. Any system reconfiguration that changes a Lustre network endpoint node ID and/or network type will require that you rescan NIDs.  See Handling network address changes (updating NIDs) in IML Help. Re-write Target Configuration: Update each target with the current NID for the server with which it is associated. This is necessary after making changes to server/target configurations and is done after rescanning NIDs. See Handling network address changes (updating NIDs) in IML Help. Install Updates: When an updated release of Intel® Manager for Lustre* software is installed at the manager server, a notification is displayed at the manager GUI that updated software is also available for installation on a managed server or servers. After clicking the Install Updates button, a list of servers (default: all) to be included in this update operation is displayed in the Update dialog. Clicking the Run button in this dialog will cause the updated packages to be installed on the managed servers.

Volume Configuration Lustre Installation and Configuration IML – Configuration – Volumes Note: The Volumes listed are the Linux server volumes available from each of the servers added into IML. Help: All GUI pages and tabs -> Configuration page -> Volumes tab The Volumes tab is for adding volumes to a file system. Each Lustre target corresponds to a single volume. Only volumes that are not already in use as Lustre targets or local file systems are shown. If servers in the volume have been configured for high availability, primary and secondary servers can be designated for a Lustre volume. A volume may be accessible on one or more servers via different device nodes, and it may be accessible via multiple device nodes on the same host. On the Volume Configuration tab, you can do the following: •View the status of all volumes in all file systems. •View each volume's name, primary server, failover server, volume size, and volume status. ▪A green status light for the volume indicates that the volume has a primary and failover server. ▪A yellow status light means that there is no failover server. ▪A red status light indicates that this volume is not available.

Power Control – Add PDU/IPMI Lustre Installation and Configuration IML – Configuration – Power Control Note: Due to virtualized environment actually adding Power Control to the IML is a virtual demonstration Step 1: Click on the “Configuration” tab in the upper toolbar Step 2: Click on the “Power Control” tab in the Configuration toolbar Step 3: Click on the “Add PDU” tab on the Power Control page Step 4: Click on the “Device Type” tab on the Add PDU pulldown menu to observe the list of supported device types. Step 5: Click on “Close” Note: Due to virtualized environment Power Control is a virtual demonstration

Creating a new MGT Lustre Installation and Configuration IML – Configuration – MGTs Creating a New MGT Step 1: Click on the MGTs tab in the Configuration page Step 2: Click on Select Storage Step 3: Highlight stxx-mgt Note: This should have a 1GB Capacity with a Primary server stxx-mds1 and a Secondary server stxx-mds2 Step 4: Click on Create new MGT Note: You may get the message: The selected volume stxx-mgt contains a filesystem of type ext4, although it may not be in use. Do you wish to overwrite it? Click Overwrite Help: All GUI pages and tabs -> Configuration page -> MGTs tab At the MGT tab, you can do the following: •View your existing management target (if configured). Here you can determine the Capacity, Type, and high availability (HA) Status of the MGT. If this is an HA target, then the primary and secondary servers are identified. A green check mark indicates this target and server are functioning normally. •Select storage for a new MGT and then create a new MGT.  This task is not common; MGTs are created when you click Create File System at the Configuration > File Systems tab. Under MGT Configuration for an existing MGT, you can perform these actions under Actions: •Stop: Stop the MGT. When an MGT is stopped, clients are unable to make new connections to the file systems using this MGT. However, the MDT and OST(s) stay up if they were started before this MGT was stopped, and can be stopped and restarted while this MGT is stopped. •Failover: Clicking Failover will forcibly migrate the target to its failover server. Clients attempting to access data on the target while the migration is in process may experience delays until the migration completes. If this button is not displayed, then the MGT has already failed-over and this button will display as Failback. Otherwise, a secondary server has not been configured. •Failback: Migrate the target back to its primary server. Clients attempting to access data on the target while the migration is in process may experience delays until the migration completes. This button is displayed only after a target has failed-over.

Creating a new MGT Lustre Installation and Configuration IML – Configuration – MGTs Creating a New MGT Step 1: Click on the MGTs tab in the Configuration page Step 2: Click on Select Storage Step 3: Highlight stxx-mgt Note: This should have a 1GB Capacity with a Primary server stxx-mds1 and a Secondary server stxx-mds2 Step 4: Click on Create new MGT Note: You may get the message: The selected volume stxx-mgt contains a filesystem of type ext4, although it may not be in use. Do you wish to overwrite it? Click Overwrite Help: All GUI pages and tabs -> Configuration page -> MGTs tab At the MGT tab, you can do the following: •View your existing management target (if configured). Here you can determine the Capacity, Type, and high availability (HA) Status of the MGT. If this is an HA target, then the primary and secondary servers are identified. A green check mark indicates this target and server are functioning normally. •Select storage for a new MGT and then create a new MGT.  This task is not common; MGTs are created when you click Create File System at the Configuration > File Systems tab. Under MGT Configuration for an existing MGT, you can perform these actions under Actions: •Stop: Stop the MGT. When an MGT is stopped, clients are unable to make new connections to the file systems using this MGT. However, the MDT and OST(s) stay up if they were started before this MGT was stopped, and can be stopped and restarted while this MGT is stopped. •Failover: Clicking Failover will forcibly migrate the target to its failover server. Clients attempting to access data on the target while the migration is in process may experience delays until the migration completes. If this button is not displayed, then the MGT has already failed-over and this button will display as Failback. Otherwise, a secondary server has not been configured. •Failback: Migrate the target back to its primary server. Clients attempting to access data on the target while the migration is in process may experience delays until the migration completes. This button is displayed only after a target has failed-over.

Creating a new MGT Lustre Installation and Configuration IML – Configuration – MGTs Creating a New MGT Step 1: Click on the MGTs tab in the Configuration page Step 2: Click on Select Storage Step 3: Highlight stxx-mgt Note: This should have a 1GB Capacity with a Primary server stxx-mds1 and a Secondary server stxx-mds2 Step 4: Click on Create new MGT Note: You may get the message: The selected volume stxx-mgt contains a filesystem of type ext4, although it may not be in use. Do you wish to overwrite it? Click Overwrite Help: All GUI pages and tabs -> Configuration page -> MGTs tab At the MGT tab, you can do the following: •View your existing management target (if configured). Here you can determine the Capacity, Type, and high availability (HA) Status of the MGT. If this is an HA target, then the primary and secondary servers are identified. A green check mark indicates this target and server are functioning normally. •Select storage for a new MGT and then create a new MGT.  This task is not common; MGTs are created when you click Create File System at the Configuration > File Systems tab. Under MGT Configuration for an existing MGT, you can perform these actions under Actions: •Stop: Stop the MGT. When an MGT is stopped, clients are unable to make new connections to the file systems using this MGT. However, the MDT and OST(s) stay up if they were started before this MGT was stopped, and can be stopped and restarted while this MGT is stopped. •Failover: Clicking Failover will forcibly migrate the target to its failover server. Clients attempting to access data on the target while the migration is in process may experience delays until the migration completes. If this button is not displayed, then the MGT has already failed-over and this button will display as Failback. Otherwise, a secondary server has not been configured. •Failback: Migrate the target back to its primary server. Clients attempting to access data on the target while the migration is in process may experience delays until the migration completes. This button is displayed only after a target has failed-over.

Create a new File System Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System Step 1: Click on the “File Systems” tab in the Configuration toolbar Step 2: Click on Create File System Help: All GUI pages and tabs -> Configuration page -> File Systems tab The File Systems tab lets you configure, view and manage multiple file systems. Click Create File System to begin the process of creating a new file system. See Create a new Lustre* file system Under Current File Systems, for each file system you can: •view the file system name •view the management server •view the metadata server •view the number of connected clients •view total file system capacity (Size) •view available free space •check file system status. A green check mark indicates that the file system is operating normally. No warnings or error messages have been received. Under Actions, you can: •Remove the file system: This file system is removed and will not be available to clients. However this file system's contents will remain intact until its volumes are reused in another file system. •Stop the file system: This stops the metadata and object storage targets, thus making the file system unavailable to clients. If the file system has been stopped, to restart the file system, click Start.
  To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

Create a new File System (cont) Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System (cont) 1. Set file system options: Step 1a: Type in a File system name: Lustre01 Step 1b: Click on Set Advanced Settings and observe the possible parameter changes Note: Use care when changing these parameters as they can significantly impact functionality or performance. For help with these settings, contact your storage solution provider. Note: Advanced settings can be updated later in the Configuration > File Systems > Lustre01 > Update Advanced Settings menu 2. Choose one management target (MGT): Step 2a: Click on the drop down menu and select stxx-mds2 3. Choose one metadata target (MDT): Step 3a: Click on Select Storage Step 3b: Click on stxx-mdt (10 GB) on stxx-mds1 Step 3c: Click on Set Advanced Options Note: These advanced settings can only be set at the time of file system creation. Default values are recommended Observe: Stripes without EA block:13 Estimated file limit: 5.31M files 4. Choose object storage targets (OSTs): Step 4: Click on Select All Observe: Estimated file system capacity: 40GB 5. Finish Step 5: Click on Create File System   To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

Create a new File System (cont) Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System (cont) 1. Set file system options: Step 1a: Type in a File system name: Lustre01 Step 1b: Click on Set Advanced Settings and observe the possible parameter changes Note: Use care when changing these parameters as they can significantly impact functionality or performance. For help with these settings, contact your storage solution provider. Note: Advanced settings can be updated later in the Configuration > File Systems > Lustre01 > Update Advanced Settings menu 2. Choose one management target (MGT): Step 2a: Click on the drop down menu and select stxx-mds2 3. Choose one metadata target (MDT): Step 3a: Click on Select Storage Step 3b: Click on stxx-mdt (10 GB) on stxx-mds1 Step 3c: Click on Set Advanced Options Note: These advanced settings can only be set at the time of file system creation. Default values are recommended Observe: Stripes without EA block:13 Estimated file limit: 5.31M files 4. Choose object storage targets (OSTs): Step 4: Click on Select All Observe: Estimated file system capacity: 40GB 5. Finish Step 5: Click on Create File System   To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

File system up and running Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System (cont) 1. Set file system options: Step 1a: Type in a File system name: Lustre01 Step 1b: Click on Set Advanced Settings and observe the possible parameter changes Note: Use care when changing these parameters as they can significantly impact functionality or performance. For help with these settings, contact your storage solution provider. Note: Advanced settings can be updated later in the Configuration > File Systems > Lustre01 > Update Advanced Settings menu 2. Choose one management target (MGT): Step 2a: Click on the drop down menu and select stxx-mds2 3. Choose one metadata target (MDT): Step 3a: Click on Select Storage Step 3b: Click on stxx-mdt (10 GB) on stxx-mds1 Step 3c: Click on Set Advanced Options Note: These advanced settings can only be set at the time of file system creation. Default values are recommended Observe: Stripes without EA block:13 Estimated file limit: 5.31M files 4. Choose object storage targets (OSTs): Step 4: Click on Select All Observe: Estimated file system capacity: 40GB 5. Finish Step 5: Click on Create File System   To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

HA management Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System (cont) 1. Set file system options: Step 1a: Type in a File system name: Lustre01 Step 1b: Click on Set Advanced Settings and observe the possible parameter changes Note: Use care when changing these parameters as they can significantly impact functionality or performance. For help with these settings, contact your storage solution provider. Note: Advanced settings can be updated later in the Configuration > File Systems > Lustre01 > Update Advanced Settings menu 2. Choose one management target (MGT): Step 2a: Click on the drop down menu and select stxx-mds2 3. Choose one metadata target (MDT): Step 3a: Click on Select Storage Step 3b: Click on stxx-mdt (10 GB) on stxx-mds1 Step 3c: Click on Set Advanced Options Note: These advanced settings can only be set at the time of file system creation. Default values are recommended Observe: Stripes without EA block:13 Estimated file limit: 5.31M files 4. Choose object storage targets (OSTs): Step 4: Click on Select All Observe: Estimated file system capacity: 40GB 5. Finish Step 5: Click on Create File System   To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

Advanced Settings Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System (cont) 1. Set file system options: Step 1a: Type in a File system name: Lustre01 Step 1b: Click on Set Advanced Settings and observe the possible parameter changes Note: Use care when changing these parameters as they can significantly impact functionality or performance. For help with these settings, contact your storage solution provider. Note: Advanced settings can be updated later in the Configuration > File Systems > Lustre01 > Update Advanced Settings menu 2. Choose one management target (MGT): Step 2a: Click on the drop down menu and select stxx-mds2 3. Choose one metadata target (MDT): Step 3a: Click on Select Storage Step 3b: Click on stxx-mdt (10 GB) on stxx-mds1 Step 3c: Click on Set Advanced Options Note: These advanced settings can only be set at the time of file system creation. Default values are recommended Observe: Stripes without EA block:13 Estimated file limit: 5.31M files 4. Choose object storage targets (OSTs): Step 4: Click on Select All Observe: Estimated file system capacity: 40GB 5. Finish Step 5: Click on Create File System   To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

Installing and mounting lustre clients Lustre Installation and Configuration IML – Configuration - File Systems Creating a New File System (cont) 1. Set file system options: Step 1a: Type in a File system name: Lustre01 Step 1b: Click on Set Advanced Settings and observe the possible parameter changes Note: Use care when changing these parameters as they can significantly impact functionality or performance. For help with these settings, contact your storage solution provider. Note: Advanced settings can be updated later in the Configuration > File Systems > Lustre01 > Update Advanced Settings menu 2. Choose one management target (MGT): Step 2a: Click on the drop down menu and select stxx-mds2 3. Choose one metadata target (MDT): Step 3a: Click on Select Storage Step 3b: Click on stxx-mdt (10 GB) on stxx-mds1 Step 3c: Click on Set Advanced Options Note: These advanced settings can only be set at the time of file system creation. Default values are recommended Observe: Stripes without EA block:13 Estimated file limit: 5.31M files 4. Choose object storage targets (OSTs): Step 4: Click on Select All Observe: Estimated file system capacity: 40GB 5. Finish Step 5: Click on Create File System   To view the full display of file system parameters, click on the file system name in the left column. See View All File System Parameters.

Dashboard View Lustre Installation and Configuration IML – Lustre Install & Configuration Dashboard View of Lustre01 Note: For the convenience of the class two clients are provided with installed Lustre v2.4 packages provided in IEEL v1.0.0 Step 1: Select Dashboard in the Menu bar Step 2: Select Lustre01 in the file system drop down menu Step 3: Observe the Capacity Used, Inodes used, and number of Clients connected. Note: Lustre best practices configure 2 GB for a journal file on each OST. Although this looks like a large percentage in this small virtual environment this is a minimal requirement in todays high capacity storage devices.

Questions? End of Session 04 – Lustre Installation and Configuration