1. Pablo Pinés León – FTEC 2016 Program
BE Investigate redundant network filesystem technologies for CERN's Accelerator Control System servers
Pablo Pinés León – FTEC 2016 Program

2. Introduction
BE-CO-IN is responsible for the infrastructure of CERN’s accelerators control servers
ACC-ADM team
There is currently one dedicated NFS server for each of 4 subsystems
Around 1500 being served in total
An NFS server unavailability can affect many client machines
Client machines cannot be rebooted
Recovery can take significant effort (restart required)
High Availability solution would be a clear improvement
This infrastructure can be used for other services too

3. First steps Learning Ansible and git
Evaluation of high-availability technologies
E-groups management
Benchmarking of physical and virtual systems (CPU/Storage/Network)
Support on 2FA and deployment of servers on the Technical Network

4. investigation Previous effort with GlusterFS not completely successful
CEPH suggested
It was deemed not adequate due to its resources requirements
Alternatives explored though online search (HA-NFS, Mars,…)
Active/Active solution considered (GFS, OCFS2)
Decision was taken to use Pacemaker and its associated stack
Lots of documentation
Support from big companies (Red Hat, Suse…)

5. Architecture Design Target OS is CentOS 7
Active/Passive on a 2 nodes setting
4 nodes configuration investigated
Requires a newer software version for which no precompiled packages exist
More complicated set up, specially in terms of connections
2 nodes was considered a “good enough” improvement
Heartbeat deprecated in CentOS 7

6. Software Stack Virtual IP Active Sync NFS Master Slave Filesystem Ext4
DRBD
Primary
Secondary
Resource
Manager
Pacemaker
Messaging
Corosync
Hardware
Node 1
Node 2

7. Test environments First on SLC6 machine on Openstack
Network problems (multicast, virtual IP)
Then VirtualBox on my Workstation
First on SLC6, Python version too old
Tried on Ubuntu, successful
CentOS 7 successful too
Currently, two Physical Machines (workstations)
Floating Virtual IP
Significant issues encountered

8. Cluster Status view

9. Virtual Personal Computers
VPCs are VMs for developers in the accelerator servers on CERN’s Openstack
BE, TE, EN developers
ACC Operational support
Support on the creation, configuration and troubleshooting
Issues with new templates
New ways to create VMs and meet users needs
Phasing out old infrastructure
Dealing with other groups
Monitoring on DIAMON

10. NEXT STEPs Implement the solution on real servers Add more services
Test dedicated connection for DRBD
Add more services
Continue the work on VPCs
3 screens configuration for CentOS 7
White Hat Challenge
Egroups management and other ACC-ADM tasks

11. Any questions?