Preventing a Disaster -“A GB of Backup is Worth a TB of Sleep.” www.fortlewis.edu Backup Strategies Preventing a Disaster -“A GB of Backup is Worth a TB of Sleep.” Parker Jardine – April 2017

Preventing a Disaster Disaster Prevention and Focus on Availability www.fortlewis.edu Preventing a Disaster Disaster Prevention and Focus on Availability Network Redundancy Design Vmware vSphere Metro Storage Cluster Design 3rd Datacenter Site Backup Strategies Current Backup Issues Fort Lewis College’s Backup Strategy The “1” in the 3-2-1 Backup Strategy The Ransomware Threat – Backup Strategy Changes A Second Look At Tape – And Long Term Data Retention

Preventing a Disaster Network Design www.fortlewis.edu Preventing a Disaster Network Design The current generation FLC Network is a two tier, dual core campus network fully meshed with a two tier, dual core data center network. Datacenter isolation from campus core (all user networks)

www.fortlewis.edu Preventing a Disaster Vmware vSphere Metro Storage Cluster (Uniform Configuration) Primary Storage High Availability using HPE 3PAR Peer Persistence Vmware Stretch Cluster for Datacenter and Compute HA Resource Utilization – 50% rule

Preventing a Disaster 3rd Datacenter Site Network Connectivity www.fortlewis.edu Preventing a Disaster 3rd Datacenter Site Network Connectivity Backup Storage, Servers, and Tape Libraries

Nature’s Backup Strategy www.fortlewis.edu Nature’s Backup Strategy

World Backup Day Strategy www.fortlewis.edu World Backup Day Strategy

3 – Keep 3 copies of any important file: 1 primary and 2 backups. www.fortlewis.edu 3-2-1 Backup Strategy US-CERT Recommended Backup Strategy (Including the Pros, Cons and Security Implications 1 3 – Keep 3 copies of any important file: 1 primary and 2 backups. 2 – Keep the files on 2 different media types to protect against different types of hazards. 1 – Store 1 copy offsite (e.g., outside your home or business facility). Common Backup Strategies Disk To Cloud (No onsite storage) Disk To WAN (No onsite storage) Disk To Disk To Cloud Disk To Disk To WAN Disk Disk To Disk To Tape Disk To Tape Disk To Disk 1 - https://www.us-cert.gov/sites/default/files/publications/data_backup_options.pdf

The “1” in the 3-2-1 Backup Strategy www.fortlewis.edu The “1” in the 3-2-1 Backup Strategy

The “1” in the 3-2-1 Backup Strategy www.fortlewis.edu The “1” in the 3-2-1 Backup Strategy A primary backup storage device is used for quick backup and restore processes Backups are then copied to a secondary backup storage device using a native Backup Copy Job. 1 Backup Copy Job to disk Not simply copying the backup file to another disk, but a secondary backup copy job Should the primary backup be encrypted or become corrupt, the Backup Copy Job would also fail because the backup software would not be able to interpret the data. 1 Removable hard disks This is the scenario of connecting external USB drives that can be disconnected They should be interchanged regularly and should not be kept connected to the system permanently. 1 1 - https://www.veeam.com/blog/how-to-protect-against-ransomware-data-loss-and-encryption-trojans.html

The “1” in the 3-2-1 Backup Strategy www.fortlewis.edu The “1” in the 3-2-1 Backup Strategy Tape Tapes should be exported to a secure location for optimum protection. 1 Storage snapshots and replicated VMs Semi-Offline instances of data Storage snapshots and replicated VMs are usually created and updated on a schedule Cloud backup repository Off-site data Access anywhere Now native functionality within some backup software 1 - https://www.veeam.com/blog/how-to-protect-against-ransomware-data-loss-and-encryption-trojans.html

Fort Lewis College’s Backup and Recovery Strategy www.fortlewis.edu Fort Lewis College’s Backup and Recovery Strategy Current Backup Issues Growth of on-premise data is still increasing substantially VM Servers are still growing Camera data, Logging data, ERP data Data integrity and Backup Verification Virtual Machine Incident Different procedures for backing up different systems Appliances and unique systems Threat of Ransomware within the environment Veeam Endpoint protection agents Not Cluster Aware (For our Legacy File Share and SQL Clusters) Unlimited Budgets? Primary Storage redundancy, network redundancy, multiple data centers, cooling, generators, fire suppression, backup storage, tape libraries, off-site storage repositories IT staff resources

4.1 Backup Plan Fort Lewis College’s Backup and Recovery Strategy www.fortlewis.edu Fort Lewis College’s Backup and Recovery Strategy 4.1 Backup Plan Server backups are performed nightly, every day of the week including weekends and holidays. Server backup verification jobs will run daily to validate the entire disk contents and check for any silent data corruption for the most recent backups. All backups are retained for the length of time related to the retention policy applied to the backup job before recycling. See section 4.2 for retention policies. Active full server backups are performed monthly, occurring on the last Saturday and Sunday of the month. Windows File Shares will be protected with Volume Shadow Copies, allowing 1 month of previous versions on folders and files. All backup data will be stored in the primary backup storage location. The primary backup location is a fully redundant datacenter on campus. This location has its own redundant switching, backup power, and cooling.

Fort Lewis College’s Backup and Recovery Strategy www.fortlewis.edu Fort Lewis College’s Backup and Recovery Strategy A third copy of the most recent ERP database server and supported middleware servers will be moved to an off-site cloud provider daily (On the roadmap). A third copy of all production backup data will be encrypted and placed on drone to fly up to the mountain site. It will then then return the following week for the next data load and system charge (On the roadmap). The entire IT systems administration team will manage and monitor backups. Backups will be automated using Veeam Backup and Replication and Veritas Backup Exec. Daily backup job summary emails will be sent directly to the IT Managers and delegated IT staff as desired.

Fort Lewis College’s Backup and Recovery Strategy www.fortlewis.edu Fort Lewis College’s Backup and Recovery Strategy Any backup related issues will be reported to the Director of Information Technology and action will be taken to quickly fix the problem. Baseline backups will be performed on the server before moving the server into production. Decommission server backups will be performed on the servers before removing it from the environment. Decommission server backups will be retained for 30 days after removal.

Fort Lewis College’s Backup and Recovery Strategy www.fortlewis.edu Fort Lewis College’s Backup and Recovery Strategy 4.2 Retention Policies Non-Production Retention Policy (14 day retention) Test and Non-Production Server Data: All test and non-production server data is retained for 14 days. Production Retention Policy (3 month retention) Production Server Data: All production server data, including servers that are used to monitor and manage the environment. Production Extended Retention Policy ( 6 month retention) Email and ERP system data: All on-premise email data and the college’s critical ERP system data is retained for 6 months. Archive Retention Policy (1 year retention) File Share Data: All student, faculty, and staff file share data located on the M, O and other mapped drives. The last monthly active full backup is retained for a 1 year.

The Ransomware Threat – Backup Strategy Changes www.fortlewis.edu The Ransomware Threat – Backup Strategy Changes https://www.fbi.gov/news/stories/incidents-of-ransomware-on-the-rise Latest Ransomware Threat LOCKY Ransomware family called Osiris (Nov 2016 – March 2017) secondary malware that included a keystroke logger spoofed Email Delivery Notifications from UPS, FEDEX or USPS with a zip files attached the Zip file was a JavaScript file disguised as a word document If the JavaScript file was executed it would retrieve executables from  compromised WordPress sites on the Internet

The Ransomware Threat – Backup Strategy Changes www.fortlewis.edu The Ransomware Threat – Backup Strategy Changes Safeguarding Backup Repositories from Ransomware Backup Repository Servers should not be connected to AD Should be protected as much as possible from the rest of the network Local user accounts only Backup Software service account access Only account that has access to the backup repository data NAS Backup Repository Only backup service account should have access to the backup repository data Domain Administrators Should never login to a local desktop with a domain administrator account This can lead to the ransomware spreading around the network very quickly Firewall must be turned on! And not be configured for IP/ANY A virus scanner with an activated real-time search

A Second Look At Tape – And Long Term Data Retention www.fortlewis.edu A Second Look At Tape – And Long Term Data Retention Security Benefits Becoming an increasingly popular option for IT to leverage again in regards to encryption Trojans Tapes do not enable direct data access, and thus provide protection against ransomware The Enterprise Strategy Group, Inc has completed a detailed economic analysis for data archiving. Source: ESG White Paper, Analyzing the Economic Value of LTO Tape for Long-term Data Retention, 2016

A Second Look At Tape – And Long Term Data Retention www.fortlewis.edu A Second Look At Tape – And Long Term Data Retention Source: ESG White Paper, Analyzing the Economic Value of LTO Tape for Long-term Data Retention, 2016

A Second Look At Tape – And Long Term Data Retention www.fortlewis.edu A Second Look At Tape – And Long Term Data Retention http://resources.idgenterprise.com/original/AST-0163679_3_3_2016_ESG_Economic_Value_of_LTO_Tape_TCO_REV.pdf Source: ESG White Paper, Analyzing the Economic Value of LTO Tape for Long-term Data Retention, 2016

Questions? Defrag Animation www.fortlewis.edu