1. Airbus Archive Technologies
Mark Curtis & Phil Capp
22nd September 2017

2. Introduction Mark Curtis – LTA Architect, Service Manager
Phil Capp – Operations Manager
History
Archive Base Requirements
Current Archiving Projects Supported
Future
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

3. History ESA Projects Providing Archive capabilities since 1991
ERS-1, ERS-2 and ENVISAT Missions
Originally all on physical media and stored on shelf managed through manual database
HDDT, Exabyte, DLT, DAT, Optical Disk
Transcribed, using provided infrastructure and tools into Managed Archive System (Robotics and Database)
Size of archive at end of project ~1 Pbyte (~20 years of data)
Transcribed out of Managed Archive System to transport medium for repatriation
Sentinel-1 and Sentinel-2 LTA services (ongoing)
SWARM (ongoing)
Airbus Projects
GEOSTORE ( contains numerous clients )
16 November, 2018
LTDP Workshop – ESRIN 22/09/17

4. Archive Base Requirements
There are a number of Base Requirements which are seen, by Airbus DS, as crucial considerations when implementing Archive services:
Heterogeneous Filesystem Support (GFS) including file level security
Fast Access (In-Band Multiple Fibre Channel Connections)
Near line and Offline Storage (Robotics and Shelf)
Multiple Targets/Media Types
Multiple Copies of Files
Multiple Versions of Files
Data Fault Tolerance – what was copied to tape was what was received on disk
Automatic Media Fault Detection and Recovery
Data Compression at Hardware Level
High Throughput
High Availability
COTS – Support into the Future
16 November, 2018
LTDP Workshop – ESRIN 22/09/17

5. Archive Base Requirements
There are a number of Base Requirements which are seen, by Airbus DS, as crucial considerations when implementing Archive services:
Scalability
High Density (Small Footprint)
Low Power Consumption and Cooling
Open Standards
Industry Standard Media Formats
Support for Encryption ( embedded )
Longevity of Media (> 5 Years), Obsolescence versus Storage life
Protected Archive ( air gapped )
16 November, 2018
LTDP Workshop – ESRIN 22/09/17

6. Airbus Adopted Archive Architecture
Hierarchical Tiered storage required (many writes few reads)
Quantum StorNext
Quantum Tape Libraries (i10K, i2K, i6K)
Brocade Fabric Controllers
Disk Cache (Hitachi Data Systems) – AMS1000, AMS2500, HUS150, G600
Block Storage
LTO (LTO-4, LTO-5, LTO-6 LTO-7)
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

7. Sentinel LTA Service Requirements
Sentinel LTA Service Requirements: Daily I/O Capacity: 30 TB (Terabytes) Maximum Single file sizes: 200 GB (Gigabytes) Maximum number of files per day: Minimum sustained LTA interface transfer rate: 220 MegaBytes/Sec Architecture implementation surpasses service requirements
16 November, 2018
LTDP Workshop – ESRIN 22/09//2017

8. LTO Roadmap
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

9. Preferred Archive Architecture
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

10. LTA Archive Service
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

11. Current Projects Current Archiving Projects Sentinel-1 LTA
Scalar i6K, StorNext, LTO-6 & LTO-7, 0.5 Pbyte Cache (HDS G600)
Currently 5 Pbyte
Growth ~10TB/day
Sentinel-2 LTA
Scalar i6K, StorNext, LTO-6, 0.25 Pbyte Cache (HDS HUS150)
Currently 2 Pbyte
Growth ~4TB/day
Geostore (Airbus Services)
Scalar i6K, StorNext, LTO-4 & LTO-5, 0.2 Pbyte Cache (HDS HUS150)
Currently 0.4 Pbyte
Slow growth rate
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

12. Current Projects Geostore (Airbus Services)
Scalar i6K, StorNext, LTO-4, 0.25 Pbyte Cache (HDS HUS150)
Currently 0.2 Pbyte
No significant growth
Multiple clients and business areas
Financial Institutions, Insurance Companies, Government Agencies, Local Authorities
Data Types:
Aerial Photography
DTM
DSM
CIR
LIDAR
National Vector & Raster Imagery
> 100,000,000 files
File sizes 1K – 100GB
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

13. More Faster Longer Updates and Future Updates
Full cache replacement for Sentinel-1 LTA
Replaced HDS AMS2500 (0.25PB) with HDS G600 (0.5PB)
5 PB Filesystem migrated to new storage ~1 Day interruption to service
Future
Larger Volumes
Higher Throughput
Longer Storage
More
Faster
Longer
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

14. Updates & Future Future (continued)…………… Technologies being looked at:
Public Cloud
Private Cloud
Hybrid Cloud
Multiple Sites across Private Networks
Object storage to replace traditional block storage
The proverbial flash versus disk versus cloud versus tape argument
16 November, 2018
LTDP Workshop – ESRIN 22/09/2017

15. Thank you