Download presentation
Presentation is loading. Please wait.
Published byBaldwin Walker Modified over 8 years ago
1
NetCDF4 Trade Study PO.DAAC Data are available compressed – GNUzip or Bzip2 Data compressor PO.DAAC OPeNDAP and THREDDS servers bottleneck: – Uncompressing of data to local store for processing if not already in cache. Convert NetCDF3 datasets to NetCDF4 with internal compression applied. – Using NetCDF utility software ‘nccopy’ (-deflate flag)
2
NC3NC3 (.bz) NC4nccopyTHREDDS (@ ~30Mbs transfer rate) NetCDF4 Test 1 Granule155.0 MB7.5 MB15.5 MB~54 secs Aggregation Array: [time] [lat] [lon] Subset Array Size: [365][500][600] 1 Year of data56.0 GB3.0 GB5.6 GB5.5 hrs 6 minutes (ascii output) 4 minutes (binary output) GHRSST L4 UKMO OSTIA (6km) NC3NC3 (.bz) NC4nccopyTHREDDS (@ ~30Mbs transfer rate) NetCDF4 Test 1 Granule32.4 MB2.6 MB3.9 MB~20 secsAggregation Array: [time] [lat] [lon] Subset Array Size: [365][500][600] 1 Year of data11.4 GB1.0 GB1.4 GB2 hrs3 minutes (ascii output) 2 minutes (binary output) GHRSST NAVO K10 SST (10 km) NC3NC3 (.bz) NC4nccopyTHREDDS (@ ~30Mbs transfer rate) NetCDF4 Test 1 Granule2.9 GB120.0 MB265.0 MB~53 minsAggregation Array: [time][lat][lon] Subset Array: [20][500][600] 1 minute (ascii output) 1 minute (binary output) GHRSST L4 JPLOUROCEAN G1SST (1km)
3
GHRSST L4 UKMO OSTIA (6km) (7200x3600) (20 pixels/deg) (21 days aggr) NC3 (.bz) NC4 Subset (1) Array size: [21,301,801]7.5 mins45 secs Subset (2) Array size: [21,601,1601]8.0 mins55 secs GHRSST NAVO K10 SST (10 km) (3600x1800) (10 pixels/deg) (21 days aggr) NC3 (.bz) NC4 Subset (1) Array size: [21,151,401]2.5 mins4 secs Subset (2) Array size: [21,301,801]3.0 mins10 secs GHRSST L4 JPLOUROCEAN G1SST (1km) (36000x16000) (100 pixels/deg) (6 days aggr) NC3 (.bz) NC4 Subset (1) Array size: [6,1501,4001](too long)1 min Subset (2) Array size: [6, 3001,8001](too long)3 mins (0 o,-160 o ) (15 o,-120 o ) (0 o,-180 o ) (30 o,-100 o ) * * * * Subset (1) Subset (2) Subset Regions NetCDF3 (Bzip2) vs. NetCDF4 (Internal Compression) with THREDDS @ ~30Mbits/sec transfer rate
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.