MODLAND Volumes and Loads Status MODIS Land Science Team Workshop July 15, 2003 Robert Wolfe MODIS Land Team Support Group NASA GSFC Code 922, Raytheon ITSS

July 15, 2003 – MODLAND Workshop – R. Wolfe 2 Land – EDC DAAC 2/96 baseline was 154GB/day 10/99 baseline is 330GB/day (with combined Aqua/Terra products) Current Collection 4 reprocessing volume is 310GB/day (no combined products) Land volume will be 240GB/day in Collection 5 reprocessing (reduced L2G volume and other changes) in Fall 2004

July 15, 2003 – MODLAND Workshop – R. Wolfe 3 Land – NSIDC DAAC 2/96 baseline was 13GB/day 10/99 baseline is 40GB/day Current Collection 4 reprocessing volume is 15GB/day Snow and Sea Ice product volume will be 25GB/day with Collection 5 reprocessing in Fall 2004

July 15, 2003 – MODLAND Workshop – R. Wolfe 4 Proposed C5 Volume Changes Objective to estimate: –how to further reduce the size of the L2G products, and –start generating 500m data for a number of products that are currently only produced at 1km Changes implemented for a Collection Version 5 reprocessing –likely to begin next fall –will mean changes for downstream software Compression could also help with throughput –will be considered separately Did not consider: –changing the compositing period for 8-day products –eliminating any products (except when replaced by 500m products) –any Land products archived at NSIDC (only EDC products)

July 15, 2003 – MODLAND Workshop – R. Wolfe 5 L2G Option A Reduce the number of L2G products archived from 7 to 2 per tile –incorporate the pointer and geoangle products into the surface reflectance products Savings from: –removing the "orb_cov" field –eliminating the "line" and "sample" fields –removing overlap between scans Impact: reduces the L2G products to less than half the current size –From 251 GB/day to 120 GB/day –But … MOD09HK is now a more complicated product because it has both a 1km and 500m grid within the same file

July 15, 2003 – MODLAND Workshop – R. Wolfe 6 L2G Option B Turn the L2G product into a "AGG" product –eliminate the need to have both a daily L2G surface reflectance and a daily Aggregated surface reflectance Same criteria for selecting observations as in the current MODAGG code will be used –maximal angular distribution of the good observations; no more than 4 orbits/day Non-compact (full) form is used –each layer is stored as a single 2-dimensional array –savings from reducing the need for storing the links between the different resolutions –additional space is needed to go to the non-compact form, and to add the texture field Impact: the L2G products to about 2/3 of current size –From 251 GB/day to 165 GB/day (vs. 120 GB/day for Option A)

July 15, 2003 – MODLAND Workshop – R. Wolfe 7 Generating four current 1km products at 500m Candidate 1km products: BRDF/Albedo, LAI/FPAR, PSN/NPP and Land Cover Need an intermediate AGG/TEX at 500m or L2G option B Current candidate 1km product size: 6.5 GB/day –excludes 1km AGG/TEX (current size 51.4 GB/day, reduced volume: 25.7 GB/day) Estimated new 500m product size: 17.8 GB/day –excludes 500m AGG/TEX (reduced volume: GB/day) Easily fits with L2G Option B or for L2G Option A if AGG/TEX is not archived

July 15, 2003 – MODLAND Workshop – R. Wolfe 8 Summary L2G Option Add 500m Keep 1km Archive Agg/Tex New volume archived (GB/day) Savings from current allocation of 330 GB/day A1*Yes No19541% A2YesNo 18843% A3Yes 3232% A4YesNoYes29112% B1*Yes No24027% B2YesNo 23329%

July 15, 2003 – MODLAND Workshop – R. Wolfe 9 Volume Drivers Main drivers –Extending Terra and Aqua mission lifetimes three years (from 5 to 8+ years) –Volume growth from ’96 baseline –Archive management issues Previous collection must be kept for 6 months after current one – tape reuse is more difficult than anticipated Solution: Technology refresh –Anticipated for about this time –New drives write 3 times as much data on tapes, current tapes are being reused

July 15, 2003 – MODLAND Workshop – R. Wolfe 10 Strawman MODLAD Volume Recommendations Reduce size of L2G products using Option A or B and start producing new 500m products (reduces volume by ≥ 27%) Implement internal HDF compression where feasible –Must consider any impact on end-user analysis tools Perform technology upgrades to keep tape archive current Plan for C5 reprocessing at 5.6X (means reprocessing time is about 1 year) – by reducing overall size (including oceans) the bottleneck will be Level 1 processing Increase size of data-pools and tie data-pools to ordering interface (so data in pool doesn’t need to be staged from tape) Work with land community to better understand product usefulness