The Land, Atmosphere Near-real-time Capability for EOS (LANCE) Ed Masuoka Code GESDIS Combined Branch Meeting

What is LANCE? Building on existing EOSDIS elements (MODAPS/LAADS; OMI SIPS, GES DISC), provides data from MODIS, OMI, AIRS, MLS, and AMSR-E instruments in near real-time (< 3 hours from observation) Utilizes PGE Code from Standard Science Products, but: –All products generated with a “nrt” extension to distinguish them from standard products –Requirements for ancillary data inputs have been relaxed (e.g., MODIS cloud mask, land surface reflectance, aerosols, and clouds) High operational availability achieved by: –Extra compute and ftp servers –Redundant power sources –Redundant production strings on separate networks (in progress) Applications of LANCE data include: –Numerical weather & climate prediction/forecasting –Monitoring of Natural Hazards –Disaster Relief 2 –Agriculture –Air quality –Homeland Security

LANCE Architecture GES DISC MODAPS OMI SIPS AIRS MLS MODIS AMSR-E OMI EDOS Level Zero Processing Facility Format: RBD Format: S-PDS GMAO SPoRT NRL UMBC USGS Others Users Protocols: HTTP/FTP RBD: Rate Buffered Data S-PDS: Session Based Production Data Set Format: RBD Operating LANCE Element Observation to availability latency: min. Transfer latency: bandwidth dependent Data Available: -AIRS L1 and L2 -MLS L2 Protocols: HTTP/FTP LANCE Architecture Data Available: -MODIS L1, L2, L2G and selected L3 -AMSR-E L1, L2 and L3 Data Available: -OMI L1B and L2 LANCE Element Under Development

LANCE Product Categories 4 InstrumentProduct CategoriesAverage LatencyStatus AIRS Radiances, Temperature and Moisture Profiles, Clouds and Trace Gases, Ephemeris/Attitude 75 – 140 minutesOperating AMSR-E L1A Raw Data, Soil Moisture, Snow Water Equivalent, Temperature N/AUnder Development MLS Ozone, Temperature, Ephemeris/Attitude Data 75 – 140 minutesOperating MODIS Radiances, Cloud/Aerosols, Water Vapor, Fire, Snow Cover, Sea Ice, Land Surface Reflectance, Ephemeris/attitude 90 – 145 minutes (L1 and L2 products) Operating OMI Ozone, Clouds, Aerosols, Trace Gases 100 – 165 minutesOperating MODIS also generates daily L2G and L3 Climate Modeling Grid for surface reflectance: these products have a latency of hours

5 LANCE-MODIS Architecture Database server modnrt1db Compute servers nrt1min Gigabit Ethernet switch Scheduling host modnrt1 RAID 6 Storage modnrt1 High Operational Availability achieved by: 1. Redundant power: 2 sources of conditioned power (UPS 9 and 10) for each component 2. Extra compute and ftp servers (Dell 2950) 15 servers (2 can fail w/o impact to production) 2 nd ftp server (currently manual fail-over) 3. Redundant production strings on separate networks ftp server nrt1 PDR server modpdr01 nrt2 2 nd string to deployed in Spring, 2010 DOORS Science and Engineering Network EMSnet Mission Network Browse Server PDR server

LANCE-MODIS Operational Data Flow 6

7 Example Latencies for LANCE-MODIS and Standard Forward Products Standard ProcessingNRT Processing (typical) Product CategoryTerra(hrs)Aqua(hrs)Terra/Aqua (hrs) L1/Cloud Mask L2 Snow L2 Sea Ice L2 Fire L2 Clouds L2 Aerosol L2 LSR

8 Comparison of MODIS Standard and NRT Products Land Surface Reflectance over Midwest. Slightly more haze is visible in NRT view West of Great Lakes. Standard ProcessingNear-Real-Time Processing

9 Comparison of MODIS Standard and NRT Products Cloud top temperature over the Midwest. Thin clouds over lake Superior show sensitivity to GDAS ancillary data Standard ProcessingNear-Real-Time Processing

10 Comparison of MODIS Standard and NRT Products For LSR the Match is the percentage of NRT data with <1% error margin when compared to the operational Collection 5 codes For Snow, Sea Ice, and Fire, the Match is the exact pixel to pixel match between NRT and operational Collection 5 codes Omission and Commission errors are computed as a percentage of the snow,sea ice, fire in the operational Collection 5 products Short Name Science Data Match (% Global) #pixel (%Global) Omission Error #pixel (%) Commission Error #pixel (%) MOD09LSR-B198.94N/A MOD09LSR-B299.12N/A MOD09LSR-B399.32N/A MOD09LSR-B499.16N/A MOD10L2 Snow (2.1%) (0.13%) (0.14%) MOD29L2 Sea Ice (5.5%) 8383 (0.06%) (0.1%) MOD14L2 Fire (0%) 2 (0.06%) 3 (0.09%)

AIRS Level1-B VIS/NIR Quick Browse Image Near-Real Time AIRS L1 Product Quality Standard Product 11

AIRS NRT Data Product Quality Same PGEs used to produce Standard Data Products used in NRT processing NRT L1 data nearly identical to L1 Standard Product –Some minor differences in geolocation due to use of predicted ephemeris –Occasional minor differences in granules along contact session boundaries (~0.1 K) –Processing option: retain data at session boundaries to ensure data overlap when generating L1B –To date data latency has superseded minor quality improvement NRT L2 products compare favorably to the L2 Standard Data Products. –NRT system uses surface climatology in place of dynamic ancillary data (GFS) as input to science data processing –Surface Air Temperature differences within 1 K for areas where sfc pressure is approximated by climatology, otherwise differences can be as much as 10 K in selected areas –Total Water Vapor differences within 5% over most of the globe; as much as % where large surface pressure differences are observed, particularly at high latitudes

13 LANCE-MODIS Data Access and Services LANCE-MODIS Web Site –URL: –Provides access to a wide variety of information including; MODIS ftp site, registration, latencies, metrics, PGE versions and production rules, data outages, and access to other LANCE systems –Provides access to land and atmospheres browse products  7 days of browse products are maintained in rolling archive User Registration –Registration is required before data may be accessed –Present registration process is temporary pending the development of an EOSDIS-wide user registration system –URL: User Data Access –Subscriptions  Provide data directly as available (push) –Direct access to public  Download from directories (pull)  Hostname: nrt1.modaps.eosdis.nasa.gov; Directory Path: allData/1/esdtx/year/dataday  7 days of data products are archived in a rolling archive

LANCE-AIRS/MLS User Access and Data Services 5 day rolling archive available FTP –Pull (anonymous, subscription) –Push (subscription) Near Real Time Data Web Site – Currently no format conversion or subsetting offered for NRT data products Spatially qualified subscriptions available Web Map Service (WMS) offered for AIRS NRT products –L1 visible radiances – CO concentration –SO2 indicator

15 Present Users of the LANCE-MODIS System MODIS Rapid Response System, NASA/GSFC Ocean Color Data Processing System, NASA/GSFC Climate and Radiation Branch, NASA/GSFC Foreign Agricultural Service University of Wisconsin Direct Broadcast Laboratory, NASA/GSFC Naval Research Laboratory, Monterey GES DISC, NASA/GSFC FNMOC, Monterey USGS, Sioux Falls University of Maryland, Baltimore University of Maryland, College Park Battelle Laboratory

U.S. Department of the Interior U.S. Geological Survey eMODIS System USGS/EROS Terra MODIS LANCE USGS FEWS NET Processing System (stack, smooth, create products) EDOS MODIS L0 Data T+3hrs T+9hrs November 2009 T+12hrs EW&EM/FEWS NET Desired/required 24 hour or less data delivery schedule FEWS NET Decision Support System (USAID FFP) MODIS L2 Data Example Application: eMODIS Africa Expedited Product Flow EW&EM NDVI and Anomaly Maps Input Data Deadline: Tuesday 10:00 a.m. FEWS Impact Assessment Maps

U.S. Department of the Interior U.S. Geological Survey 17 Proposed Modifications to LANCE-MODIS Following the LANCE Workshop in December, the following changes to LANCE-MODIS are being planned/considered: –Addition of a redundant system using different power and on a different network –Adding a Land Surface Reflectance product –Adding the AMSR-E products –Adding a variety of sub-setting capabilities –Adding re-projection and mosaicing capabilities –Adding a variety of data formats (e.g. GeoTiff) –Integration of the Rapid Response System with LANCE-MODIS –Generate products derived from Land Surface Temperature to support the Flood Mapping program

18 Backup

19 NRT Products from LANCE-MODIS Short-namePGE DescriptionProduct Description MxD01 L1A Raw Radiances and GeolocationMODIS Raw Radiances in Counts 5-Min L1A Swath MxD03 L1A Raw Radiances and GeolocationMODIS Geolocation Fields 5-Min Swath, 1km MxD021KM L1B CalibrationMODIS Calibrated Radiances 5-Min L1B Swath 1km MxD02HKM L1B CalibrationMODIS Calibrated Radiances 5-Min L1B Swath 500m MxD02QKM L1B CalibrationMODIS Calibrated Radiances 5-Min L1B Swath 250m MxD07_L2 L2 Cloud Masks/Atmospheric Profiles MODIS Temperature and Water Vapor Profiles 5-Min L2 MxD35_L2 L2 Cloud Masks/Atmospheric Profiles MODIS Cloud Mask and Spectral Test Results 5-Min L2 MxD04_L2 L2 Atmosphere MODIS Aerosol 5-Min L2 Swath 10km MxD05_L2 L2 Atmosphere MODIS Total Precipitable Water Vapor 5-Min L2 Swath 1km MxD06_L2 L2 Clouds MODIS Clouds 5-Min L2 Swath 1km and 5km MxD10_L2 L2 Snow Cover MODIS Snow Cover 5-Min L2 Swath 500m MxD10L2C L2 Snow Cover MODIS Coarse Snow Cover 5-Min L2 Swath 5km MxD29 L2 Sea Ice MODIS Sea Ice Extent 5-Min L2 Swath 1km MxD29L2C L2 Sea Ice MODIS Coarse Sea Ice Extent 5-Min L2 Swath 5km

20 NRT Products from LANCE-MODIS Short-namePGE DescriptionProduct Description MxD09 Land Surface Reflectance MODIS Surface Reflectance 5-Min L2 Swath 250m, 500m and 1km MxD09CMA/M xD09CMG L3 Daily Land Surface Reflectance MODIS Surface Reflectance L deg Tile Climate Modeling Grid MxD09GST/G HK/GQK L2G Land Surface Reflectance MODIS L2G Daily, tiled 1km/500m/250m surface reflectance MxD09GA/GQ L2G Land Surface Reflectance MODIS Light L2G Daily, tiled 500m and 1km/250m MxDTBGD L2G Land Surface Reflectance MODIS L2G Daily, tiled daytime thermal bands 1km MxD14 L2 Thermal Anomalies/Fire MODIS Thermal Anomalies/Fire 5-Min L2 Swath 1km MxD00F Session-Based L0 PDS file splitter Session-Based L0 PDS file, 5-min Swath MxD00S Session-Based L0 PDS Original Session-Based L0 file from EDOS MxD02SSH L1B Calibration 5 km subsampled L1B MYDGB0 Aqua Attitude and Ephemeris Aqua Attitude and Ephemeris in Session L0 format

1 All datatypes appended with _NRT to denote difference from production datatypes AIRABRAD_NRTAqua/AIRS Level 1B AMSU-A1 and AMSU-A2 Combined Geolocated and Calibrated Brightness Temperatures AIRIBQAP_NRTAqua/AIRS Level 1B Infrared (IR) Quality Assurance Subset AIRIBRAD_NRTAqua/AIRS Level 1B Infrared (IR) Geolocated and Calibrated Radiances AIRVBQAP_NRTAqua/AIRS Level 1B Visible/Near Infrared (VIS/NIR) Quality Assurance Subset AIRVBRAD_NRTAqua/AIRS Level 1B Visible/Near Infrared (VIS/NIR) Geolocated and Calibrated Radiances AIRI2CCF_NRTAqua/AIRS FINAL AIRS Level 2 Cloud Clear Radiance Product AIRX2RET_NRTAqua/AIRS Level 2 Standard Final Retrieval Product AIRX2SUP_NRTAqua/AIRS Level 2 Support Product PM1EPHND_NRTPreprocessed Aqua Platform Definitive Ephemeris Data from FDS in Native format PM1ATTNR_NRTPreprocessed Aqua Platform Refined Attitude Data in Native format ML2O3_NRTAura/MLS L2 Ozone (O3) Mixing Ratio ML2T_NRTAura/MLS L2 Temperature AURATTN_NRTPreprocessed Aura platform attitude product AUREPHMN_NRTPreprocessed Aura platform ephemeris product NRT Datatypes available from GES DISC

22 Product Dependencies on Ancillary Data The L2 codes search back for ancillary data a maximum of the times indicated on the arrows The L2 Snow, Sea Ice, and Fire do not use ancillary data products Cloud MaskL2 Aerosols L2 LSRL2 Cloud GFSREYNSSTSeaIceNISEGDASTOAST 60h 8d 120h 14d 7d 72h 84h 14d 21d 120h(optional) 36h Unchanged From Standard PGE Ancillary Data