Science Review Panel Meeting Biosphere 2, Tucson, AZ - January 4-5, 2011 Vegetation Phenology and Vegetation Index Products from Multiple Long Term Satellite Data Records ESDRs Distribution and User Support – Miscellaneous Topics Calli Jenkerson ERT, USGS EROS Center NASA MEASURES #NNX08AT05A

U.S. Department of the Interior U.S. Geological Survey MEaSUREs VIP Miscellaneous Topics January 2011 Calli Jenkerson, ERT USGS EROS Center

Miscellaneous Topics  Beta ESDR File Transfer  Metrics  GMCD Entry  ESDR Metadata  Long Term Archive  ESDR Access MEASURES VIP ESDRs Science Review Panel -3-

Beta ESDR File Transfer  Expectations  Test data set characteristics  Infrastructure  Schedule  Current test results  Plans  Questions MEASURES VIP ESDRs Science Review Panel -4-

Beta ESDR File Transfer  ESDR production at University of Arizona  Storage and distribution at USGS EROS  Expected final totals:  Number of Files <=87,656  Uncompressed Volume ~5.7 TB  Compressed Volume ~1.5 TB  based on 30 year record  2 daily products (VI, Phenology)  4 data sets (AVHRR2, AVHRR3, MODIS, SPOT) MEASURES VIP ESDRs Science Review Panel -5-

Beta ESDR File Transfer Data SetFile Format Time Range# Files Volume (GB) AVHRR V2 (from LTDR2)HDF AVHRR V3 (from LTDR3)HDF MOD09CMGHDF SPOT_CMGHDF TOTAL1,  Beta collections available as of June 2010 MEASURES VIP ESDRs Science Review Panel -6-

Beta ESDR File Transfer  Utilize FTP  UofA pushes to public staging area at EROS  ~2 TB disk  Unique VIP user  EROS moves files across firewall to VIP storage  ~6 TB disk MEASURES VIP ESDRs Science Review Panel -7-

Beta ESDR File Transfer  Schedule  Roughly once every year  July 20 - August 3, 2009  June 7-11, 2010  June 2011  December 2011?  June 2012?  As data are available MEASURES VIP ESDRs Science Review Panel -8-

Beta ESDR File Transfer  Test results  2009 Transfer Test  Alpha data collection  Pushed 230 GB  Obstructed by limited connections (5) to staging area  <1 Mbs average throughput MEASURES VIP ESDRs Science Review Panel -9-

Beta ESDR File Transfer  2010 Transfer Test  Beta data collection  Pushed 309 GB  Initiated multiple sessions to maximize number of connections to staging area  ~1.7 Mbs continuous throughput Transfer StatisticJune 7-11, 2010 Total FTP Transactions 1243 Total Transaction TimeJune 7 12:07:09 CST – June 11 17:20:19 Total Uploaded Gigabytes GB Total Continuous Upload Time Hours61.66 Hr Average Continuous Upload Rate Kilobytes per second (3 connections) 1692 Kbs MEASURES VIP ESDRs Science Review Panel -10-

Beta ESDR File Transfer  Perform transfers to the EROS VIP environment as ESDR data are available  Not likely to explore performance enhancement  Timeliness of public product availability is not currently an issue for this historical collection  Future plans to archive ESDRs in LP DAAC will not use the beta transfer model for ingest MEASURES VIP ESDRs Science Review Panel -11-

Beta ESDR File Transfer  Questions? MEASURES VIP ESDRs Science Review Panel -12-

Metrics  Expectations  Tools  Record of Report  ESDSWG-MPARWG  Plans  Questions MEASURES VIP ESDRs Science Review Panel -13-

Metrics  Report project metrics as appropriate to mark progress and results  Ingest metrics: volume and number of files stored in EROS VIP systems  Distribution metrics: Web site hits, distinct users, services invoked, file downloads  User metrics: registered users, user services contacts, method of contact, subject of inquiry, escalation level, response time MEASURES VIP ESDRs Science Review Panel -14-

Metrics  Earth Science Data Systems (ESDS) Metrics Collection Tool (MCT)  Established for REASoN projects  Carried over for MEaSUREs  Earth Science Data and Information System (EOSDIS) Metrics System (EMS)  Established for Earth Observing System (EOS) data centers  Potential for implementation of EMS-MEaSUREs MEASURES VIP ESDRs Science Review Panel -15-

Metrics  VIP Metrics Reporting Record:  ESDS MCT entry  2009 July  2010 July, September, November  Ingest and Distribution metrics only  Describes primarily test community activity  User metrics will be added as Project-specific after public release of ESDRs MEASURES VIP ESDRs Science Review Panel -16-

Metrics VIP Client Ingest Metrics (since June 2009) Data SetLayers in Client Time RangeTotal # Files Total GB AVHRRV2 (from LTDR2) NDVI, EVI (daily) AVHRRV3 (from LTDR3) NDVI, EVI (daily) MOD09CMGNDVI, EVI (daily) SPOT_CMGNDVI (daily)98 37 MOD13C2NDVI, EVI (monthly)114 2 eMODIS CONUS NDVI (weekly) TOTAL1,758 files 491 GB MEASURES VIP ESDRs Science Review Panel -17-

Metrics VIP Client Distribution Metrics (since June 2009) 38Web Page Visitors 43Distinct Users 31Domain.gov 12Domain.edu 49Products Delivered 5.13GB Delivered 18,547Services Provided MEASURES VIP ESDRs Science Review Panel -18-

Metrics  Solicitation requirement to participate in the Earth Science Data System Working Groups (ESDSWG)  VIP has interested parties in Standards and Interfaces, Technology Infusion, Architecture and Reuse Working Groups  VIP is a contributing member of the Metrics Planning and Reporting Working Group (MPARWG) MEASURES VIP ESDRs Science Review Panel -19-

Metrics  Attended ESDSWG Meetings 2009, 2010  Participated in ESDSWG MPARWG Telecons 2009, 2010  Contributions:  Experiences  Goals  Feedback  Recommendations MEASURES VIP ESDRs Science Review Panel -20-

Metrics  Continue membership in MPARWG  Respond as needed to MPARWG chairs  Continue quarterly reporting in ESDS MCT  Refine metrics collection methods to more precisely express services rendered MEASURES VIP ESDRs Science Review Panel -21-

Metrics  Questions? MEASURES VIP ESDRs Science Review Panel -22-

GCMD Entry  Global Change Master Directory  NASA sponsored repository for Earth and environmental science information  Data sets  Data services  Intended to consolidate such things into a one-stop-find-everything-site MEASURES VIP ESDRs Science Review Panel -23-

GCMD Entry  Uses keywords to navigate through a topical database  Every NASA product is described by  Entry ID  Summary  Geographic Coverage  Data Set Citation  Temporal Coverage  Location / Science Keywords  ISO Topic Category  Platform  Instrument  Originating Center  Data Center  Distribution  Contact  Pubs/Refs MEASURES VIP ESDRs Science Review Panel -24-

GCMD Entry  Adding a data set is a simple matter of completing a web form  Approval for addition is at GCMD discretion  Preference given to data sets with:  Compliant metadata (ECS or ISO)  Relevance to at least one of about 1,500 science topics MEASURES VIP ESDRs Science Review Panel -25-

GCMD Entry Sample GCMD Entry MEASURES VIP ESDRs Science Review Panel -26-