1. UWG 2013 Meeting Publish-Subscribe (Datacasting)

2.  Goal  PO.DAAC to publish availability of our data  Why?  So users can subscribe and no longer have to crawl FTP Goal

3. High Level Functions: Publishing Consumers Archive Ingest Web Services & Publishing Direct Data Access High-Level Access Tools Web Portal Visualization Inventory Information Providers Data In Information In Data Out Information Out Data Providers Data Providers

4.  Datacasting  Extension of RSS  Work funded by NASA ACCESS 2005, 2009  Data providers publishes availability of new data granules via XML feed  Data consumers subscribes to feed to learn about recently available data granules Solution

5.  Datacasting enables  Down select items in feed based on multiple criteria  Temporal  Spatial  Other metadata  Automatically download selected data files  Mash up across different types of feeds  Combine newspaper articles, scientific reports, event analyses, data, etc. in one place  And much more By the way…

6. Publishing DatacastingRSS Consumers Archive Ingest Web Services & Publishing Direct Data Access High-Level Access Tools Web Portal Visualization Inventory Information Providers Data In Information In Data Out Information Out Data Providers Data Providers DatacastingRSS

7. Publish-Subscribe Sequence

8. DatacastingRSS <rss xmlns:datacasting="http://datacasting.jpl.nasa.gov/datacasting" xmlns:georss="http://www.georss.org/georss" xmlns:gml="http://www.opengis.net/gml" version="2.0"> MetOp-A ASCAT Level 2 25.0 km Ocean Surface Wind Vectors http://podaac-test.jpl.nasa.gov/dataset/ASCAT-L2-25km This dataset contains operational near-real-time Level 2 ocean surface wind vector retrievals... Wed, 27 Feb 2013 07:49:44 GMT... PODAAC-ASOP2-25X01 METOP-A/ASCAT ascat_20130226_190001_metopa_32990_eps_o_250_2101_ovw.l2.nc http://podaac-test.jpl.nasa.gov/dataset/ASCAT-L2-25km ascat_20130226_190001_metopa_32990_eps_o_250_2101_ovw.l2.nc <enclosure length="847714" type="application/x-netcdf" url="ftp://podaac-ftp.jpl.nasa.gov/allData/ascat/preview/L2/25km/2013/057/ascat_20130226_190001_metopa_32990_eps_o_250_2101_ovw.l2.nc.gz" /> Wed, 27 Feb 2013 07:49:44 GMT... -89.37 0.02 89.26 359.99 Tue, 26 Feb 2013 19:00:01 GMT Tue, 26 Feb 2013 20:41:59 GMT......

9. Link to DatacastingRSS Feed

10. Client Tool: Scripts  Scripts in Python, Perl, MATLAB, etc. can be written to download data granules as they are made available  Sample MATLAB script for downloading data granules: xml = xmlread('http://podaac-test.jpl.nasa.gov/ws/metadata/granule/?format=datacasting&datasetId=PODAAC- ASOP2-25X01'); enclosures = xml.getElementsByTagName('enclosure'); for i = 0:enclosures.getLength-1 url = char(enclosures.item(i).getAttribute('url')); [pathstr, name, ext] = fileparts(url); filename = [name, ext]; if exist(filename, 'file') disp(['File exists: ' filename]); else urlwrite(url,filename); disp(['Downloaded: ' filename]); end exit;

11.  Perl tool for subscribing to feeds and downloading and maintaining files on disk Client Tool: Data::Downloader % dado config init --filename./ascat-l2-25km.conf % dado feeds refresh % dado files download % dado config init --filename./ascat-l2-25km.conf % dado feeds refresh % dado files download Terminal

12. Client Tool: Datacasting Feed Reader Subscribe to DatacastingRSS Filter feed items Download data Create mash up