Other Services in Hyrax

Other Services in Hyrax HTML: Web-page form interface ASCII: Data responses Info: Metadata as formatted HTML Catalogs: Directory and THREDDS responses GridFTP: Direct access to the BES

HTML Form Interface The Form interface provides a uniform way to access data It’s quite primitive because it must work for all data - even data with no semantic metadata When a constraint is formed with this interface, the subsetting must be done using only syntactic information like the physical size of an array. This interface is part of every Hyrax and most other DAP servers, although there are some slight variations among different implementations It provides a way to view the structure of the data and form a constraint so that a complete URL can be cut and pasted into another application

ASCII ASCII is the lowest common denominator for data since many programs can read it Hyrax has a handler that transforms the DAP data response (binary data with structural metadata) into a CSV ASCII response. You can use a spreadsheet to read data Try it!

Find some data in the form interface Build a constraint with the form (remember this is ASCII we’re talking about, so data volume is a consideration!) and then hit ‘Get ASCII’

FireFox puts the new data into a tab, other browsers open a separate window…

Try it with a spreadsheet; type (or paste) the URL into the ‘Location’ box. Then hit ‘OK.’ This example is using Kspreadsheet from the Virtual machine; Excel works too.

Data come back in the spreadsheet cells. You may be prompted to supply information Depending on the spreadshhet

Info The DDS (structural) and DAS (semantic) metadata responses can be combined and returned as an HTML page Add the extension ‘.info’ to the DAP URL Try it…

Catalogs OPeNDAP directory is a kind of catalog Conceptually based on the HTML directory pages Provides a click-to-navigate interface Different servers implement it differently and this has created problems with uniformity for clients THREDDS provides a framework for cataloging data sources Provides a way to list several different access protocols Listings iin XML

Catalogs, cont. To get the OPeNDAP directory listing from an old server, end the URL in a slash (/) To get this from Hyrax, use either a slash or ‘contents.html’ To get a THREDDS catalog from any DAP server, use ‘catalog.xml’ for the XML listing or ‘catalog.html’ for the HTML version The HTML THREDDS catalog supports a click-to-navigate interface like the old OPeNDAP directory Try it

THREDDS examples Switch Hyrax to the THREDDS HTML view: Choose the HTML view

The THREDDS HTML view The top-level THREDDS catalog on our test server defines a single data root directory (SVN Test Data Archive) This illustrates how THREDDS can be used to control the view of data presented by the server Use ‘catalog.xml’ in place of ‘catalog.html’ to get the catalog data in an XML document.

Traverse the links to find data

THREDDS data set page THREDDS catalogs can list more than one access mechanism - here we see on the DAP, but WCS, WMS, et c., are other possibilities

GridFTP Hyrax does not actually support GridFTP but the BES does. The BES was originally developed as a DAP-capable server for the Earth Systems Grid project (now ESG II) The BES can be run standalone in appropriate environments and the GridFTP protocol can be used to connect to it. Using GridFTP, DAP requests can be made and DAP responses transferred to ESG clients.

Numerical Grid Computations within BES CSAC - Community Specro-polarimetric Analysis Center Two part problem; Compute the so called Stokes vector for a given vector parameter space P using a model F(x,y,λ; P) Correct P to improve the fit of the theoretical signal to the observed signal using non-linear optimization The purpose is to determine a vector parameter space for which the model produces the best fit in the Least Square sense. The complete problem is solved with a Numerical Library called MERLIN Very computational Intensive, once took up to 6 hours now takes 16 minutes.

High Performance Solution Create multiple versions of the computing node waiting for data to crunch Create a central place (controller) to browse spectra- polarimetry data inventories and launch inversions upon user’s request. Let the controller decide how to distribute the problem. Requires multiple identical processes waiting to be called for duty.

Merlin Inversion Engines Distributing the Work GRID BES Server 1 Client BES Server 2 APACHE TOMCAT MODULE . . Linux Dualcore Xeon 64-bit processor with 10 Terabyte RAID topology From the previous slide … this is accomplished using the BES on distributed machines. Each BES dynamically loads the Merlin Inversion Engine and module to work on a particular slice of the data. Each of these BES servers is running on a machine with 2 64-bit AMD Opteron processors. They can also run the BES on MacPro EightCore machines and include them in the topology of the system. BES Server 10 Merlin Inversion Engines 20 64-bit AMD Opteron processors running on Solaris 10. Can also plug in MacPro EightCore machines