1. Other Services in Hyrax

2. Other Services in Hyrax
HTML: Web-page form interface
ASCII: Data responses
Info: Metadata as formatted HTML
Catalogs: Directory and THREDDS responses
GridFTP: Direct access to the BES

3. HTML Form Interface
The Form interface provides a uniform way to access data
It’s quite primitive because it must work for all data - even data with no semantic metadata
When a constraint is formed with this interface, the subsetting must be done using only syntactic information like the physical size of an array.
This interface is part of every Hyrax and most other DAP servers, although there are some slight variations among different implementations
It provides a way to view the structure of the data and form a constraint so that a complete URL can be cut and pasted into another application

4. ASCII
ASCII is the lowest common denominator for data since many programs can read it
Hyrax has a handler that transforms the DAP data response (binary data with structural metadata) into a CSV ASCII response.
You can use a spreadsheet to read data
Try it!

5. Find some data in the form interface
Build a constraint with the form (remember
this is ASCII we’re talking about, so data
volume is a consideration!) and then hit
‘Get ASCII’

6. FireFox puts the new data into a tab, other
browsers open a separate window…

7. Try it with a spreadsheet; type (or paste)
the URL into the ‘Location’ box. Then hit
‘OK.’ This example is using Kspreadsheet
from the Virtual machine; Excel works too.

8. Data come back in the spreadsheet cells.
You may be prompted to supply information
Depending on the spreadshhet

9. Info
The DDS (structural) and DAS (semantic) metadata responses can be combined and returned as an HTML page
Add the extension ‘.info’ to the DAP URL
Try it…

10. Catalogs OPeNDAP directory is a kind of catalog
Conceptually based on the HTML directory pages
Provides a click-to-navigate interface
Different servers implement it differently and this has created problems with uniformity for clients
THREDDS provides a framework for cataloging data sources
Provides a way to list several different access protocols
Listings iin XML

11. Catalogs, cont.
To get the OPeNDAP directory listing from an old server, end the URL in a slash (/)
To get this from Hyrax, use either a slash or ‘contents.html’
To get a THREDDS catalog from any DAP server, use ‘catalog.xml’ for the XML listing or ‘catalog.html’ for the HTML version
The HTML THREDDS catalog supports a click-to-navigate interface like the old OPeNDAP directory
Try it

12. THREDDS examples Switch Hyrax to the THREDDS HTML view:
Choose the HTML view

13. The THREDDS HTML view
The top-level THREDDS catalog on our test server defines a single data root directory (SVN Test Data Archive)
This illustrates how THREDDS can be used to control the view of data presented by the server
Use ‘catalog.xml’ in place of ‘catalog.html’ to get the catalog data in an XML document.

14. Traverse the links to find data

15. THREDDS data set page
THREDDS catalogs can list more than one access mechanism - here we see on the DAP, but WCS, WMS, et c., are other possibilities

16. GridFTP Hyrax does not actually support GridFTP but the BES does.
The BES was originally developed as a DAP-capable server for the Earth Systems Grid project (now ESG II)
The BES can be run standalone in appropriate environments and the GridFTP protocol can be used to connect to it.
Using GridFTP, DAP requests can be made and DAP responses transferred to ESG clients.

17. Numerical Grid Computations within BES
CSAC - Community Specro-polarimetric Analysis Center
Two part problem;
Compute the so called Stokes vector for a given vector parameter space P using a model F(x,y,λ; P)
Correct P to improve the fit of the theoretical signal to the observed signal using non-linear optimization
The purpose is to determine a vector parameter space for which the model produces the best fit in the Least Square sense.
The complete problem is solved with a Numerical Library called MERLIN
Very computational Intensive, once took up to 6 hours now takes 16 minutes.

18. High Performance Solution
Create multiple versions of the computing node waiting for data to crunch
Create a central place (controller) to browse spectra- polarimetry data inventories and launch inversions upon user’s request.
Let the controller decide how to distribute the problem.
Requires multiple identical processes waiting to be called for duty.

19. Merlin Inversion Engines
Distributing the Work
GRID
BES Server 1
Client
BES Server 2
APACHE
TOMCAT
MODULE . .
Linux Dualcore Xeon 64-bit processor with 10 Terabyte RAID
topology
From the previous slide … this is accomplished using the BES on distributed machines. Each BES dynamically loads the Merlin Inversion Engine and module to work on a particular slice of the data. Each of these BES servers is running on a machine with 2 64-bit AMD Opteron processors. They can also run the BES on MacPro EightCore machines and include them in the topology of the system.
BES Server 10
Merlin Inversion Engines
20 64-bit AMD Opteron processors running on Solaris 10. Can also plug in MacPro EightCore machines