Installing the THREDDS and Aggregation Servers ● Install and verify the Tomcat servlet engine ● Install and verify the THREDDS servlet (which also contains the Aggregation Server)

Install Tomcat ● Expand/unpack Jakarta/Tomcat > cd tutorial > tar -xzf /scr/OPeNDAP_Software/Servers/ Binary/jakarta-tomcat tar.gz This will add the jakarta-tomcat directory.

Install THREDDS ● Put the THREDDS server 'war' file into Tomcat > cp /scr/OPeNDAP_Software/Servers/Binary /thredds.war jakarta-tomcat /webapps ● Note that the jakarta-tomcat /content is empty > ls jakarta-tomcat /content

Start Tomcat ● Start Tomcat > jakarta-tomcat /bin/startup.sh ● Starting Tomcat automatically configures it and any servlets it contains ● Verify the THREDDS server installation > ls jakarta-tomcat /content ● You should see that a 'thredds' subdirectory and 'thredds.INIT' file have been created.

Verify the Installation

Verify the THREDDS Server

Why Tomcat? ● There are two dominant server types: – Those based on C/C ++ and – those based on Java. – Apache is the most commonly used C/C ++ based server. – Tomcat is the most commonly used Java based server. ● If the server that you want to use is Java based, then Tomcat is a logical choice. ● The Aggregation Server is a Java based server.

Special Servers and Services ● THREDDS catalogs ● The Aggregation server

THREDDS Catalogs ● THREDDS is an alternative to the File Server ● Like the File Server, THREDDS catalogs – Provide a way to group data granules independently from their physical location in a file system – May span several computers ● In addition, THREDDS catalogs – May have several associated access mechanisms (e.g., OPeNDAP, FTP,...) – Provide a hierarchical catalog: directory to inventory – Are stored as XML files

Validate the Catalog ● Get a catalog for the pathfinder data – Go to in your web browser – Down load and save the file pathfinder.xml to ~/tutorial/pathfinder_data/

Validate the THREDDS catalog

THREDDS Validated Response

Use the Catalog HTML Service ● THREDDS provides a service which transforms the XML catalog into HTML – The catalog service of the THREDDS server is at: – Provide your THREDDS catalog: as an argument to the THREDDS catalog service; i.e., enter the following in your browser: er.xml

Following a Link

The Aggregation Server (AS) ● What the AS can do: configurating the AS ● Hands-on

Aggregatio n ● The Aggregation Server (AS) creates virtual data sets from: – NetCDF file – OPeNDAP data sources ● The AS is configured using THREDDS catalogs – It can aggregate local NetCDF files (and serve the files individually) – It can aggregate local and remote data using OPeNDAP servers

Types of Aggregation ● Limited to Array and Grid variables ● Three types of aggregation: – JoinNew: Combine arrays using a new dimension – JoinExisting: Combine using an existing dimension – Union: For the union of existing variables

Type: Join New

Type: Join Exist ● A, B and C are aggregated. ● They each have an x, y and z dimension. ● When the AS gets a request, it knows that different values of z map to the different files, A B or C ● JoinNew and JoinExist are often used to combine arrays of time-series data.

Type: Union ● The data sets A and B contain some common and some unique variables. ● The Union aggregation combines different parameters from the data sets into one logical data set. ● Often this is used to combine tables of data.

Configuratio n The data provider: ● Defines an aggregation using a THREDDS catalog, and ● Controls the performance of the AS via the web.xml file for the servlet. – For example, setting the cache size to accommodate all the files in a JoinNew or JoinExist aggregation will improve performance.

The Aggregation Server (AS) ● What the AS can do: configuring the AS ● Hands-on

Configure the AS for the Pathfinder Data ● Download/Edit the AS configuration file ● Verify operation of the AS

Install the New Catalog ● Go to and save the file 'stennisTutorialAS.xml' to the directory: ~/tutorial/jakarta-tomcat /content/thredds/dodsC ● Change to that directory > cd ~/tutorial/jakarta-tomcat /content/thredds/dodsC and look at the files there > ls ● You should see catalogConfig.xml, catalogExample.xml and stennisTutorialAS.xml

Install the New Catalog (continued) ● Move the stennisTutorialAS.xml file to catalogConfig.xml > mv stennisTutorialAS.xml catalogConfig.xml ● The AS configuration file must be called cataloConfig.xml!

Restart Tomcat ● Restart Tomcat to reinitialize the AS > cd ~/tutorial/jakarta-tomcat > bin/shutdown.sh > bin/startup.sh ● After any change to the AS configuration file, Tomcat must be restarted.

Access the AS

The DDS Shows the Added Dimension

Exercis e ● Compare the DDS from the aggregation with one from a single pathfinder data file: – In a second browser window (or tab), type – Select one of the files and access it's DDS

Access Data ● Use the AS to get the Data Request Form ● In the form, access values for the 'dsp_band_1' variable by clicking the variable's checkbox ● Try a constraint like: time 0:1:1 lat 0:100:1023 lon 0:100:1023

Data Returned

Letting Others Know about Your Server ● The Global Change Master Directory (GCMD) ● The NVODS data set list ● THREDDS catalogs

Why Register Your Server? ● The DAP provides data access services only; it does not provide data location services. ● The DAP was designed to be used in concert with other protocols and systems which will provide lookup, location and searching capabilities.

Where Can I Register My Server? ● The Global Change Master Directory ● The NVODS Dataset List ● THREDDS catalogs ● Your own HTML pages

The GCMD ● The Global Change Master Directory (GCMD) is a registry of Earth-Science data sources. ● The GCMD: – Has been in existence for more than a decade. – Stores very complete metadata for each registered data source. – Has a team of “domain specialists” who work with providers to write the metadata records. – Provides explicit support for OPeNDAP data sources

The GCMD (continued) ● To register a data set with the GCMD, visit their home page and follow the Add to the GCMD link.

The NVODS Dataset List ● The National Virtual Ocean Data System (NVODS) maintains a list of data sets as an XML document. ● Registration is easy: – Go to the OPeNDAP home page – Click on Data Sources in the menu on the left hand side of the page. – Click on Register your server. ● The NVODS list provides almost no metadata ● It is very complete ● This list is accessible on-line at the web page through an HTML interface as well as from the ODC.

THREDDS ● You can write THREDDS catalogs ● THREDDS catalogs can be linked together ● Unidata maintains a catalog of catalogs

HTML and Google ● You write your own web pages with links to your datasets ● Let Google crawl your site. – A Google search on “DODS Salinity Data” returned about 4,000 hits, the first page listed data from the Arabian Sea Experiment, Massachusetts, Maine and IFREMER (France).

Server Registration: Conclusions ● There are a number of ways to register your server, although the DAP doesn't provide any explicit support for this activity. ● We have largely left the data location problem untouched since other groups have devoted 100% of their effort to solving that problem. ● Existing registration and search options are diverse and powerful.