Imaging Node Meeting Atlas II Status and Plans August 2, 2006

In the Beginning…Atlas  Original Atlas was CGI-BIN based  Perl  Usually one mission per application  Though code became fairly multi-mission in flavor  Copy and tweak  Perl::DBI interface to various databases  Decision made to make completely multi-mission and augment with OODT software  Always knew this was necessary, but other priorities usurped development  MER Archive support allowed for developer focus  Original Atlas was CGI-BIN based  Perl  Usually one mission per application  Though code became fairly multi-mission in flavor  Copy and tweak  Perl::DBI interface to various databases  Decision made to make completely multi-mission and augment with OODT software  Always knew this was necessary, but other priorities usurped development  MER Archive support allowed for developer focus

Atlas II and OODT  Atlas II takes much experience learned from the original Atlas with the following changes:  Split into distinct layers and functions (at various levels of readiness)  Web Service (use of OODT here)  Performs searches across all data sets within the system. System can be distributed. As a WS, this part is intended for machine/code use.  User Interface (aka Forms)  The human useable part that utilize the others  Processing  Calibration, mosaic, geometry, etc…  Retrieval (use of OODT here)  Data download  Shopping cart style bulk ordering  Atlas II takes much experience learned from the original Atlas with the following changes:  Split into distinct layers and functions (at various levels of readiness)  Web Service (use of OODT here)  Performs searches across all data sets within the system. System can be distributed. As a WS, this part is intended for machine/code use.  User Interface (aka Forms)  The human useable part that utilize the others  Processing  Calibration, mosaic, geometry, etc…  Retrieval (use of OODT here)  Data download  Shopping cart style bulk ordering

Atlas II Web Service and OODT  The Atlas II Web Service performs the meta-data search functions of the Atlas II system. As a distributed system it utilizes (at least) two ways to access meta-data:  OODT DIS query via the OODT Profile Server  SQL  Both allowed so that we can talk to most repositories in the Imaging Node domain.  Some facilities not running OODT, but allow SQL  Some facilities don’t allow SQL, but are willing to run OODT Profile Servers  We’re kinda hosed of neither is allowed…  The Atlas II Web Service performs the meta-data search functions of the Atlas II system. As a distributed system it utilizes (at least) two ways to access meta-data:  OODT DIS query via the OODT Profile Server  SQL  Both allowed so that we can talk to most repositories in the Imaging Node domain.  Some facilities not running OODT, but allow SQL  Some facilities don’t allow SQL, but are willing to run OODT Profile Servers  We’re kinda hosed of neither is allowed…

Atlas II Web Service and OODT (2)  Imaging node currently interacts with several database and schema configurations:  MySQL (IN, UofA, probably everybody else on the planet ;-))  Sybase (e.g. MIPL, Geo)  Oracle (e.g. USGS)  Imaging node currently interacts with several database and schema configurations:  MySQL (IN, UofA, probably everybody else on the planet ;-))  Sybase (e.g. MIPL, Geo)  Oracle (e.g. USGS)

Atlas II Web Service and OODT (3)  Why not just use OODT Profile searches directly?  OODT searches syntax, not semantics  Is longitude East positive?  Is projection planetocentric? Planetodetic?  What is the equivalence of the Green filter on VGR to the wavelength of filter 3 on GLL?  What is the SCET of FDS ?  These are the kinds of things that Atlas II worries about. Once the user-perspective query is translated, it is shipped off the the various OODT Profile servers configured for use.  Why not just use OODT Profile searches directly?  OODT searches syntax, not semantics  Is longitude East positive?  Is projection planetocentric? Planetodetic?  What is the equivalence of the Green filter on VGR to the wavelength of filter 3 on GLL?  What is the SCET of FDS ?  These are the kinds of things that Atlas II worries about. Once the user-perspective query is translated, it is shipped off the the various OODT Profile servers configured for use.

OODT Profile Server configuration  So, how do you know how to query a meta-data collection? Especially one that is not local?  Profile Servers have an XML configuration file that Atlas II uses (and augments with its own spicings)  Mapping of schema  Searchable parameters (make sure only non-pathological items are queried).  Return-ables parameters (default, required, possible)  Atlas II knows how to use these configurations to do many things within the system:  Query construction (OODT DIS or SQL)  User interface creation  So, how do you know how to query a meta-data collection? Especially one that is not local?  Profile Servers have an XML configuration file that Atlas II uses (and augments with its own spicings)  Mapping of schema  Searchable parameters (make sure only non-pathological items are queried).  Return-ables parameters (default, required, possible)  Atlas II knows how to use these configurations to do many things within the system:  Query construction (OODT DIS or SQL)  User interface creation

Atlas II Retrieval and OODT  Downloading of data via (at least) two mechanisms:  OODT Product server  Understands that a set of data may be required to fulfill order for an “EDR”. (E.g., data files and labels bundled together into a ZIP archive).  ‘wget’ style URL scripting straight to a web server (Atlas II to generate script for client)  In all cases, URL accesses the data and we intend for them to be immutable.  Downloading of data via (at least) two mechanisms:  OODT Product server  Understands that a set of data may be required to fulfill order for an “EDR”. (E.g., data files and labels bundled together into a ZIP archive).  ‘wget’ style URL scripting straight to a web server (Atlas II to generate script for client)  In all cases, URL accesses the data and we intend for them to be immutable.

Atlas II Status  Currently we are supporting both Cassini and MER operationally and have a development version for HiRISE.  Atlas II support both mission specific and multi- mission queries.  Mission specific queries:    (This is password protected)  Currently we are supporting both Cassini and MER operationally and have a development version for HiRISE.  Atlas II support both mission specific and multi- mission queries.  Mission specific queries:    (This is password protected)

Atlas II Status (cont.)

 Multi-mission query:  The same code base is used for the multi- mission view  Common parameters of each logical grouping (data set) is dynamically found and displayed and searched across missions.   Multi-mission query:  The same code base is used for the multi- mission view  Common parameters of each logical grouping (data set) is dynamically found and displayed and searched across missions. 

Atlas II Status (cont.)

Atlas II Configuration  Profile Server and/or SQL (JDBC) access to catalog  We could mirror the meta-data here, but why?  UofA dedicated to uptime preservation.  XML Profile server configuration to be shared with IN. HiRISE drives this. This encapsulates the schema so we know the mapping of the various fields to the PDS keywords.  Profile Server and/or SQL (JDBC) access to catalog  We could mirror the meta-data here, but why?  UofA dedicated to uptime preservation.  XML Profile server configuration to be shared with IN. HiRISE drives this. This encapsulates the schema so we know the mapping of the various fields to the PDS keywords.

Atlas II and OODT Overview

Data Migration Plan  What is required?  Sybase to MySQL to increase performance  Schema optimization  Configuration file to integrate with Atlas II and OODT  Order of data migration from Atlas I web- interface to Atlas II: THEMIS, Galileo, MPF, Voyager, Viking Orbiter, MGS, ODY (GRS), MEX, Clementine, Magellan  What is required?  Sybase to MySQL to increase performance  Schema optimization  Configuration file to integrate with Atlas II and OODT  Order of data migration from Atlas I web- interface to Atlas II: THEMIS, Galileo, MPF, Voyager, Viking Orbiter, MGS, ODY (GRS), MEX, Clementine, Magellan

Additional Features  HiRISE interface to JPEG2000 server  Necessary to support RDR retrieval of large files.  External processing interface. This is the ability to remotely process images, i.e. stretching, multi-format downloads for reasonably sized images.  Plan to incorporate image processing via MIPL/JAI software.  HiRISE interface to JPEG2000 server  Necessary to support RDR retrieval of large files.  External processing interface. This is the ability to remotely process images, i.e. stretching, multi-format downloads for reasonably sized images.  Plan to incorporate image processing via MIPL/JAI software.