PDAP Implementation for Clementine Data ISAS/JAXA Yukio Yamamoto July 14,

Basic design of PDAP system in JAXA

IPDA Architecture and PDAP IPDA Architecture Standards Process Architecture Standards Data Architecture Standards Technology Architecture Standards Archive Planning Preservatio n Planning Preservatio n Planning Data Standards Data Standards Technology Standards Technology Standards Peer Review Peer Review Data Model Object Model Data Dictionary Data Formats Grammar Repository Structure Repository Structure Name Space Protocol Portals Registries Archive Tools Scope required for implementation

Data Model preferred from the implementation of PDAP system 4 PDS PSA FITS … Domain Specific Format (Project Specific Format) IMAGE PRODUCT SPECTRUM PRODUCT FLYBY PRODUCT … PDAP Shared Model (Interoperable Model) Currently, the target of JAXA’s PDAP implementation is for “PDAP shared model”

PDAP INPUT parameters RETURN_TYPE, RESOURCE_CLASS PDAP DATABASE (PDAP MODEL) DATA_SET_ID, PRODUCT_ID, START_TIME, STOP_TIME, INSTRUMENT_TYPE, INSTRUMENT_NAME BASIC LONGITUDE, LATITUDE MAP_PROJECTED_PRODUCT FLYBY_PRODUCT DISTANCE_TO_TARGET PHASE, ROLL, PITCH,YAW PDAP ENGINE MISSION DATA NO OPERATOR Target of PDAP parameters 1 ） PDAP Engine 2 ） PDAP Intermediate Model 3 ） Original Data We want to search by PDS original meta data. What is RESOURCE_CLASS ? Pagination, etc.

What does RESOURCE_CLASS determine in JAXA implementation? 6 URL PARAMETERS URL PARAMETERS SEARCH ALGORITHM SEARCH ALGORITHM VOTABLE OUTPUT VOTABLE OUTPUT FILTER OUTPUT FILTER PDAP Metadata PDAP Metadata “RESOURCE_CLASS” defines input parameters and output items. For the flexible expansion, the pluggable RESOURCE_CLASS is designed in JAXA implementation. Additional Information Additional Information PDS Data Archive PDS Data Archive RESOURCE_CLASS determines 1.URL parameters 2.PDAP metadata structure 3.Search Algorithm 4.Filter Algorithm 5.VOTable Outputs URL PARAMETERS URL PARAMETERS VOTABLE OUTPUT VOTABLE OUTPUT PDAP ENGINE PDAP ENGINE PDAP specification: RESOURCE_CLASS only determines URL parameters and VOTable outputs. JAXA PDAP implementation: PDAP ENGINE Conversion program

RESOURCE_CLASS Modeling with UML Class Diagram 7 Database Columns Database Columns Output Filter (getter) Output Filter (getter) RESOURCE CLASS RESOURCE CLASS URL Parameters (setter) URL Parameters (setter) Search Algorithm Search Algorithm The UML diagram is useful for design of RESOURCE_CLASS

RESOURCE_CLASS=“PRODUCT” 1/2 URL PARAMETERS 8 URL PARAMETERSVALUE FORMATPARAMETER FORMATSEARCH ALGORITHM(SQL) DATA_SET_IDCHAR(40)Arbitrary charactersDATA_SET_ID= ‘(VALUE)’ DATA_SET_NAMECHAR(60)Arbitrary charactersDATA_SET_NAME = ‘(VALUE)’ PRODUCT_IDCHAR(40)Arbitrary charactersPRODUCT_ID = ‘(VALUE)’ TARGET_TYPECHAR(20)Arbitrary charactersTARGET_TYPE = ‘(VALUE)’ TARGET_NAMECHAR(120)Arbitrary charactersTARGET_NAME = ‘(VALUE)’ INSTRUMENT_TYPECHAR(30)Arbitrary charactersINSTRUMENT_TYPE = ‘(VALUE)’ INSTRUMENT_NAMECHAR(60)Arbitrary charactersINSTRUMENT_NAME = ‘(VALUE)’ START_TIMETIMEYYYY-MM-DDThh:mm:ss[.fff]START_TIME >= ‘(VALUE)’ STOP_TIMETIMEYYYY-MM-DDThh:mm:ss[.fff]STOP_TIME < ‘(VALUE)’ RESOURCE_CLASS “PRODUCT” is the simplest case for the PDAP implementation. 1.All “SEARCH Algorithms” of URL parameters can be described as SQL WHERE phrase 2.There is no “FILTER Algorithms” for database response

RESOURCE_CLASS=“PRODUCT” 2/2 VOTABLE OUTPUT 9 DATABASE COLUMNTYPEOUTPUT FILTER ucdutypedata type array size DATA_SET_IDCHAR(40)-DATA_SET_IDpds:DATA_SET.DATA_SET_IDchar* PRODUCT_IDCHAR(40)-PRODUCT_IDpds:PRODUCT.PRODUCT_IDchar* DATA_SET_NAMECHAR(60)-DATA_SET_NAMEpds:DATA_SET.DATA_SET_NAMEchar* TARGET_TYPECHAR(20)-TARGET_TYPEpds:PRODUCT.TARGET_TYPEchar* TARGET_NAMECHAR(120)-TARGET_NAMEpds:PRODUCT.TARGET_NAMEchar* INSTRUMENT_TYPECHAR(30)-INSTRUMENT_TYPEpds:PRODUCT.INSTRUMENT_TYPEchar* INSTRUMENT_NAMECHAR(60)-INSTRUMENT_NAMEpds:PRODUCT.INSTRUMENT_NAMEchar* START_TIMETIMESTAMP-START_TIMEpds:PRODUCT.START_TIMEchar* STOP_TIMETIMESTAMP-STOP_TIMEpds:PRODUCT.STOP_TIMEchar* DATA_ACCESS_REFE RENCE CHAR(1024)-DATA_ACCESS_REFER ENCE char* VOTable output is composed of database response directly

RESOURCE_CLASS=“MAP_PROJECTED_PRODUCT” 1/2 URL PARAMETERS 10 URL PARAMETERSVALUE FORMATPARAMETER FORMATSEARCH ALGORITHM LONGITUDEDOUBLEMin/MaxSee “Search for ROI algorithm” LATITUDEDOUBLEMIN/MAXSee “Search for ROI algorithm” SPICE_REFERENCE_FRAMECHAR(40)Arbitrary charactersSee “Frame conversion algorithm” RESOURCE_CLASS “MAP_PROJECTED_PRODUCT” has additional URL parameters: LONGITUDE, LATITUDE, and SPICE_FRAME. These parameters required comparably complex algorithms

RESOURCE_CLASS=“MAP_PROJECTED_PRODUCT” 2/2 VOTABLE OUTPUT 11 DATABASE COLUMNTYPEOUTPUT FILTER ucdutypedata type array size CENTER_LONGITUDEFLOAT*8-CENTER_LONGITUDEpds:PRODUCT.CENTER_LONGITU DE double CENTER_LATITUDEFLOAT*8-CENTER_LATITUDEpds:PRODUCT.CENTER_LATITUD E double MAP_PROJECTION_TYPECHAR(20)-MAP_PROJECTINO_T YPE pds:MAP_PROJECTINO_TYPEchar* COORDINATE_SYSTEM_TYPECHAR(25)-COORDINATE_SYSTE M_TYPE pds:COORDINATE_SYSTEM_TYPEchar* COORDINATE_SYSTEM_NAMECHAR(30)-COORDINATE_SYSTE M_NAME pds:COORDINATE_SYSTEM_NAM E char* UPPER_LEFT_LONGITUDEFLOAT*8See “Footprint filter” FOOTPRINT-char* UPPLER_LEFT_LATITUDEFLOAT*8 LOWER_LEFT_LONGITUDEFLOAT*8 LOWER_LEFT_LATITUDEFLOAT*8 LOWER_RIGHT_LONGITUDEFLOAT*8 LOWER_RIGHT_LATITUDEFLOAT*8 UPPLER_RIGHT_LONGITUDEFLOAT*8 UPPER_RIGHT_LATITUDEFLOAT*

(x1,y1) (x2,y2) (x3,y3) (x4,y4) S P(x,y) min(x1,x2)max(x3,x4) min(y2,y3) max(y1,y4) P(x,y) QUERY SQL SELECT * FROM data WHERE id IN ( SELECT id FROM data WHERE (x >= min(x1,x2) AND x <= max(x1,x2)) AND (y >= min(y1,y2) AND y <= max(y1,y2)) ) AND x >= line_xpos(x1,y1,x2,y2,y) AND x <= line_xpos(x3,y3,x4,y4,y) AND y >= line_ypos(x1,y1,x2,y2,x) AND y <= line_ypos(x3,y3,x4,y4,x) Search Algorithm for Region Of Interest (ROI) (LONGITUDE/LATITUDE) FUNCTIONS line_xpos(x1,y1,x2,y2,y) := x1+(y-y1)(x2-x1)/(y2-y1) line_ypos(x1,y1,x2,y2,x) := y1+(x-x1)(y2-y1)/(x2-x1) PARAMETERS x := LONGITUDE (URL parameter) y := LATITUDE (URL parameter) x1 := UPPER_LEFT_LONGITUDE (from DB) y1 := UPPER_LEFT_LATITUDE (from DB) x2 := LOWER_LEFT_LONGITUDE (from DB) y2 := LOWER_LEFT_LATITUDE (from DB) x3 := LOWER_RIGHT_LONGITUDE (from DB) y3 := LOWER_RIGHT_LATITUDE (from DB) x4 := UPPER_RIGHT_LONGITUDE (from DB) y4 := UPPER_RIGHT_LATITUDE (from DB) To serach products in ROI, there is two step of searches. The first search is rough search to make a short list of products using a rectangle framing the quadrangle of image four corners. The second search is strict search using functions

Footprint Filter To make “FOOTPRINT” fields in VOtable, four corners of image are used. For generic polygonal footprint, filter is not yet implemented in JAXA implementation. QUADRANGLE FOOTPRINT (lon1,lat1) (lon2,lat2)(lon3,lat3) (lon4,lat4) DATABASE COLUMN NAMEVALUE UPPER_LEFT_LONGITUDELon1 UPPLER_LEFT_LATITUDElat1 LOWER_LEFT_LONGITUDElon2 LOWER_LEFT_LATITUDElat2 LOWER_RIGHT_LONGITUDElon3 LOWER_RIGHT_LATITUDElat3 UPPLER_RIGHT_LONGITUDElon4 UPPER_RIGHT_LATITUDElat4 Footprint Filter (lon1:lat1,… Example of FOOTPRINT FILTER (SQL) SELECT ‘POLYGON (‘ || UPPER_LEFT_LONGITUDE || ‘:’ || UPPLER_LEFT_LONGITUDE || ‘,’ || LOWER_LEFT_LONGITUDE || ‘:’ || LOWER_LEFT_LONGITUDE || ‘,’ || LOWER_RIGHT_LONGITUDE || ‘:’ || LOWER_RIGHT_LONGITUDE || ‘,’ || UPPER_LEFT_LONGITUDE || ‘:’ || UPPER_LEFT_LONGITUDE || ‘,’ || ‘)’ FROM MAP_PROJECTED_PRODUCT or Application Filter Database

Hierarchy of PDS compatible product 14 DATA_SET PRODUCT OBJECT PRODUCT … … OBJECT … … File Directory Inside of file (Data Body) The concept of “DATA_SET” is a directory, and “PRODUCT” corresponds to a file. A “PRODUCT” consists of one or several data. Hierarchy of PDS compatible product TARGET … … Inside of data UML expression

Table structure of RESOURCE_CLASS “PRODUCT” 15 A file of multiband imager Band1 object Band2 object Band3 object PRODUCT_ID: ABC Two Targets found in Bnad 3 object: Moon, Mars DATA_SET_IDPRODUCT_IDOBJECT_IDTARGET_ID SAMPLE_DATA_SETABCBAND1Moon SAMPLE_DATA_SETABCBAND2Moon SAMPLE_DATA_SETABCBAND3Moon SAMPLE_DATA_SETABCBAND3Mars Normalized structure of the RESORUCE_CLASS “PRODUCT” The file structure is directory described in the followings: DATA_SET_IDPRODUCT_IDOBJECT SAMPLE_DATA_SETABC BAND1Moon BAND2Moon BAND3Moon Mars VOTable has the same structure of this table

Implementation of PDAP for Clementine

System Overview of PDAP for Clementine PDAP Engine (PHP Program) PDAP Engine (PHP Program) JAXA PDAP System PDAP Database (PostgreSQL) PDAP Database (PostgreSQL) Browser 1.Search Request 2.Response Clementine Data Clementine Data Clementine Data Clementine Data Clementine Data Clementine Data NASA JPL 3.HTTP Request 4.HTTP Response First, a user accesses JAXA PDAP System to search Clementine products, then retrieves data and thumbnail image from the NASA JPL website

Available “RESOURCE_CLASS”es for CLEMENTINE DATA DATA_SET_IDDATA TYPEAVAILABLE RESOURCE_CLASS CLEM1-L-H-5-DIM-HIRES-V1.0IMAGEPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L-RSS-1-BSR-V1.0-PRODUCT CLEM1-L-RSS-5-BSR-V1.0-PRODUCT CLEM1-L-U-5-DIM-BASEMAP-V1.0IMAGEPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L-U-5-DIM-UVVIS-V1.0IMAGEPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0IMAGESee below for more details 18 DATA_SET_IDPRODUCT_IDINSTRUMENT_IDRESOURCE_CLASS CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0LUA3916K.063UVVISPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0LHD1656F.063HIRESPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0LLA1658F.063LWIRPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0LNA1671F.063NIRPRODUCT, MAP_PROJECTED_PRODUCT CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V

LABEL CONVERSION 1/3 (RESOURCE_CLASS=DATA_SET) 1.URL for DATA_SET is currently set to the root diretory on the URL: ex) If DATA_SET_ID is “CLEM1-L-RSS-5-BSR-V1.0”, DATA_ACCESS_REFERENCE is “ Feedback for PDAP specification: The PDAP version 0.3 document should describe how to identify the resource type: directory, file, etc. One of resolutions, a MIME Content-Type may be available. (cf Clementine PDS LABELOther informationDatabase Column DATA_SET_ID MISSION_NAMEDATA_SET_NAME “SATELLITE” fixedTARGET_TYPE “MOON” fixedTARGET_NAME DATA_SET dependenceINSTRUMENT_TYPE INSTRUMENT_NAME START_TIME STOP_TIME URL for DATA_SET 1 DATA_ACCESS_REFERENCE

LABEL CONVERSION 2/3 (RESOURCE_CLASS=PRODUCT) Clementine PDS LABELOther informationDatabase Column DATA_SET_ID MISSION_NAMEDATA_SET_NAME PRODUCT_ID “SATELLITE” fixedTARGET_TYPE “MOON” fixedTARGET_NAME PRODUCT dependenceINSTRUMENT_TYPE INSTRUMENT_NAME START_TIME STOP_TIME URL for PRODUCTDATA_ACCESS_REFERENCE URL for PRODUCT LABELLABEL_ACCESS_REFERENCE Feedback for PDAP specification: “LABEL_ACCESS_REFERENCE” is convenient for detached format

LABEL CONVERSION 3/3 (RESOURCE_CLASS=MAP_PROJECTED_PRODUCT) MAPextended Input and Output 21 Clementine PDS LABELOther informationDatabase Column CENTER_LONGITUDE CENTER_LATITUDE “POLAR STEREOGRAPHIC” fixedMAP_PROJECTION_TYPE “SATELLITE” fixedCOORDINATE_SYSTEM_TYPE “MOON” fixedCOORDINATE_SYSTEM_NAME “MOON_ME” fixedSPICE_REFERENCE_FRAME URL of product imageIMAGE_ACCESS_REFERENCE URL of product thumbnail iconICON_ACCESS_REFERENCE RETICLE_POINT_LATITUDE RETICLE_POINT_LONGITUDE - See “MAP_PROJECTED_PRODUCT database structure” for details UPPER_LEFT_LONGITUDE UPPER_LEFT_LATITUDE LOWER_LEFT_LONGITUDE LOWER_LEFT_LATITUDE LOWER_RIGHT_LONGITUDE LOWER_RIGHT_LATITUDE UPPER_RIGHT_LONGITUDE UPPER_RIGHT_LATITUDE Additional database columns of MAP_PROJECTED_PRODUCT

MAP_PROJECTED_PRODUCT database structure: PRODUCT.FOOTPRINT 1/2 lua3916k.063 (285.39,13.92) (lat,lon)= (287.38,13.91) [deg] (287.40,15.37)(285.39,15.38) PDS LABELVALUE CENTER_LATITUDE14.65 CENTER_LONGITUDE RETICLE_POINT_LATITUDE( 13.91, 13.92, 15.38, 15.37) RETICLE_POINT_LONGITUDE( , , , ) Ex) Part of pds label in lua3916k.063 PDAP INTERNAL KEYWORDVALUE CENTER_LATITUDE14.65 CENTER_LONGITUDE UPPER_LEFT_LONGITUDE UPPLER_LEFT_LATITUDE13.91 LOWER_LEFT_LONGITUDE LOWER_LEFT_LATITUDE15.37 LOWER_RIGHT_LONGITUDE LOWER_RIGHT_LATITUDE15.38 UPPER_RIGHT_LONGITUDE UPPER_RIGHT_LATITUDE13.92 PDAP Database DATA_SET_ID “CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0” uses “RETICLE_POINT_LATIDUE” and “RETICLE_POINT_LONGITUDE” in PDS labels to create FOOTPRINT Information. The principle points are defined as the upper left pixel(UL), the upper right pixel(UR), the lower left(LL), and the lower right(LR). For more details, see both of “edrsis.asc” and “errata.txt”. 1:UL2:UR 3:LR4:LL N

MAP_PROJECTED_PRODUCT database structure: PRODUCT.FOOTPRINT 2/2 UI80N075 (107.4E,84N)(42.7E,84N) (60E,77N) (90E,77N) PDS LABELVALUE CENTER_LATITUDE CENTER_LONGITUDE0.0 MAXIMUM_LATITUDE MINIMUM_LATITUDE EASTERNMOST_LONGITUDE WESTERNMOST_LONGITUDE PDS LABEL does not contain the four corners' information. PDS LABEL of UI80N075.IMG HTML FILE: browse/750nm/medium/ui80n075.htm 42.7°E 107.4°E 84°N 84°N 77°N 60.0°E 90.0°E PDAP INTERNAL KEYWORDVALUE UPPER_LEFT_LONGITUDE42.7 UPPLER_LEFT_LATITUDE84 LOWER_LEFT_LONGITUDE60 LOWER_LEFT_LATITUDE77 LOWER_RIGHT_LONGITUDE90 LOWER_RIGHT_LATITUDE77 UPPER_RIGHT_LONGITUDE107.4 UPPER_RIGHT_LATITUDE84 Create PDAP DATABASE from HTML PDAP Database to create FOOTPRINT DATA_SET_ID “CLEM1-L-U-5-DIM-UVVIS-V1.0” and “CLEM1-L-H-5-DIM-HIRES-V1.0” does not have FOOTPRINT Information in their PDS labels, but HTML files contain four corners’ information

STOP_TIME = “N/A” Some products do not have “STOP_TIME” while “START_TIME” exists though. Ex) clem1-l_e_y-a_b_u_h_l_n-2-edr-v1.0/cl_0022/lun103/luxxxxxx/luxxxxxi/lue3177i.103 DATA_SET_ID = "CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0“ PRODUCT_ID = "LUE3177I.103“ … START_TIME = T14:51:20.339Z STOP_TIME = "N/A" … EXPOSURE_DURATION = It is assumed that STOP_TIME has the same value of START_TIME since the exposure duration is too short. LUE3177I

Feedback to PDAP specification

1.DATA_ACCESS_REFERENCE should provide “Directory Reference” Especially for RESOURCE_CLASS=DATA_SET, DATA_ACCESS_REFERENCE should provide a reference to directory. Most of NASA PDS products can be accessed from internet, and their URLs is structured by the name of “DATA_SET_ID”, which means that the directory name of DATA_SET_ID is convenient for users. The huge archived file including whole files of “DATA_SET” is unrealistic. DATA_SET_ID is convenient for users. The huge archived file including whole files of “DATA_SET” is unrealistic Ex) If DATA_ACCESS_REFERENCE shows directory, the following URL can be used:

2.Ambiguity of Search Result for Time Input SQL1 can only search ‘Case 3’, but SQL2 can search between case 2,3,4,and 6. Which is better response as a search result ? Case 1 Case 2Case 4 Case 5 Case 3 Case 6 SQL1: SELECT * FROM PRODUCT WHERE START_TIME >= ‘a’ AND STOP_TIME <= ‘b’ Case 1 Case 2Case 4 Case 5 Case 3 Case 6 START_TIME:aSTOP_TIME:b SQL2: SELECT * FROM PRODUCT WHERE STOP_TIME >= ‘a’ AND START_TIME <= ‘b’

3.Ambiguity definition of RESOURCE_CLASS OUTPUT For example, DATA_SET has sometimes several instruments, and multiple targets: The DATA_SET_ID “CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0” has the following instruments: 1.ULTRAVIOLET/VISIBLE CAMERA 2.B STAR TRACKER CAMERA 3.LIDAR HIGH-RESOLUTION IMAGER 4.LONG WAVELENGTH INFRARED CAMERA 5.NEAR INFRARED CAMERA 6.ULTRAVIOLET/VISIBLE CAMERA The PDAP version 0.3 document does not describe the detail for the output, which may cause the inconsistency between the different implementations. No.DATA_SET_ID 1CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0 OR No.DATA_SET_IDINSTRUMENT_NAMETARGET_NAME 1CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0ULTRAVIOLET/VISIBLE CAMERAMOON 2CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0ULTRAVIOLET/VISIBLE CAMERASKY 3CLEM1-L/E/Y-A/B/U/H/L/N-2-EDR-V1.0NEAR INFRARED CAMERAMOON … Developer does not recognize which implementation is official from the PDAP version 0.3 document

4. How to specify the longitude/latitude ? MAP LONGITUDE=-10/120 MAP LONGITUDE=0/120,350/360 Both two are intrinsically same

Concept of ATTRIBUTES In “PDAP version 0.3”, RESOURCE_CLASS can be specified. The member of this RESOURCE CLASS is in the followings: - DATA_SET - PRODUCT - IMAGE PRODUCT - FLYBY PRODUCT - etc. IMAGE PRODUCT assume that the map-projected image, which should be “MAP PROJECTED PRODUCT.” FLYBY ATTRIBUTE can be defined when TARGET AND PRODUCT, which means MAP_PROJECTED_ATTRIBUTE and FLYBY_ATTRIBUTE are not exclusive. Appendix A PDAP System Internal Design

Appendix B VOTable Spliting for Pagination Large VOTable File VOTable 1 VOTable 2 VOTable 3 ABCD ABCD 1234 ABCD 5678 ABCD Original Table Splitted Tables OffsetLimitPrevNext 44http://... Meta Table for Pagination OffsetLimitPrevNext 04http://... OffsetLimitPrevNext 84http://

Appendix B Splitted VOTable Example <VOTABLE version=“1.2” xmlns:xsi=“ xmlns=“ xmlns:stc=“ > Meta Header for Split Splitted VOTable Splitted VOTable