Stuff about DX/GS

Overview

Installation of client/server packages 1 Complex installation: pydxs (pydxc) package into …/lib/python2.4/site- packages/ dxs (dxc) into /usr/local/…/dxs/ - holds config file, dataset listings, utility scripts, log dir etc. Client has web content: /srv/www/htdocs/dx/ Requires >15 paths/variables to be set pre- installation.

Installation of client/server packages 2 Re-using python’s distutils package: When creating source distribution: python setup.py sdist When unpacking/building: tar –xvzf …tgz cd … python setup.py install [options] All complexity of installing in different locations is done in setup.py

Utilities 1 DateTimeManager.py: Python utils for date/time objects with: add(, ) functionality createList(,,, ) String formatting of time objects Simple Calendar support SessionObject.py Session management via a python shelve object written to a file.

Utilities 2 LogManager.py: Uses python’s logging package

Configuration Configuration files and vars: I used *.py config files. Moving towards using python’s ConfigParser I use UPPER_CASE to show vars are from config file. In order to allow multiple

Overlap CSML and DX/GS have overlap: Date/time funcs Subset to multiple files/single file rules File-naming rules Identify axis etc rules Feature (GML) analysis – sort of

Error Handling Server hierarchy based on: DXError Error raised may be caught elsewhere in the system. Administrator will be mailed Need sensible object to send back to client (I’ve used string for errors). But still dysfunctional! CGI uses DEBUG switch to determine if using cgitb module.

DX/GS walk through DX(GS)Controller class: def __init__(self, args): - takes a set of arguments controls flow of request. def _parseArgs(self, args): - parses arguments. They may be selections or actions. def _checkSecurity(self): - if required, calls security code to check what user is allowed to do. def _constructSessionObject(self): - constructs a session object (new or saved on server) to store user session info. def _generateOptions(self): - generate a list of lists containing the next set of options available to user based on existing session object. def _checkCredit(self): - check user has enough credit to get what they’ve requested. def _createOutput(self, costOnly=None): - create output file(s).

DX Server Details 1 (old)

DX Server Details 2 (old)

Handling multiple data models in the DX DX now has standard interfaces (internally) for multiple models: DataHandler.py: openDataFile(…), getVariables(…), getDomain(…), readVariableSubsetIntoMemory(…) OutputHandler.py: writeVariable(…), writeGlobalAttributes(…)

Python style Creating large packages, makes sense for UML to look like content, hence: 1 class per module Class name same as module name, e.g.: CDMSDataHandler.py holds class “CDMSDataHandler” Class names begin with capital Methods being with lower case Internal only methods prefixed with “_”.

Security Interface Agnostic to type of security used Understands concept of roles and users so access could be restricted by either/both. Can switch on/off Interface: (roles, token)=validate(username, password, token) (roles, users)=ResourceAccess( )

Experience I have Build cycle for Python Web Service: Change, build, deploy, test Web Service testing with command-line clients Unit testing with: if __name__==“__main__”: Allows me to test each component Encourages me to keep interface objects simple

ZSI experience Change of interface slightly with version 2.0 Have encoded version specifics: if zsiversion>=2.0: response=response[0][0] Error catching problem: Error stack ‘gets lost’ with ZSI unless I define error classes.

Web interface issues Have traditionally kept Javascript minimal Ajax paradigm means time to move forward. CSS should be doing all styling and should be consistent for all NDG web look and feel. Can we identify third-party libraries to build upon: rico, prototype, x library etc.

Ongoing issues (feedback?) WSDL never settling down: Requirements of server interface expanding Future (WFS/WCS) requirements suggest all should be XML rather than SOAP-Arrays which are currently being used. Job submission/queuing: Not an operational solution yesterday – maybe have a basic one (sQimPy).

Things I’d like us to do Use one python installation across NDG (needs to be CDAT though!?) Use common GUI styling Use common code for date/time/data object manipulation Use common feature identification code