OPeNDAP Developer’s Workshop Feb Server-side Functions for Geo-spatial Selection James Gallagher 22 Feb 2007

OPeNDAP Developer’s Workshop Feb Why build server-side functions? Some operations are much more efficient to do before sending the data. Some operations are possible only after performing several data accesses. But the real reason… Hide the mechanics of an operation behind an interface.

OPeNDAP Developer’s Workshop Feb Why is ‘Hiding’ Important for Data? Operations such as geo-referenced selection are different for each dataset/format convention. An interface can –combine knowledge of several conventions –be specialized –return better error messages than generic operators

OPeNDAP Developer’s Workshop Feb Server-side functions in Hyrax Four* server-side functions: –version(): Get info on installed functions –geogrid(): Given a Grid variable and a latitude-longitude box, return the data within that box in a Grid –geoarray(): Similar to geogrid() but for an Array –linear_scale(): Scale data using y=mx+b These are in libdap *Really 3.1 since version() is a placeholder

OPeNDAP Developer’s Workshop Feb Geo-spatial selection for a Grid variable How does geogrid() make getting data easier?

OPeNDAP Developer’s Workshop Feb First, the ‘old’ way… A user/client looks at the Grid’s map vectors - technically a metadata request Based on the values in the maps, they figure out the start and stop indexes They use those indexes to make the real request Summary: The request requires two data accesses, plus scanning the map vectors to determine the indexes

OPeNDAP Developer’s Workshop Feb geogrid() geogrid() arguments: –The dataset Grid name; –A latitude/longitude bounding box and –Zero or more relational expressions Evaluation of the Map vectors is still the ‘algorithm,’ but it’s done by the server, hidden from the user/client

OPeNDAP Developer’s Workshop Feb The geogrid() Function /data/nc/coads_climatology.nc "geogrid(SST,50,-70,48,-68,\"1000<TIME<2000\")" The data: Grid { Array: Float32 SST[TIME = 2][COADSY = 3][COADSX = 3]; Maps: Float64 TIME[TIME = 2]; Float64 COADSY[COADSY = 3]; Float64 COADSX[COADSX = 3]; } SST = { Array: {{{-1e+34, -1e+34, -1e+34},{-1e+34, -1e+34, -1e+34},{-1e+34, -1e+34, -1e+34}}, {{-1e+34, -1e+34, -1e+34},{-1e+34, -1e+34, -1e+34},{-1e+34, -1e+34, -1e+34}}} Maps: { , }, {47, 49, 51}, {-71, -69, -67} };

OPeNDAP Developer’s Workshop Feb The geoarray() Function Intent: Provide the same sort of geo-spatial selection capability for Arrays as for Grids. Current version: User/client must supply the geo-spatial extent of the data RSN* version: The function will use extent information when present in the dataset A future version will also be savvy about different projections *Really Soon Now

OPeNDAP Developer’s Workshop Feb The linear_scale() Function Perform simple linear scaling (y=mx+b) Use COARDS attributes if present Provide a way for clients to tell the function which coefficients to use

OPeNDAP Developer’s Workshop Feb The version()function Returns information on the functions installed and their versions Returns two responses: –version() human-readable text –version(xml) returns xml This is really just the first attempt at a discovery system for these functions!

OPeNDAP Developer’s Workshop Feb How they work The function is run by the constraint evaluator When a handler is initialized, it loads the functions it understands There is now a set of ‘generic’ functions in libdap; a handler can override those

OPeNDAP Developer’s Workshop Feb Implementing a Server-side function Functions take four arguments: –A list of arguments which may be constants or variables in the data set –A count of those arguments –A DDS (really DDX) for the data set –The string that names the dataset Once called, a server-side function can call any method is libdap. The function must return one of the libdap data types – that’s the function’s return value.

OPeNDAP Developer’s Workshop Feb What else do I need to know to write one? You should also understand how libdap evaluates constraint expressions How the libdap read() methods operate How to access attribute values from a from the libdap 3.x DDS object (which is actually the binary representation of the DDX)

OPeNDAP Developer’s Workshop Feb Future directions We need feedback on –The functions/features implemented so far –The function loading scheme –Discovery

OPeNDAP Developer’s Workshop Feb Features Are there specific conventions that should be supported? Other functions? Things the functions do that they shouldn’t

OPeNDAP Developer’s Workshop Feb Function loading/evaluation Now all of the functions are compiled into libdap Should functions be loaded from a shared library at run time? Configured in the BES? Should functions be an option for metadata (DDX) requests too? (we do support constraints for ‘syntax’ metadata requests)

OPeNDAP Developer’s Workshop Feb Discovery There must be a way to discover functions, true of false? The current version() function is really only a placeholder What are the requirements for discovery? Should we use the Data response? Error response? Something new?

OPeNDAP Developer’s Workshop Feb Vexing questions Server-side functions seem like a very powerful tool, but they get little use, why? Assuming these are to be driven by the community, how do we sustain that?