OECD Expert Group on Statistical Data and Metadata Exchange Geneva May 2007 OECD.Stat SDMX Web Service (II) Jens Dossé, OECD

OECD SDMX web service Fully conforms to SDMX standard v2 Fully conforms to SDMX standard v2 Currently two methods: extract data in GenericData message extract Data Structure Definitions (Key Families), concepts and code lists Currently two methods: extract data in GenericData message extract Data Structure Definitions (Key Families), concepts and code lists Method planned for the future: extract Metadata Method planned for the future: extract Metadata Secured access: integrates authorization to OECD.Stat Secured access: integrates authorization to OECD.Stat Still needs a few additional conventions bridging SDMX-ML limits Still needs a few additional conventions bridging SDMX-ML limits

Additional conventions I For time series data, independently of the presence of a FREQUENCY dimension, the requested time frequency must be specified using a TIME_FORMAT attribute: Format values: P1Y=annual, P6M=semi-annual, P3M=quarterly and P1M=monthly data Time formats: yyyy, yyyy-mm or yyyy-mm-dd with digits y=year, m=month and d=day, e.g. ' '. This attribute is then also present in the resulting generic data message at the Series level of time-series data. For time series data, independently of the presence of a FREQUENCY dimension, the requested time frequency must be specified using a TIME_FORMAT attribute: P1M Format values: P1Y=annual, P6M=semi-annual, P3M=quarterly and P1M=monthly data Time formats: yyyy, yyyy-mm or yyyy-mm-dd with digits y=year, m=month and d=day, e.g. ' '. This attribute is then also present in the resulting generic data message at the Series level of time-series data. Non-time-series data: omit all time specifications in the query message The response messages will use 9999 as (obligatory) time periods Non-time-series data: omit all time specifications in the query message The response messages will use 9999 as (obligatory) time periods NEW: If the or nodes are not present then all the available values are returned NEW: If the or nodes are not present then all the available values are returned

Additional conventions II OECD.Stat data warehouse: Each dataset (multidimensional cube) has its own Data Structure Definition. Datasets can share common dimensions but which can have additional non-common codes OECD.Stat data warehouse: Each dataset (multidimensional cube) has its own Data Structure Definition. Datasets can share common dimensions but which can have additional non-common codes Several datasets within one message Query message can request data from several datasets NEW: Different datasets can now be returned within one SDMX generic message using MessageGroups Several datasets within one message Query message can request data from several datasets NEW: Different datasets can now be returned within one SDMX generic message using MessageGroups

Additional conventions III For simplicity during the start-up of this generic web service, the returned message headers only contain required information For simplicity during the start-up of this generic web service, the returned message headers only contain required information Maximum size of generic data message: 5 MB message is truncated if necessary Requests need to be split if this limit is exceeded Maximum size of generic data message: 5 MB message is truncated if necessary Requests need to be split if this limit is exceeded

(Still) open challenges for SDMX web services !!!: Big message sizes for any data format: need standard for compression !!!: Big message sizes for any data format: need standard for compression !!!: Security: need standard for authentication/encryption !!!: Security: need standard for authentication/encryption !!!: Need for another message format: generic cross-sectional format remove the special status of the Time dimension (as not all statistical data are time series) !!!: Need for another message format: generic cross-sectional format remove the special status of the Time dimension (as not all statistical data are time series) Registering SDMX web services in the SDMX registry Registering SDMX web services in the SDMX registry Multilaterally connecting organisations SDMX web services to build international networks Multilaterally connecting organisations SDMX web services to build international networks

OECD SDMX web service client example Simple web application Simple web application Provides example query messages Provides example query messages Source code freely available Source code freely available Secured access to all data in data warehouse (integrates authentication and authorization to OECD.Stat) Secured access to all data in data warehouse (integrates authentication and authorization to OECD.Stat)

Contacts/Links