ESS VIP ICT Project Task Force Meeting 5-6 March 2013

Working package 1: Translation to SDMX standard(01/01/2013 - 01/06/2013) 1.1 Selection of data tables for the pilot Complete 1.2 Analysis of the selected Eurobase tables Complete 1.3 Creation of starting packages for the pilot 50% 1.4 Analysis of SDMX compliance Complete 1.5 Creation of ICT data related codelist 75% 1.6 Creation of SDMX 2.0 DSDs for data transmission 50% 1.7 Loading of the DSDs in the registry 1.8 Creation of associated dataflow Complete 1.9 Testing the transmission chain internally (SDMX-RI – ESTAT) Complete 1.10 Creation of guidelines for Member States on how to join the ICT 50% pilot with different scenarios 1.11 Drafting of the specifications for the ICT data dissemination application Work Package 1: Translation to SDMX standard Selection of data tables for the pilot In phase I, the development of the pilot will only concern the dissemination of aggregate data from the Enterprise survey. In order to reduce the workload for the pilot countries, a selection of around 40 Eurobase tables (Enterprise survey 2012 – NACE rev 2) will be used for the tests. This data has already been validated and processed. On the basis of the selected tables, creation of starting packages for Member States. Analysis of SDMX compliance - Each table will be analysed to evaluate if its structure is compliant with SDMX 2.0. The structure of the tables could eventually be revised at this stage. The data of each table will be split at country level to be transferred to the pilot countries and to be uploaded in their local database systems. Creation of ICT data related code lists New code lists will be created if necessary. Creation of SDMX 2.0 DSDs for data transmission Each DSD (Data Structure Definition) provides definitions and additional details for the dimensions and attributes used in representing data in a particular subject-matter domain. Statistical concepts such as frequency, reference area and institutional sector can play the role of dimensions. Concepts such as units of measurement or methodological comment can be attributes. While concrete dimension values are used to identify and point to a concrete data point (or set of data points), concrete attribute values provide qualitative information about observations (or sets of observations). In particular datasets, dimensions take their values from predefined code lists. Attributes, on the other hand, can take their values from code lists (coded attributes, unit of measure) or can be free text (non-coded attributes). In practice, some attributes must always have a value (they are defined as mandatory in a DSD) while others only have a value for each of the object(s) they are associated with under certain conditions (conditional attributes). Loading of the DSDs in the registry It was decided to use the SDMX registry in the context of this project. The Euro-SDMX registryis a metadata registry which implements the SDMX registry specifications. It provides a web-based user interface as well as web services for interacting with the SDMX structural metadata objects in use within Eurostat and with statistical partners. It will enable National Statistics Institutes and other external organizations to obtain Data Structure Definitions (DSD) and other structural metadata, such as the Metadata Structure Definitions (MSD) for the Euro-SDMX Metadata Structure (ESMS). The DSD is one of the SDMX artifacts needed to read and interpret the SDMX data file. Creation of associated dataflow In SDMX, data sets are reported or disseminated according to a data flow definition. The data flow definition identifies the data structure definition (key family) and may be associated with one or more subject matter domains; this facilitates the search for data according to organised category schemes. A “data flow”, in this context, is an abstract concept of the data sets, i.e. a structure without any data. The data flow is one of the SDMX artifacts needed to read and interpret the SDMX data file. Creation of the starting packages for each pilot country Eurostat will facilitate the preparation phase for each country participating to the pilot by providing a set of structure files and associated data to be used in NSIs premises or provide support to countries willing to do the data extraction from their database. Testing the transmission chain internally (Eurostat test premises.) Creation of guidelines for Member States on how to join the ICT pilot with different scenarios. The guidelines will contain detailed instruction for the installation of the different components of the SDMX-RI and related tools and/or how the solution used for the Census Hub could be reused. Drafting of the specifications for the ICT data dissemination application: In order to test the Pull with Hub transfer mode, it will be necessary to adapt the census hub application for the dissemination of ICT data.

Working package 2: Data dissemination (01/05/2013 – 30/12/2013) 2.1 Analysis of the requirements 2.2 Development of the dissemination application 2.3 Documentation and testing (contractor) 2.4 First release 2.5 ESTAT review and testing 2.6 Corrective maintenance 2.7 Downloading the DSDs from the SDMX Registry 2.8 Based on the guidelines produced in WP 1: 2.8.1 Load the data into a database 2.8.2 Load the DSDs in the Mapping Assistant (SDMX-RI) 2.8.3 Do the mapping 2.8.4 Deploy/reuse the same Web Services as for the Census 2.8.5 open the connection for ESTAT 2.9 Testing of the connection with the pilot countries NSI The requirements how to prepare a proposed dissemination product for ICT data will be analysed and a dissemination application will be designed and developed. For the purpose of this pilot phase the current application developed under the census hub project will be reused. First version of the application is to be released for test and review in September. The second version is planned to be operational December. The connection to be pilot countries is to be enabled as of June/July As the Ict domain will go for SDMX compliance we think that this pilot phase will facilitate the preparation of SDMX compliance in each NSIs by following the standard way of data dissemination and benefit from the knowledge obtained from using the tool ESTAT offer and build SDMX capacity inside the house. With the proposed steps to be executed by the MS we would like to put in place the reusability of already installed in the MS components and to reuse the existing tools and applications .Member states are asked to: download the Dsds from the Registry; load the data files prepared by ESTAT; upload the SDMX structure files in the Mapping Assistant, do the mappings (1 to 1 mapping); use the same test web service or deploy a new one and open the connection to ESTAT