ESSnet on SDMX phase II June 7-8, Luxembourg

Statistical Information System Architecture and Integration SISAI Objectives of ESSnet on SDMX phase II Implementation of SDMX at international and National level; The further development of the SDMX technical standards used for data and metadata; The further development of SDMX Content-oriented Guidelines; The evaluation adaptation and further development of SDMX IT architecture and IT tools; The increase of SDMX capacity and knowledge at all levels. June 7-8, Luxembourg 2

Statistical Information System Architecture and Integration General information Invitation letter: The end of September 2010. MBGA submitted to Eurostat: The begin of November 2010. Countries: Bulgaria, Italy, Slovak Republic and Sweden. Coordinator: Italy. Amount: 595,492 euro (70% contribution from EC). Starting date: 28.03.2011 Duration: 18 Months. WPs proposed: 5 Work packages. June 7-8, Luxembourg

Statistical Information System Architecture and Integration Workpackages presented WP1- Management of the action (IT) WP2- Pc-Axis SDMX Integration phase II (SW) WP3- Support of SDMX application for Micro Data handling (SK) WP4 - Harmonization of existing SDMX tools (IT,BG) WP5- Capacity building actions (IT) June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP1 – Management of the action Objectives Ensure the success of the action through: contacts with the Commission and co-partners; monitoring of the different Work-Packages; identification of the changes to the plan and submit them to the Commission; identification of risks an problems and suggestion of solutions. June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP1 – Management of the action Activities Organization of meetings: Kick-off meeting held in Lisbon 1st April 2011. Three coordinating meetings (09/2011, 03/2012, 09/2012). Coordination meetings with CORE. Monitoring of the different WPs with a monthly progress report Inform Eurostat on working progress: Quarterly reports. Intermediate report. Final report. June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP2 – Pc-Axis SDMX integration phase II Objectives Reporting of statistics to international organizations in SDMX format. To set up routines to send data in accordance with SDMX-RI. Use the dissemination databases and the PC-Axis environment. June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP2 – Pc-Axis SDMX integration phase II Starting point SDMX files from a SQL database on a Web Hotel with RSS notification; SDMX files from a SQL database to a Web service without RSS notification; SDMX files from a SQL database to a Web Service with RSS notification (not finalized in Phase I) SDMX files from PC-Axis files on a Web hotel with RSS notification SDMX files from PC-Axis files to a Web service without RSS notification June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP2 – Pc-Axis SDMX integration phase II June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP2 – Pc-Axis SDMX integration phase II Activities Enhance and consolidation of the starting point architecture: new functionality in the PX-Web tool to integrate the architecture with graphical tools that use SDMX (eXplorer, SDMXDataExpress), installation and test of new versions of the SDMX-RI. Test the architecture with statistical more complex in terms of number of variables, attributes and content (ex. Foreign Trade, Production community, Maritime Statistics and Structural Business Statistics) to see if they can be covered by it. Present the results reached in phase I to the other countries taking action solving problems that can arise when the solution developed in phase I will be used June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP2 – Pc-Axis SDMX integration phase II Deliverables D.1 Software: a new version of PX-Web with new functionality implemented during the project D.2 Documentation: a more complete User’s Manual of architecture previously defined containing more use cases referred to other statistics. D.3 Workshop for Pc-Axis community and for other NSIs, in which will be described the process and tools used WP2- gantt   Apr May Jun Jul Aug Sep Oct Nov Dic Jan Feb Mar M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 D.1 D.2 Beta release Final release D.3 June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP3 – SDMX for Micro-data handling Objectives To support the implementation of SDMX, both from a methodological and technological point of view, in the survey design and data collection phases of the statistical business process. Compare the two standards SDMX and DDI, propose a possible SDMX extension. June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP3 – SDMX for Micro-data handling Starting point Definition of three Metadata Structure Definitions’ (MSDs) to describe the structure of the questionnaire: codelist, modules of questionnaire and the item of the different modules (Metadata Editor v1.0 - Metadata Technology) ECollect-X application: starting from the MSDs defined, creates a form to be filled by respondent. Two pilots studies to test the application and flow defined: Monthly questionnaire from survey in tourism statistics Questionnaire data on population, part A from Census on Population, Housing and Apartments June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP3 – SDMX for Micro-data handling Activities Creation of a tool that creates automatically a Data Structure Definition (“DSD generator”) starting from MSDs used to describe the questionnaire. The DSD will be used to map micro-data stored into the Micro-Data Database using ECollect-X application. Enhance ECollect-X with new functionalities (more complex calculation and validation). Test the software implemented, “DSD generator”, with two surveys. Analyze DDI vs SDMX standard: a comparative study of the DDI standard with the SDMX standard and their application in the micro-data processing an/or dissemination June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP3 – SDMX for Micro-data handling June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP3 – SDMX for Micro-data handling Deliverables D.1 Software: “DSD generator” application and enhancement of ECollect-X D.2 Documentation: user and system documentation D.3 Documentation: a report on the results of the pilot implementation of the “DSD generator” developed in the WP3 D.4 Documentation: documentation result of DDI vs SDMX analysis WP3- gantt   Apr May Jun Jul Aug Sep Oct Nov Dic Jan Feb Mar M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 D.1 D.2 D.3 D.4 June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP4 – Harmonization of existing tools Objectives Create the necessary instruments to allow the community of developers to reuse or to adapt to their own needs some modules of the SDMX-RI in order to reduce costs and to contribute to the further development of the SDMX-RI Upgrade the SDMX-RI with two modules that allow the user the creation of a “SDMX compliant” database starting from scratch. Actually the SDMX-RI can be used on dissemination databases already existing. June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP4 – Harmonization of existing tools Activities Analyze the SDMX-RI, its software module and its documentation giving suggestion on how to improve it. Provide two examples on how to use some modules of SDMX-RI. Integrate two tools on SDMX Istat framework inside the SDMX-RI. This is will be reached with: Merge of the two modules in one “Loader” Rewriting in C# Use modules developed by Eurostat Interact with the “Mapping Store” June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP4 – Harmonization of existing tools Deliverables D.1 Technical document that describes where the documentation should be improved so that the software can be easily re-used D.2 Step-by-Step tutorial on two examples D.3 Technical document on how to integrate Manager and Client modules (EGR included) D.3.a Software packages D.3.b Software documentation WP4- gantt   M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 D.1 Beta release Final release D.2 D.3.a D.3.b June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP5 – Capacity Building Objectives The increase of SDMX capacity and knowledge at all levels Activities Analysis of available documentation produced by Eurostat in order to build on existing documentation and to identify gaps/ improvements needed Producing a Handbook on “How to use SDMX” for statisticians Workshop on SDMX based on the model of UNESWA (United Nation's Economic and Social Commission for Western Asia's) for IT staff and statisticians June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP5 – Deliverables Deliverables D.1 A handbook on how to use SDMX D.2 Workshop documentation: agenda, slides, workshop user guide WP5- gantt   M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 M13 M14 M15 M16 M17 M18 D.1 D.2 June 7-8, Luxembourg

Statistical Information System Architecture and Integration Progress Report WP1: Preparation of the kick off meeting and reporting of results (Minutes) Internal meeting between ESSnet on SDMX phase II and CORE ESSnet Presentations: Global conference on SDMX (Washington – May 20110), SISAI meeting WP2: Planning meeting for the working package 7th of April. Management of WP (progress report and Production sheet) Start of the work with test of PX-Edit and the SDMX-RI. June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP3: analyzed (Eurostat) libraries for handling files in XML started the analysis of the Mapping Assistant studied some features of a developed software for micro data Infostat which contains procedures that may be included in X-ECollect involving complex calculations and validations WP4 (Bulgaria): analysis of the documentation on the SDMX-RI beginning of a test project in java for the module "query parser" analysis and tracking of consistency between Documentation and the Source Code of the Query Parser Module June 7-8, Luxembourg

Statistical Information System Architecture and Integration WP4 (Italy): definition of a revised proposal definition of modules to be analyzed (sdmxModel, sdmxIO, QueryParser, DataGenerator) analysis of the documentation on the SDMX-RI analysis of part of the code on the SDMX-RI (SdmxModel) starting in step by step tutorial on SdmxModel WP5: Not yet started June 7-8, Luxembourg

Statistical Information System Architecture and Integration ESSnet on SDMX phase II – ESSnet CORE Meetings between coordinator in Rome the 13 May 2011 Description of the CORE scenario and the phases involved in the implementation of APIs interface (sample allocation, calibration and estimates computation steps) Where SDMX can enter in the CORE scenario? Microdata: proposal to analyze the “Process and Transitions” and “Transformation and expressions” as they have been defined inside the SDMX-IM. Macrodata: dissemination databases conform to the schemas covered by the SDMX-RI June 7-8, Luxembourg