Ioannis Xirouchakis / Unit B3 The ICT Hub Joint meeting of the ESS.VIP.BUS ICT Project and the ESS.VIP IT TF Luxembourg, 3 March 2014 Ioannis Xirouchakis / Unit B3

Completed activities 1/3 ICT Hub GUI implementation (basic version) Collection and drafting of requirements Implementation of the requirements Deployment and testing Establishment of central node Establishment of new hosting environment Deployment of ICT Hub application and administration tools

Completed activities 2/3 Establishment of country nodes Test datasets (starting packages) preparation for the 11 pilot countries Guidelines preparation for the pilot countries for re-using the SDMX-RI Implementation and deployment of country nodes: Austria, Cyprus, Finland, Ireland, Italy, Netherlands, Poland, Slovenia and Spain Connection establishment between country nodes and ICT Hub

Completed activities 3/3 Analysis of on-the-fly aggregation of statistics Design of scenarios and tests for extending the ICT Hub to support on-the-fly aggregations Testing of the scenarios Preliminary analysis of results Outlining the Enhanced Hub (as an extension to the Census and ICT Hubs)

The Hub logic

Advantages of the Hub logic End-user obtains the latest data revision available No voluminous data transmissions to central node No need to maintain large central database Changes reflected on the Hub immediately after submission at country level Countries own their data and assume responsibility Countries revise data without technical impact on the Hub or other country nodes Countries don't change the structure of their dissemination databases

Extensibility of the Hub logic Very limited technical effort required to add another country to the Hub Limited effort required to expand the system to include new datasets (corresponding to new DSDs and codelists) The same dissemination databases can be made available to other Hubs with limited effort (for data mapping mainly)

Challenges of the Hub logic Data availability on the Hub depends on country node availability Response to queries can be slower compared to the traditional approach: the Hub needs to collect data from country nodes Data validation takes place only once: at country level

The ICT Hub GUI Assumptions for basic version (Phase I) Reuse of the existing Census Hub application Reuse of the SDMX Reference Infrastructure Dissemination of published data No availability requirements for country nodes Extension/adaptation to ICT domain in terms of structural metadata (dataflows, DSDs, codelists) Only minor effort invested for modification/adaptation of the Census Hub GUI

Conclusions With the work package complete All objectives of the work package were achieved by the pilot countries and Eurostat Extension of the Hub logic to other statistical domains is possible with minor technical effort Certain investment is necessary (mainly) for adaptation of the GUI to another statistical domain The particularities of each statistical domain can play a role in the selection of the Hub logic for dissemination

Live demo of the ICT Hub [link]

Sample workflow 1 On Select dataset screen Indicator(s) pane  select E_ADE (not a mandatory step for selecting dataflow) On Selected dataflow pane the colour of relevant dataflows changes  select isoc_bde15disc and press Select dataflow On Specify dimensions screen choose: Under GEO: e.g. Spain, Italy, Cyprus, Poland, Slovenia, Finland Under INDIC_IS: E_ADE (other indicators in the same dataflow also appear) Under SIZEN_R2: 10_C10_S951_XK, 10_C10_N82_XK, 10_BB, 10_NBB Under TIME: click all entries Under UNIT: PC_ENT (the system proposes all units relevant to the indicator) Click Select layout Click View data the maximum response time is 20 seconds (timeout setting) and it depends on the slowest country response

Sample workflow 2 On Select dataset screen Indicator(s) pane  click Code radio-button and select E_BROAD and E_IBUY On Selected dataflow pane select isoc_ci_it_en2 and press Select dataflow On Specify dimensions screen choose: Under GEO: e.g. Ireland, Spain, Italy, Cyprus, Netherlands, Austria, Poland Under INDIC_IS: E_BBOTH, E_BROAD, E_BROAD2, E_DSL Under SIZEN_R2: 10_ICT_T, 10_K6FINS Under TIME: 2012 Under UNIT: PC_ENT, PC_ENT_IACC Click Select layout Click View data several countries return [0] when a record does not exist in their dissemination database, while the expected result would be [:] or [] this is a Census Hub particularity and can be modified

Sample workflow 3 On Select dataset screen Selected dataflow pane  click Code radio-button, select isoc_sk_itsp_en2 and press Select dataflow On Specify dimensions screen choose: Under GEO: any countries Tab INDIC_IS is now absent (structural metadata form the GUI dynamically) Under SIZEN_R2: select all Under TIME: 2012 Under UNIT: PC_ENT, PC_ENT_CUSE Click Select layout Click View data Click Download Save data in the desired format (XLS, CSV, SDMX)

Sample workflow 4 On Select dataset screen Selected dataflow pane  select isoc_bde15disc and press Select dataflow On Specify dimensions screen choose: Under INDIC_IS select E_SISC  under UNIT units PC_ENT and PC_ENT_CUSE are proposed Under INDIC_IS select E_SISC, E_SISUCU  under UNIT units PC_ENT, PC_ENT_CUSE, PC_ENT_SISC are proposed Under INDIC_IS select E_SISC, E_SISUCU, E_ADECU  under UNIT all units are proposed Under GEO: any countries Under SIZEN_R2: 10_BB, 10_NBB Under TIME: 2012 Click Select layout Click View data

Thank you for your attention! Contact: Ioannis.Xirouchakis@ec.europa.eu Unit B3, Eurostat