ESTAT International Seminar on Modernizing Official Statistics: Meeting Productivity and New Data Challenges Tianjin, People’s Republic of China October 2013 Meeting Challenges of Unstructured Data: European Statistical System experience Mariana Kotzeva, Advisor Hors Classe, Eurostat Eurostat

This presentation: describes the current state and next steps with regard to big data in the European statistical system; Outlines 3 concrete projects related to big data launched by Eurostat in 2013; Outlines challenges ahead putting a focus on the issues that represent unique opportunity for co-operation at supranational and global level. ESTATEurostat Mariana Kotzeva, Eurostat

Unstructured data – what does it mean? Data that are not arranged in columns and rows, not properly titled or identified; Data for which it is necessary to apply tools to categorise, tag, pool, identify key elements and make correlations in order to gain insight from them; Content Unstructured data: refers to s, documents and other objects that are made up of free-following text; ESTATEurostat Mariana Kotzeva, Eurostat

ESTAT Big data? Volume Velocity Variety Eurostat Mariana Kotzeva, Eurostat

Is unstructured data the same as big data? Unstructured data is just one piece of the puzzle; Unstructured data is responsible for the Variety characteristic of the big data; Making sense of unstructured data isn’t about technology, it is a business intelligence; To transfer into actionable intelligence is the value added that unstructured data bring to big data. ESTATEurostat Mariana Kotzeva, Eurostat

ESTATEurostat Mariana Kotzeva, Eurostat Big data is high on the agenda of the European Statistical System: Eurostat participates in the UNECE initiatives under the High Level Group on Modernisation of Official Statistics; Memorandum on Big Data and Official statistics of the ESS, adopted at 99 th DGINs conference in September 2013; Eurostat created an internal structure that will deal only with big data since 1 st of Jan 2014; A Task Force on big data involving Member states is in process of establishment for developing an ESS action plan and roadmap on big data;

ESTATEurostat Mariana Kotzeva, Eurostat European projects on big data launched in 2013 Internet as a data Source for Information Society statistics; Use of mobile positioning data for tourism statistics; Collecting prices via the Internet for price statistics;

Internet as a data source for Information Society statistics Current situation in Information society statistics: Based on two traditional surveys - on households and on enterprises; Purpose of the feasibility study: Which items could be collected over the Internet? Results: Items such as e-skills, e-commerce, use of e-government, use of internet is feasible to be measured over Internet; Items measuring subjective opinions and “offline” items CANNOT be measured using digital footprints; For enterprises, part of the ICT indictors would be available on the enterprise’ websites but some of them will be available only via the servers of the enterprises; Survey suggests a number of new indicators that can be derived form Internet ESTATEurostat Mariana Kotzeva, Eurostat

Use of mobile positioning data for tourism statistics Active mobile positioning; Passive mobile positioning. ESTATEurostat Mariana Kotzeva, Eurostat

Issues identified so far: Reluctance of data providers to give access to data; Privacy concerns; Technological barriers; Diversity in data protection legislation; Diversity in the data sources that introduces even higher complexity in programming generic software tools; Methodological and quality issues- i.e comparability versus relevance. ESTATEurostat Mariana Kotzeva, Eurostat

The way forward: Official statistics community – now on the move! Collaboration: Big Data goes beyond statistics authorities  national/”government” strategies needed Public-Private Partnerships (could help build-up) Multidisciplinary teams – including legal expertise Action: Demystify “data science” and build up skills Applications-driven approach and learning by doing. ESTATEurostat Mariana Kotzeva, Eurostat

Thank you for your attention!   ESTATEurostat