Steering Group Admin Project, 12 May 2016 Big Data Project Steering Group Admin Project, 12 May 2016

Ethics / Communication Action Plan Themes Policy Quality Skills Experience sharing Legislation IT Infrastructures Methods Ethics / Communication Partnerships Pilots This slide shows the topics that have been developed in the action plan and roadmap. The ESSC reinforced in its opinion the legal and ethical issues as well as the development of appropriate skills. Some of the topics should be directly further developed such as policy, communication or skills, while other topics should be further elaborated as part of the pilots that will be run at EU level. These topics are all somehow interrelated, but let me cluster or group them in the next few slides, and highlight some of the concrete actions that are underway. ----------------- NEXT SLIDES: Pilots - Experience sharing – methods Skills & IT infrastructures Legislation – policy 4. Transversal areas: quality framework + ethics/comm.

Pilots Communication Mobile Communication Social Media WWW Web Searches Analysis of websites' contents Job Advertisements Businesses' Websites E-Commerce Real estate Internet Traffic Sensors Traffic loops Smart meters Automatic Vessel Identification System Satellite Images Webcams Process generated data Reservation Systems Flight Booking transactions Trains Hotels Supermarket Cashier Data Loyalty Cards Financial transactions Mobile Payments eGovernment Crowd sourcing Voluntary Geographic Information websites (OpenStreetMap) Voluntary Information websites (Wikipedia) Community pictures collection

Big Data: Service Contract Ethical review and guidelines Communication Strategy Legal Review Development of a training strategy to bridge the Big Data skills gap in European official statistics Workshop on ESS Big Data, 13-14 Oct 2016 http://ec.europa.eu/eurostat/cros/content/ess-big-data-workshop-2016_en Start 1 January 2016 End 1 December 2017

Big Data: Service Contract Ethical implications of the use of Big Data for production of official statistics. access to the data sets the data is owed by the private sector sensitive personal information privacy issues reputation of official statistics …. Review of Code of Practice A coherent and comprehensive set of guidelines easily interpretable and applicable in the NSIs’ daily activities has to be produced (Dec 2016)

Big Data: Service Contract Communication Strategy (Dec 2016) ensure an adequate level of public awareness regarding the use of big data for the purposes of official statistics communicate their values and commitments to respect standing privacy and data protection legislation Long-term instrument SWOT analysis _Strengths/Opportunities Weaknesses/Threats to define solutions for identified weaknesses and threats Target audience and stakeholder analysis Supportive and neutral stakeholders Opposing stakeholders Policy makers Definition of messages and communication channels and tools

Big Data: Service Contract Legal Review A comprehensive survey of current and upcoming legislation within Member States and at EU level that could have an impact on use of big data for statistical purposes (1) to identify cases where using special sources for specific statistical purposes or for official statistics in general would not be consistent with relevant legislation and (2) identify clauses that would allow access and use of special sources explicitly or implicitly. Report on legal review covering basic statistical laws and framework legislations. - Dec 2016 Report on legal review covering other legislations. - Oct 2017

Big Data: Service Contract Development of a training strategy to bridge the Big Data skills gap in European official statistics Identification of skills required for the use of big data sources (May 2016) Skills framework Inventory of existing skills in Eurostat and in the NSIs in Europe (July 2016) Questionnaire to NSIs and Eurostat Analysis of the big data training needs (Oct 2016) To define the training objectives and content (Dec 2016) Competency-Based Education approach Develop a training provision strategy to bridge the skill gap (Feb 2017) e.g. ESTP courses

ESSnet Big Data Supports ESS Big Data Action Plan / BIGD project Preparation starts 2015 based on FPA/SGA construction Big Data pilots 2016-2018 / 2018-2019 “learning by doing” 2015 Framework Partnership Agreement 2016-2018 2 Specific Grant Agreements Exploration of specific data sources e.g. mobile communication data, social media, internet sites, …. Issue of multisource Statistics Contributes to generic frameworks and guidelines e.g. methodology, quality, legislation, IT, communication, architecture, …

Work Packages WP0 Co-ordination WP1 Webscraping / Job Vacancies WP2 Webscraping / Enterprise Characteristics WP3 Smart Meters WP4 AIS Data WP5 Mobile Phone Data WP6 Early Estimates WP7 Multi Domains (WP8 Methodology, foreseen for SGA-2) WP9 Dissemination

Subdivision of Pilots into Phases Data access Conditions; partnerships Data handling Production criteria; micro versus aggregated data; visualisation Methodology and technology Methodology for long lasting statistics; process design Statistical output Examples of existing and new outputs; potential users; comparison with current estimates (quality, timeliness, level of detail) Future perspectives Applicability in ESS; future production process; exploration of further possibilities of using and combining (big) data sources

WP1 Webscraping / Job Vacancies (UK,DE,EL,IT,SE,SI) Aim: “To demonstrate by concrete estimates which approaches (techniques, methodology, etc.) are most suitable to produce statistical estimates in the domain of job vacancies and under which conditions these approaches can be used in the ESS.” Start date: 1.2.2016 End date: 31.7.2017

WP1 Webscraping / Job Vacancies Deliverables SGA-1: 1.1 Inventory and qualitative assessment of job portals 1.2 Interim feasibility report 1.3 Final technical report Milestones SGA-1: 1.4 Progress and technical report of first internal WP meeting 1.5 Idem second internal WP meeting 1.6 Simulated production system deployed 1.7 Simulated production system decommissioned

WP2 Webscraping / Enterprise Characteristics (IT,BG,NL,PL,SE,UK) Aim: “To investigate whether webscraping, text mining and inference techniques can be used to collect, process and improve general information about enterprises. Challenges compared to WP1 are application of more massive scraping of websites and collecting and analysing more unstructured data.” Start date: 1.2.2016 End date: 31.7.2017

WP2 Webscraping / Enterprise Characteristics Deliverables SGA-1: 2.1 Report with legal aspects 2.2 Technical and methodological report describing webscraping, prediction and inference procedures Milestones SGA-1: 2.3 Progress and technical report of first internal WP meeting

WP3 Smart Meters (EE,AT,DK,SE) Aim: “To demonstrate by concrete estimates whether buildings equipped with smart meters can be used to produce energy statistics but can also be relevant as a supplement for other statistics, e.g. census housing statistics, household costs, impact on environment, statistics about energy production.” Start date: 1.2.2016 End date: 31.7.2017

WP3 Smart Meters Deliverables SGA-1: 3.1 Report about data access and data handling 3.2 Report about producing statistics with smart meters data (methodology and first results) Milestones SGA-1: 3.3 Progress and technical report of first internal WP meeting 3.4 Idem second internal WP meeting

WP4 AIS Data (NL,DK,EL,NO,PL) Aim: “To investigate whether real-time measurement data of ship positions (via AIS system) can be used (1) to improve the quality and internal comparability of existing statistics and (2) for new statistical products relevant for the ESS.” Start date: 1.2.2016 End date: 31.7.2017

WP4 AIS Data Deliverables SGA-1: 4.1 Report on creating a database with AIS data for official statistics: possibilities and pitfalls 4.2 Report about deriving harbour visits and linking data from port authorities with AIS data 4.3 Report about sea traffic analyses using AIS data Milestones SGA-1: 4.4 Progress and technical report of first internal WP meeting 4.5 Idem second internal WP meeting

WP5 Mobile Phone Data (ES,FI,FR,IT,RO) Aim: “To exploit this source to produce estimates of day-time populations as a mature official statistical product.” And: “(i) to gather the current experiences about data access to mobile phone data in the ESS, (ii) to obtain access to the mobile phone data for their exploitation in the 2nd period of the project and (iii) to propose some specific complementary potential outputs for official statistics based on this source of information.” Start date: 1.2.2016 End date: 31.12.2016

WP5 Mobile Phone Data Deliverables SGA-1: 5.1 Report on current status of data access to mobile phone data in the ESS 5.2 Report with recommendations for (minimum) requirements and key issues to data access to mobile phone data in order to produce statistical outputs in the field of population dynamics Milestones SGA-1: 5.3 Organisation of a meeting with telecom-providers at European level

WP6 Early Estimates (SI, FI, NL, PL) Aim: “To investigate multiple big data, administrative and other existing sources in order to produce early estimates for statistical purposes. For those sources, which will be determined to have the greatest potential, the WP team will determine the business cases related to statistics.” Start date: 1.2.2016 End date: 28.2.2017

WP6 Early Estimates Deliverables SGA-1: 6.1 List of potential big data sources and business cases for early estimates 6.2 Recommendations IT tools data collection Consumer Confidence Index and Nowcasts turnover indices 6.3 Recommendations methodology data processing Consumer Milestones SGA-1: 6.4 Interim feasibility report 6.5 Progress and technical report of internal WP meeting

WP7 Multi Domains (PL,NL,UK) Aim: “To find out how a combination of big data sources, administrative data and statistical data may enrich statistical output in the domains: Population, Tourism / border crossings and Agriculture.” Start date: 1.2.2016 End date: 28.2.2017

WP7 Multi Domains Deliverables SGA-1: 7.1 Report for the domain of Populations with information on: Data access (with legal/privacy aspects) Data quality issues Methodology (focus on combining data) Technical aspects 7.2 Report for the domain of Tourism / Border crossings (idem) 7.3 Report for the domain of Agriculture (idem) Milestones SGA-1: 7.4 List of available big data sources (in the domains) 7.5 Recommendation for using two or three big data sources (in the domains) 7.6 Progress and technical report of internal WP meeting

WP9 Dissemination (BE,BG,NL) Aim: Ensuring the exchange of information among participants Providing necessary tools and training to participants Disseminating results as best practices to the ESS and wider community of official statistics Disseminating results to anyone else potentially interested https://webgate.ec.europa.eu/fpfis/mwikis/essnetbigdata/index.php/Main_Page Start date: 1.2.2016 End date: 31.7.2017