1. Steering Group Admin Project, 12 May 2016
Big Data Project
Steering Group Admin Project,
12 May 2016

2. Ethics / Communication
Action Plan Themes
Policy
Quality
Skills
Experience sharing
Legislation IT
Infrastructures
Methods
Ethics / Communication
Partnerships
Pilots
This slide shows the topics that have been developed in the action plan and roadmap. The ESSC reinforced in its opinion the legal and ethical issues as well as the development of appropriate skills.
Some of the topics should be directly further developed such as policy, communication or skills, while other topics should be further elaborated as part of the pilots that will be run at EU level.
These topics are all somehow interrelated, but let me cluster or group them in the next few slides, and highlight some of the concrete actions that are underway.
NEXT SLIDES:
Pilots - Experience sharing – methods
Skills & IT infrastructures
Legislation – policy
4. Transversal areas: quality framework + ethics/comm.

3. Pilots Communication Mobile Communication Social Media WWW
Web Searches
Analysis of websites' contents
Job Advertisements
Businesses' Websites
E-Commerce
Real estate
Internet Traffic
Sensors
Traffic loops
Smart meters
Automatic Vessel Identification System
Satellite Images
Webcams
Process generated data
Reservation Systems
Flight Booking transactions
Trains
Hotels
Supermarket Cashier Data
Loyalty Cards
Financial transactions
Mobile Payments
eGovernment
Crowd sourcing
Voluntary Geographic Information websites (OpenStreetMap)
Voluntary Information websites (Wikipedia)
Community pictures collection

4. Big Data: Service Contract
Ethical review and guidelines
Communication Strategy
Legal Review
Development of a training strategy to bridge the Big Data skills gap in European official statistics
Workshop on ESS Big Data, Oct 2016
Start 1 January 2016
End 1 December 2017

5. Big Data: Service Contract
Ethical implications of the use of Big Data for production of official statistics.
access to the data sets
the data is owed by the private sector
sensitive personal information
privacy issues
reputation of official statistics ….
Review of Code of Practice
A coherent and comprehensive set of guidelines easily interpretable and applicable in the NSIs’ daily activities has to be produced (Dec 2016)

6. Big Data: Service Contract
Communication Strategy (Dec 2016)
ensure an adequate level of public awareness regarding the use of big data for the purposes of official statistics
communicate their values and commitments to respect standing privacy and data protection legislation
Long-term instrument
SWOT analysis _Strengths/Opportunities Weaknesses/Threats to define solutions for identified weaknesses and threats
Target audience and stakeholder analysis
Supportive and neutral stakeholders
Opposing stakeholders
Policy makers
Definition of messages and communication channels and tools

7. Big Data: Service Contract
Legal Review
A comprehensive survey of current and upcoming legislation
within Member States and at EU level that could have an impact on use of big data for statistical purposes
(1) to identify cases where using special sources for specific statistical purposes or for official statistics in general would not be consistent with relevant legislation and
(2) identify clauses that would allow access and use of special sources explicitly or implicitly.
Report on legal review covering basic statistical laws and framework legislations. - Dec 2016
Report on legal review covering other legislations. - Oct 2017

8. Big Data: Service Contract
Development of a training strategy to bridge the Big Data skills gap in European official statistics
Identification of skills required for the use of big data sources (May 2016)
Skills framework
Inventory of existing skills in Eurostat and in the NSIs in Europe (July 2016)
Questionnaire to NSIs and Eurostat
Analysis of the big data training needs (Oct 2016)
To define the training objectives and content (Dec 2016)
Competency-Based Education approach
Develop a training provision strategy to bridge the skill gap (Feb 2017)
e.g. ESTP courses

9. ESSnet Big Data Supports ESS Big Data Action Plan / BIGD project
Preparation starts 2015 based on FPA/SGA construction
Big Data pilots /
“learning by doing”
2015 Framework Partnership Agreement
Specific Grant Agreements
Exploration of specific data sources
e.g. mobile communication data, social media, internet sites, ….
Issue of multisource Statistics
Contributes to generic frameworks and guidelines
e.g. methodology, quality, legislation, IT, communication, architecture, …

10. Work Packages
WP0 Co-ordination WP1 Webscraping / Job Vacancies WP2 Webscraping / Enterprise Characteristics WP3 Smart Meters WP4 AIS Data WP5 Mobile Phone Data WP6 Early Estimates WP7 Multi Domains (WP8 Methodology, foreseen for SGA-2) WP9 Dissemination

11. Subdivision of Pilots into Phases
Data access
Conditions; partnerships
Data handling
Production criteria; micro versus aggregated data; visualisation
Methodology and technology
Methodology for long lasting statistics; process design
Statistical output
Examples of existing and new outputs; potential users; comparison with current estimates (quality, timeliness, level of detail)
Future perspectives
Applicability in ESS; future production process; exploration of further possibilities of using and combining (big) data sources

12. WP1 Webscraping / Job Vacancies (UK,DE,EL,IT,SE,SI)
Aim: “To demonstrate by concrete estimates which approaches (techniques, methodology, etc.) are most suitable to produce statistical estimates in the domain of job vacancies and under which conditions these approaches can be used in the ESS.” Start date: End date:

13. WP1 Webscraping / Job Vacancies
Deliverables SGA-1: 1.1 Inventory and qualitative assessment of job portals 1.2 Interim feasibility report 1.3 Final technical report Milestones SGA-1: 1.4 Progress and technical report of first internal WP meeting 1.5 Idem second internal WP meeting 1.6 Simulated production system deployed 1.7 Simulated production system decommissioned

14. WP2 Webscraping / Enterprise Characteristics (IT,BG,NL,PL,SE,UK)
Aim:
“To investigate whether webscraping, text mining and inference techniques can be used to collect, process and improve general information about enterprises. Challenges compared to WP1 are application of more massive scraping of websites and collecting and analysing more unstructured data.”
Start date:
End date:

15. WP2 Webscraping / Enterprise Characteristics
Deliverables SGA-1: 2.1 Report with legal aspects 2.2 Technical and methodological report describing webscraping, prediction and inference procedures Milestones SGA-1: 2.3 Progress and technical report of first internal WP meeting

16. WP3 Smart Meters (EE,AT,DK,SE)
Aim:
“To demonstrate by concrete estimates whether buildings equipped with smart meters can be used to produce energy statistics but can also be relevant as a supplement for other statistics, e.g. census housing statistics, household costs, impact on environment, statistics about energy production.”
Start date:
End date:

17. WP3 Smart Meters
Deliverables SGA-1: 3.1 Report about data access and data handling 3.2 Report about producing statistics with smart meters data (methodology and first results) Milestones SGA-1: 3.3 Progress and technical report of first internal WP meeting 3.4 Idem second internal WP meeting

18. WP4 AIS Data (NL,DK,EL,NO,PL)
Aim:
“To investigate whether real-time measurement data of ship positions (via AIS system) can be used
(1) to improve the quality and internal comparability of existing statistics and
(2) for new statistical products relevant for the ESS.”
Start date:
End date:

19. WP4 AIS Data Deliverables SGA-1:
4.1 Report on creating a database with AIS data for official statistics: possibilities and pitfalls
4.2 Report about deriving harbour visits and linking data from port authorities with AIS data
4.3 Report about sea traffic analyses using AIS data
Milestones SGA-1:
4.4 Progress and technical report of first internal WP meeting
4.5 Idem second internal WP meeting

20. WP5 Mobile Phone Data (ES,FI,FR,IT,RO)
Aim:
“To exploit this source to produce estimates of day-time populations as a mature official statistical product.” And: “(i) to gather the current experiences about data access to mobile phone data in the ESS, (ii) to obtain access to the mobile phone data for their exploitation in the 2nd period of the project and (iii) to propose some specific complementary potential outputs for official statistics based on this source of information.”
Start date:
End date:

21. WP5 Mobile Phone Data
Deliverables SGA-1: 5.1 Report on current status of data access to mobile phone data in the ESS 5.2 Report with recommendations for (minimum) requirements and key issues to data access to mobile phone data in order to produce statistical outputs in the field of population dynamics Milestones SGA-1: 5.3 Organisation of a meeting with telecom-providers at European level

22. WP6 Early Estimates (SI, FI, NL, PL)
Aim:
“To investigate multiple big data, administrative and other existing sources in order to produce early estimates for statistical purposes. For those sources, which will be determined to have the greatest potential, the WP team will determine the business cases related to statistics.”
Start date:
End date:

23. WP6 Early Estimates
Deliverables SGA-1: 6.1 List of potential big data sources and business cases for early estimates 6.2 Recommendations IT tools data collection Consumer Confidence Index and Nowcasts turnover indices 6.3 Recommendations methodology data processing Consumer Milestones SGA-1: 6.4 Interim feasibility report 6.5 Progress and technical report of internal WP meeting

24. WP7 Multi Domains (PL,NL,UK)
Aim:
“To find out how a combination of big data sources, administrative data and statistical data may enrich statistical output in the domains: Population, Tourism / border crossings and Agriculture.”
Start date:
End date:

25. WP7 Multi Domains Deliverables SGA-1:
7.1 Report for the domain of Populations with information on:
Data access (with legal/privacy aspects)
Data quality issues
Methodology (focus on combining data)
Technical aspects
7.2 Report for the domain of Tourism / Border crossings (idem)
7.3 Report for the domain of Agriculture (idem)
Milestones SGA-1:
7.4 List of available big data sources (in the domains)
7.5 Recommendation for using two or three big data sources (in
the domains)
7.6 Progress and technical report of internal WP meeting

26. WP9 Dissemination (BE,BG,NL)
Aim:
Ensuring the exchange of information among participants
Providing necessary tools and training to participants
Disseminating results as best practices to the ESS and wider community of official statistics
Disseminating results to anyone else potentially interested
Start date:
End date: