ESSnet on Big Data: preparation SGA

2 Agenda Thursday 07.01.2016 13:30 Room open 14:00 Welcome Marc Debusschere 14:10 Agenda Pieter Vlag PART 1. General approach 14:15 SGA-1 ESSnet Big Data: general approach Pieter Vlag 14:45 Questions and comments PART 2. Work packages linked to webscraping 15:00 presentation WP-1 job vacancies Nigel Swier (replacement) 15:15 presentation WP-2 scraping websites for enterprise characteristics Monica Scannapieco (repl.) 15:30 Questions or comments PART 3. Mobile phone data 15:45 presentation WP-5 mobile phone data David Salgado 16:00 Questions or comments PART 4. Smart Meters : electricity 16:15 presentation WP-3 smart Meters Maiki Ilves 16:30 presentation WP-4 AIS-data Piet Daas or Pieter Vlag 16:45 Questions or comments PART 5. Summary of 1st days 17:00 Discussion of workpackages in groups 18:00 Presentation of group results PART 6. Social event in Brussels ESSnet AdminData WP 4 2 2

3 Agenda Friday 08.01.2016 PART 6. Summary of 1st day lusions and results Agenda Friday 08.01.2016 PART 6. Summary of 1st day 09:00 summary of 1st days Pieter Vlag PART 7. Combining sources 09:30 presentation WP 7 combining sources STS Boro Nikic 09:45 presentation WP 8 combining sources SBS Anna Nowicka 10:00 Discussion PART 8. Dissemination 10:30 Presentation Marc Debusschere 10:45 Discussion PART 8. General issues 11:00 Intra-consortium agreement Meetings Review Board Contact with stakeholders Steps until submission SGA-1 by 31.01.2015 PART 9. END OF MEETING 13:00 Conclusions and results ESSnet AdminData WP 4 3 3

ESSnet on Big Data: SGA -1 Part – 1 : general approach

5 Background Invitation to organise a consortium July 2015 Preparation meeting FPA 4 September 2015 (Amsterdam) FPA signed 18.11.2015 Invitation for first SGA Early December Deadline for submission SGA-1 31.01.2016 ESSnet AdminData WP 4 5 5

Key message of this ESSnet Concrete pilots about use of Big Data for official statistics ! From exploration to exploitation ESSnet AdminData WP 4 6 6

Key message of this ESSnet Concrete pilots ! FROM EXPLORATION TO EXPLOITATION ! 5. Front viewing 4. Statistical outputs 3. Methodology and Techniques 2. Data handling ESSnet AdminData WP 4 1. Data acces 7 7

8 Pilots j.Vacancies ent.charact electricity ship position temp. pop. scraping ent.charact electricity smart meters ship position AIS temp. pop. mob. phones. Nigel Swier Monica Scannapieco Maiki Ilves Paul Esser David Salgado Combining sources STS Combining sources SBS Boro Nikic Anna Nowicka ESSnet AdminData WP 4 8 8

Pilots ESSnet AdminData WP 4 9 9

WP 1: job vacancies Aim: To demonstrate which approaches are most suitable for the production of statistical estimates in the domain of job vacancies To establish how these approaches could be used within the ESS Potential Sources: Job portals Jobs advertised on enterprise websites Third party data providers ESSnet AdminData WP 4 10 10

WP 2: enterprise characteristics The general aim is to investigate whether web scraping, in connection with techniques as text mining and machine learning, can be used to collect information concerning enterprises. This can be done at different levels: at micro level, in order to enrich the information in business registers; at aggregate level, in order to produce statistical output. At both levels, quality (accuracy) of results needs to be evaluated. While (1) is the primary target of this pilot, also (2) is relevant, and can be carried out jointly with (1). ESSnet AdminData WP 4 11 11

WP 3: electricity Aim of this pilot is to demonstrate by concrete estimates whether buildings equipped with smart meters ( = electricity meters which can be read from a distance and measure electricity consumption at a high frequency) can be used to produce energy statistics vacant houses household costs Strong point: clear plan, clear aim Weak point: one dataset (Estonia) ESSnet AdminData WP 4 12 12

13 WP 4: ship positions Use data about position of ships for Prototype reference frame of ships in european waters Traffic analyses Emission calculations Data collection: using dataset at national level (no/low costs) using one dataset at European level (high costs) ESSnet AdminData WP 4 13 13

14 WP 5: Mobile phone data Consensus Development of a business plan needed Possible hypothesis for output pilot: estimate variations in population for public needs Need to make link between output pilot and ‘required’ cleaned microdata from MNOs Link made by NSIs Link made by subcontractor Negociate with interested MNOs If data (plus budget) avialable pilot in 2nd SGA. ESSnet AdminData WP 4 14 14

Organisation ESSnet BD Lilli Japec; Piet Daas; + Review board Co-ordination Pieter Vlag; Martin van Sebille j.Vacancies scraping ent.charact electricity smart meters ship position AIS temp. pop. mob. phones. Combining sources STS Combining sources SBS Dissemination ESSnet AdminData WP 4 15 Marc Debusschere 15

Co-ordination group (management board) Co-ordinator WP-leader 1 WP-leader 2 WP-leader x Chair review board Country co-ordinator: responsible for resources ESSnet AdminData WP 4 16 16

17 Partners Partners ESSnet AdminData WP 4 17 List of Partners Partner name Partner's acronym Country 1 (Co-ordinator) Statistics Netherlands CBS NETHERLANDS (NL) 2 (Co-partner) Statistik Austria STAT AUSTRIA (AT) 3 (Co-partner) Statistics Belgium SB BELGIUM (BE) 4 (Co-partner) Bulgarian National Statistical Institute BNSI BULGARIA (BG) 5 (Co-partner) Statistisches Bundesamt DESTATIS GERMANY (DE) 6 (Co-partner) Statistics Denmark DS DENMARK (DK) 7 (Co-partner) Statistics Estonia EE ESTONIA (EE) 8 (Co-partner) Institut National De La Statistique Et Des Etudes Economiques  INSEE FRANCE (FR) 9 (Co-partner) Ministery Of Labour - France DARES 10 (Co-partner) Ministery Of Energy - France SOES 11 (Co-partner) Statistics Finland SF FINLAND (FI) 12 (Co-partner) Hellenic Statistical Authority ELSTAT GREECE (EL) 13 (Co-partner) National Institute Of Statistics ISTAT ITALY (IT) 14 (Co-partner) Central Statistical Office CSO IRELAND (IE) 15 (Co-partner) Statistics Norway SSB NORWAY (NO) 16 (Co-partner) Central Statistical Office Of Poland GUS POLAND (PL) 17 (Co-partner) Instituto Nacional De Estatistica INE PORTUGAL (PT) 18 (Co-partner) INS ROMANIA (RO) 19 (Co-partner) Statistical Office of the Republic of Slovenia SURS SLOVENIA (SL) 20 (Co-partner) Instituto Nacional de Estadistica SPAIN (ES) 21 (Co-partner) Statistics Sweden SCB SWEDEN (SE) 22 (Co-partner) Office for National Statistics ONS UNITED KINGDOM (UK) Partners ESSnet AdminData WP 4 17 17

SGA-1 versus SGA-2 ESSnet AdminData WP 4 18 18

SGA-1 planning ESSnet AdminData WP 4 19 19

Visualisation of 1st results SGA – 1 meetings Visualisation of 1st results ESSnet AdminData WP 4 20 20

21 SGA-1 meetings Co-ordination group meetings 21/22 June 2016 : Tallinn 17/18 November 2016 : Brussel Workshop (22) 23,23 February 2017 : ? The co-ordination group meeting are preceeded by a meeting between WP-member. WP-leaders have budget for an extra travel to members. ESSnet AdminData WP 4 21 21

22 Collaboration required between WPs and between ESSnet + stakeholders Two reasons: fast developments; thight budget ESSnet AdminData WP 4 22 22

Budget vs. Ambition Budget is low compared to ambitions: Needs reduction by 15 - 20 % compared to SGA-1 draft 1 Strategy: obtain more information about grades of participants more synergy of work in and between WPs more focuss in participation of individual countries prioritise WPs - EuroStat priorities - feasibility Transfer work from SGA-1 to SGA-2 if not enough: equal (percentual) reduction of budget for each participant

Reminder When thinking about budget please remind that work has to be done and you learn from collaboration, too

25 Aim of this meeting present individual WPs and recieve feedback (comments vs. ‘amendments’). `discuss the dissemination strategy present general issues and decide about the ESSnet AdminData WP 4 25 25

Aim of this WP-meeting (II) Celebrate the start of a succesfull project later Crucial decisions will be confirmed Do the hard work now; Take the difficult decisions now ESSnet AdminData WP 4 26 26

Decisions OUTPUTS OF THIS MEETING WILL BE TAKEN INTO ACCOUNT WHEN FINALISING SGA-1 ESSnet AdminData WP 4 27 27

Next steps after meetings 1st week after meeting: finalising 2nd draft with more details about costs + relationships between WPs 2nd week after meeting: approval from partners (last possibility to withdraw from SGA-1) 31.01.2016 Submission ULTIMATE AIM: START AN EXCITING PROJECT ESSnet AdminData WP 4 28 28

29 ULTIMATE AIM: START AN EXCITING PROJECT 1st week after meeting: finalising 2nd draft with more details about costs + relationships between WPs 2nd week after meeting: approval from partners (last possibility to withdraw from SGA-1) 31.01.2016 Submission ULTIMATE AIM: START AN EXCITING PROJECT START AN EXCITING AND USEFUL PROJECT ESSnet AdminData WP 4 29 29