WP8 Methodology (SGA2) Piet Daas NL, AT, BG, IT, PT, PL, SL

Why Big Data? Possibilities: Shorter time to publication Respond to current events Higher reliability More detail More efficient processes Considerations: Competences Culture Methods Quality Infrastructure

SGA2: WP8 Methodology (and more) WP8 is about: Methodology Quality IT Goal is to generalize findings What kind of methodology do we need when using Big Data for official statistics? What kind of quality checks do we perform when using Big Data for official statistics? What kind of IT-environment do we need when using for Big Data for official statistics?

A reflection A quote: “Big Data methodology does not exist” Strange as a whole new approach was developed to process, check and edit road sensor data and estimate traffic intensities based on these Conclusion: Risk of to much ‘hearsay’ Risk of thinking in the old paradigm All WP8 work should be fact based! Use these facts to develop Big Data theory!

Starting point Start with the work done in SGA1 of ESSnet on Big Data Interviewed: WP1 Web scraping jobs WP2 Web scraping enterprises WP3 Smart meters WP4 AIS WP5 Mobile phones WP6 Early estimates WP7 Statistical domains (3 cases)

Next steps Input from any other work done Are there Big Data based official statistics? At least two examples Scanner data based CPI Traffic intensity statistics using road sensor data Any more ..? ESS VIP ADMIN findings …. Keep the link with type of Big Data source in mind! - Machine2machine, admin like, human derived (UN) - Do they need different methods, quality checks, IT?

Combine all findings After combining findings, organize the WP8 workshop Invite WP8 members Invite representatives of WP 1 t/m 7 Invite representatives from 22 partners involved (if not in 1 or 2) Invite Eurostat representatives (if not in 1) Invite experts not yet included (not in 1, 2, 3, 4) (max of 25-30 people) Meet in person and discuss methodology, quality and IT in context of Big Data Ultimate goal is to obtain consensus on these topics (or at least narrow the field, focus is essential)

Output of WP8 Literature overview (month 5) Big Data and IT (month 9) include most relevant Big Data studies Big Data and IT (month 9) Quality and Big Data (month 13) Big Data Methodology (month 17)

Questions? 9

The Future The future of statistics looks BIG

Thank you for your attention! @pietdaas