1. United Nations Economic Commission for Europe Statistical Division The Data Deluge: What Does It Mean for Official Statistics? Steven Vale UNECE steven.vale@unece.org

2. Contents  What is the Data Deluge?  Response of the official statistics community  Industrialisation – a related challenge  Changing roles for national statistical organisations

3. What is the Data Deluge? The internet has 1800 exabytes of data in 2011 exa = 10^18

4. 50,000 exabytes by 2020 We live in exponential times! 27 fold growth in the next 9 years

5. Are these data interesting?  Probably 99.9% are videos, photos, audio files, text messages and other nonsense  But that still leaves 1,800,000,000,000,000,000 bytes of potentially relevant data

6. An Observation More and more people post information about themselves on Facebook, but less and less complete statistical surveys Should we create STATVILLE?

7. Private sector competitors?  Google: Real-time price indices Public Data Explorer First point of reference for the “data generation”  Facebook, store cards, credit agencies,... What if they link their data?  The private sector now understands the value of data: Can they beat us at our own game?

8.  High-Level Group for Strategic Directions in Business Architecture in Statistics  UNECE group, created by the Conference of European Statisticians in 2010  8 heads of national and international statistical organisations  Develop and promote new: Sources Processes Products Official Statistics Response

9.  Endorsed by the Conference of European Statisticians on 14 June We have to re-invent our products and processes and adapt to a changed world HLG-BAS Strategic Vision

10. The Challenges are too big for statistical organisations to tackle on their own. We need to work together

11. What does this mean in practice?  Collaboration  Coordination  Communication

12. Many international groups and projects are talking about streamlining and industrialising statistics

14. Industrialisation is:  Common processes  Common tools  Common methodologies  Recognising that all statistics are produced in a similar way: No domain is “special”  Increased flexibility to adapt to new sources and produce new outputs

15. Changing roles in NSOs?  One source = one output  Data integration – multiple sources  Process quality assurance  More focus on analysis and interpretation  Partnerships for dissemination  Changing staff and cost profiles  Changing organisational culture

16. Wider Definition of Admin Data?  New sources are “non-statistical”  But - similar issues to “traditional” administrative data sources Whatever we call the new sources, we can’t ignore them!

17. Which new sources are you going to check out?

18. Questions? steven.vale@unece.org www1.unece.org/stat/platform/display/hlgbas steven.vale@unece.org www1.unece.org/stat/platform/display/hlgbas