1. Big Data and Official Statistics The UN Global Working Group
Ronald Jansen
Chief, Trade Statistics Branch
United Nations Statistics Division
and

2. Overview What is Big Data? What is Big Data for Official Statistics?
What are the Challenges?
UN Global Working Group on
Big Data for Official Statistics

3. What is Big Data? Big Data characteristics Volume Velocity Variety
Big Data benefits
Auto-generated
Timely
Granular
Big Data sources:
Mobile Devices
Digital Transactions
Sensors
Social Networking

4. Big Data for Development
UN Global Pulse
Research & Development
Big Data Partnerships
Pulse Lab Network
New York
Jakarta
Kampala

5. Big Data for Development
UN Global Pulse
Early warning
Real-time awareness
Real-time feedback

6. What is Big Data for Official Statistics?
Big Data sources / Statistics
Mobile Phone Positioning Data / Tourism statistics
Supermarket scanner data / Price statistics
Vehicle tracking device / Transport statistics
Twitter/ Consumer confidence statistics
Satellite imagery / Agriculture statistics

7. Big Data Challenges Methodology Privacy IT infrastructure Human skills
What are the Challenges?
Big Data Challenges
Methodology
Privacy
IT infrastructure
Human skills
Partnerships

8. Volatility (no time series?) Standardization ? Modelling
Methodological Challenges
Methodology
Representativeness ?
Volatility (no time series?)
Standardization ?
Modelling

9. Partnership Challenges
Big Data partnerships opportunities
Big Data providers
Data Sources
Research institutes and technology providers
- Upstream processing
- IT infrastructure,
Academia and scientific communities
- Analytical capacity and modelling
National Statistical Offices
- Standards and methodology

10. Example of use of Mobile Phone data for Official Statistics

11. Mobile Phone Positioning Data
Call Detail Record (CDR)
Data Detail Record (DDR) (Internet Protocol Detail Record)
Location Updates (LA)
Radio coverage updates (Abis data)

12. National Statistical Office
Example 1: Estonia
Data provider/ source
Mobile phone operator
Provides raw data to research company
Research company
Provides IT infrastructure
Preparation of data
Ensuring privacy and confidentiality
Academia
Providing modeling and analytics
National Statistical Office
Receives pre-processed and aggregated data
Prepares and disseminates statistics
Shared responsibility for data processing and analysis

13. National Statistical Office
Example 2: Netherlands
Responsibility for data
Data source/ provider
Vodafone
Makes data available to research institute
Research company
Intermediate commercial partner
Provides IT platform for data filtering and privacy
National Statistical Office
Receives pre-processed Big Data
Analyze, prepare and disseminate statistics

14. UN Global Working Group on Big Data for Official Statistics

15. Milestones Feb 2013 Friday Seminar on Big Data
Mar 2014 Report to Statistical Commission
Decision 45/110 – GWG on Big Data
Oct 2014 Conference on Big Data – Beijing
Mar 2015 Report of GWG to Stat Commission
Oct 2015 Conference on Big Data – Abu Dhabi

16. UN Global Working Group on Big Data
Mandate (Decision 45/110 – 2014)
Provide strategic vision, direction and coordination of a global programme on Big Data for official statistics;
Promote practical use of sources of Big Data for official statistics, while building on the existing precedents in Big Data and finding solutions for
Methodological issues,
Legal issues of access to data sources;
Privacy issues
Data security issues;
Cost benefit analysis

17. UN Global Working Group on Big Data
Mandate
(continue)
Promote capacity building, training and sharing of experience;
Foster communication and advocacy of use of Big Data for policy applications;
Build public trust in the use of private sector Big Data for official statistics including issues of confidentiality and privacy.

18. International Organizations
Composition of the GWG
Countries
Australia
Denmark
Italy
Mexico
Netherlands
USA
Bangladesh
China
Colombia
Morocco
Philippines
Tanzania
Egypt
UAE
Oman
International Organizations
ITU
OECD
UN Global Pulse
UNECE
UNESCAP
UNSD
World Bank

19. Task Teams Advocacy and Communication Big Data and SDG indicators
Access and Partnerships
Skills, Training, Capacity Building
Cross-cutting issues (Quality framework)
Mobile Phone Data
Satellite Imagery
Social Media Data

20. Strong participation Orange, Telenor, World Economic Forum, NASA,
Google,
Data Pop,
World Pop,
Flowminder,
UNU-EHS,
ODI (UK),
Positium,
UPenn,
Harvard

21. Advocacy and Communication
Advocacy/information materials on Big Data for official statistics, such as brochures, papers, presentations, videos, etc.;
Good practices of using big data in official statistics, such as use cases; and
Joint materials with other task teams to promote and communicate the outcomes of their work

22. Big Data and SDG indicators
Assessment of the potential use of Big Data for monitoring progress on the 169 SDG targets by conducting a survey, taking account of the draft proposals of indicators,
An inventory of research work on Big Data that could be used to estimate one or more SDGs, and
Good practices and lessons learned from a pilot research in 1-2 countries on estimating 2-3 SDG indicators using Big Data.

23. Access and Partnerships
Assessment of existing good practices of data access
A set of principles as conditions for access to Big Data sources
Templates for MOUs or services agreements
Good practices and examples of partnerships

24. Training, Skills and Capacity Building
Assessment tool for the needs of Big Data skills and for the institutional ‘readiness’ for Big Data skills;
Corresponding training curriculum;
Technical seminar on Big Data in 2015; and
MOU for a global network of training institutions on Big Data

25. Cross-cutting issues
Inventory of methodologies, data analytics /visualisation tools as well as quality assurance frameworks and refined classification of Big Data;
Examples and good practices for the repository of Big Data projects;
Assessment of Big data sources for the production of official statistics, determine their impact on the production process and identify best practices for managing the necessary changes within an NSO

26. Big Data sources
Lessons learned from one or more pilot projects using mobile phone data / satellite imagery / social media data;
Guidelines on the use of Big Data sources for official statistics, including template of a project plan

27. Big Data and Classifications

28. Classification of Type of Big Data
Social Networks (human-sourced information)
Traditional Business systems (process-mediated data)
Internet of Things (machine-generated data)

29. Classification of Type of Big Data
Social Networks (human-sourced information)
  Social Networks: Facebook, Twitter, Tumblr etc.
  Blogs and comments
  Personal documents
  Pictures: Instagram, Flickr, Picasa etc.
  Videos: Youtube etc.
  Internet searches
  Mobile data content: text messages
  User-generated maps  

30. Classification of Type of Big Data
Traditional Business systems (process-mediated data)
21. Data produced by Public Agencies
2110. Medical records
22. Data produced by businesses
2210. Commercial transactions
2220. Banking/stock records
2230. E-commerce
2240. Credit cards

31. Classification of Type of Big Data
Internet of Things (machine-generated data)
31. Data from sensors
311. Fixed sensors
3111. Home automation
3112. Weather/pollution sensors
3113. Traffic sensors/webcam
3114. Scientific sensors
3115. Security/surveillance videos/images
312. Mobile sensors (tracking)
3121. Mobile phone location
3122. Cars
3123. Satellite images
32. Data from computer systems
3210. Logs
3220. Web logs

32. Global survey (June-Aug 2015)
Big data are data sources with a high volume, velocity and variety of data, which require new tools and methods to capture, curate, manage, and process them in an efficient way

33. Part I: Office Management of Big Data
Big Data strategy
Big Data strategy – Advocacy and Communication
Big Data strategy – linking to the SDG indicators
Access, privacy and confidentiality
Part II: Current status of Big Data projects
Data sources
Technologies
Quality assessment
Part III: Guidelines on Big Data for Official Statistics

34. Ronald Jansen United Nations Statistics Division jansen1@un.org
Questions ?
Ronald Jansen
United Nations Statistics Division