UK Data Warehouse Work 23 rd May 2012 Paul Tutton, Sarah Ravenhill.

Slides:



Advertisements
Similar presentations
Statistics NZs experience in using Administrative Data in an Integrated Programme of Economic Vince Galvin General Manager Strategy & Communications.
Advertisements

Paul Smith Office for National Statistics
Validation, editing and imputation of the Family Resources Survey John Snow (DWP) Family Resources Survey ESDS Government FRS User Meeting April 2008.
P4 – Features and Functions of Information Systems
Taxation in the UK JA Stenning Taxation in the UK Unit 6: Fiscal Policy.
Results and next steps from the ESSnet Admin Data Alison Pritchard Business Outputs & Developments, Office for National Statistics, UK 4 December 2012.
1 Moving from a dynamic cohort microsimulation model to a dynamic population microsimulation model Moving from a dynamic cohort microsimulation model to.
A Data Curation Application Using DDI: The DAMES Data Curation Tool for Organising Specialist Social Science Data Resources Simon Jones*, Guy Warner*,
1 Business Exchange Structures Concepts.
Quality assuring the UK business register Andrew Allen.
LEVERAGING THE ENTERPRISE INFORMATION ENVIRONMENT Louise Edmonds Senior Manager Information Management ACT Health.
United Nations Economic Commission for Europe Statistical Division Applying the GSBPM to Business Register Management Steven Vale UNECE
Environment Change Information Request Change Definition has subtype of Business Case based upon ConceptPopulation Gives context for Statistical Program.
Linking administrative and survey data - employment variable for enterprises and establishments in Finnish Business Register Jaakko Salmela Statistics.
Combining administrative and survey data: potential benefits and impact on editing and imputation for a structural business survey UNECE Work Session on.
Work Package 5: Integrating data from different sources in the production of business statistics Daniel Lewis Office for National Statistics (UK)
DATA FOUNDATION TERMINOLOGY WG 4 th Plenary Update THE PLUM GOALS This model together with the derived terminology can be used Across communities and stakeholders.
1 Workshop on Informal Employment and Informal Sector Data Collection: Strategy, Tools and Advocacy Amman, April 2008 Phase2: Sample design, measurement.
Response to UKSA Monitoring Review on Income and Earnings statistics Joint response from ONS, DWP, HMRC July
Chapter 19 Emerging Management Practices Cost Accounting Foundations and Evolutions Kinney and Raiborn Seventh Edition COPYRIGHT © 2009 South-Western,
12th Meeting of the Group of Experts on Business Registers
CONCEPTUAL MODELLING OF ADMINISTRATIVE REGISTER INFORMATION AND XML - TAXATION METADATA AS AN EXAMPLE Ottawa, May 2005.
Project preparation and appraisal. Preparation of project report and appraisal are intimately tied up.
1 Meeting of the OECD Short-term Economic Statistics Expert Group June 2002 REVIEW OF CONCEPTS AND NATIONAL EXPERIENCES IN THE COMPILATION OF: DEMAND.
Quality issues on the way from survey to administrative data: the case of SBS statistics of microenterprises in Slovakia Andrej Vallo, Andrea Bielakova.
The Adoption of METIS GSBPM in Statistics Denmark.
Module 6: Quantifying gaps and measuring coverage ILO, 2013.
Deliverable 2.6: Selective Editing Hannah Finselbach 1 and Orietta Luzi 2 1 ONS, UK 2 ISTAT, Italy.
Combining survey and administrative data to create a new input data file for National Accounts processes Shaun McLaughlin Central Statistics Office, Ireland.
Register-based migration statistics and using additional administrative data sources Barica Razpotnik Statistical Office of the Republic of Slovenia UNECE.
Data Warehouse Development Methodology
2 Copyright © Oracle Corporation, All rights reserved. Defining Data Warehouse Concepts and Terminology.
Explaining the statistical data warehouse (S-DWH)
USING THE METADATA IN STATISTICAL PROCESSING CYCLE – THE PRODUCTION TOOLS PERSPECTIVE Matjaž Jug, Pavle Kozjek, Tomaž Špeh Statistical Office of the Republic.
Copyright 2010, The World Bank Group. All Rights Reserved. 1 GOVERNMENT FINANCE STATISTICS INTRODUCTION TO GOVERNMENT FINANCE STATISTICS Part 2 This lecture.
Environment Change Information Request Change Definition has subtype of Business Case based upon ConceptPopulation Gives context for Statistical Program.
for statistics based on multiple sources
>>. ESSnet Measuring Global Value Chains 1.Globalisation indicators 2.Methodological development and support for International Organisation and Sourcing.
New sources – administrative registers Genovefa RUŽIĆ.
Chapter 18 Emerging Management Practices Cost Accounting Foundations and Evolutions Kinney, Prather, Raiborn.
Pilot Census in Poland Some Quality Aspects Geneva, 7-9 July 2010 Janusz Dygaszewicz Central Statistical Office POLAND.
Chapter 14 Compensation Control & Administration Harcourt, Inc. items and derived items copyright © 2001 by Harcourt, Inc. All rights reserved. Requests.
Faisal Acc 301 (Chapter 21)
© 2011 Pearson Education GDP: A Measure of Total Production and Income 5 When you have completed your study of this chapter, you will be able to 1 Define.
2 Copyright © 2006, Oracle. All rights reserved. Defining Data Warehouse Concepts and Terminology.
DATA FOR EVIDENCE-BASED POLICY MAKING Dr. Tara Vishwanath, World Bank.
2.4.1and unit content Students should be able to: Define national income and show that it can be seen as a circular flow (and draw this) Explain.
The evolution of the England and Wales census in a European context Garnett Compton, ONS RSS Conference, 9 September 2015.
Wiesbaden Group - Tallinn 2010 Session 4:Lessons Learnt from an Administrative Data Failure Andrew Allen.
National Income Accounts
From Gray and Guppy – Tips for Creating Indicators:
A Framework for Monitoring Economic Development: Datasets of Interest
Statistics for policy use
Chapter 2 – Monetary Poverty
Transforming household finance statistics in the UK
MSDs and combined metadata reporting
Wiesbaden Group - Tallinn 2010 Session 4:Lessons Learnt from an Administrative Data Failure Andrew Allen.
Directors of Social Statistics, March 2011
Logical information model LIM Geneva june
Issues in Administrative Data
Business and IT Architecture for ESS validation
CARICOM TRADE IN SERVICES STATISTICS (CTIS)PROJECT June-December 2016
From Gray and Guppy – Tips for Creating Indicators:
Applying the ESS EARF in a VIP project: The ESS.VIP Validation example
IMPROVEMENT OF SBS DATA QUALITY
National Income Accounting (NIA)
Metadata used throughout statistics production
Presentation to SISAI Luxembourg, 12 June 2012
From Gray and Guppy – Tips for Creating Indicators:
Presentation transcript:

UK Data Warehouse Work 23 rd May 2012 Paul Tutton, Sarah Ravenhill

Outline 1.Background 2.Approach 3.Warehouse Concepts 4.Prototyping & Modelling 5.Data Harmonisation 6.Recommendations and Next Steps

1. Background Other Services Data Sources Data Repository Staging Operational Data Store Data Consumers

2. Approach What do we want? How do we want to work? Does that work? Build it and see What can we put in there? How would we implement one? What are the costs and benefits?

3. What and How Define Interrogate Store Data And Metadata Validate Derive Aggregate Input & Update Extract Find Gaps

4. Build It… Integrate data from multiple sources Make extracts to support current and new statistics Define a method for describing extracts Identify gaps in extracts Automate choice between or combination of sources

Source Level Indicators

Variable Level Indicators Rate my data – what are we consistently suspicious of?

4. …and See Warehouses work Statistical processes must change Shared Information Models are important Think about the minimum acceptable amount of data

5. Assess Potential Conceptual Overlap Meaning of the Data Conceptual Overlap Meaning of the Data Dataset Shape Shape of the population Dataset Shape Shape of the population Statistical Activity Process surrounding the data Statistical Activity Process surrounding the data Harmonisation Analysis

5. Analysis Steps List your sources Describe variables Pool the list Find the concepts Classify variables Assess results

5. Overlap findings Exact Replication Conceptually Close Otherwise Derivable General Feasibility Combinations Small numbers found

Employee Count Employment Foreign Investment Hours/ Pay Pension Schemes Employee Count Employment Foreign Investment Hours/ Pay Pension Schemes 5. Example Concepts Acquisitions/ expenditure Business Operation Business Structure Comments/ Narrative Disposals/ Income Acquisitions/ expenditure Business Operation Business Structure Comments/ Narrative Disposals/ Income Profit/ Loss Statistical Units Stock Taxes/ National Insurance Turnover

5. Interview Findings Pooling data: May assist imputation Enables consist stories across outputs Allows congruence checking at unit level Is more useful if it exposes timelier sources to output managers Is of more benefit for some subjects than others (e.g. employment over finance)

6. Recommendations and Next Steps Continue development of CIM Analyse extent of process change due to movement away from survey silos Implement a warehouse in stages: Integrate storage first De-duplicate and harmonise once integration is complete Consider the addition of statistical processing facilities to reap further benefits