STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY LIFECYCLE Vicki Crompton and Mike Sivyer Atlantic DLI Training.

Slides:



Advertisements
Similar presentations
Data Quality Assurance and Dissemination International Workshop on Energy Statistics Aguascalientes, Mexico.
Advertisements

Statistics NZs experience in using Administrative Data in an Integrated Programme of Economic Vince Galvin General Manager Strategy & Communications.
DLI & Research Data Centres Creating a better understanding of these two programs Chuck Humphrey Data Library University of Alberta April 2004.
Elizabeth Hamilton Atlantic DLI Training April 29, 2005.
DLI Training Nesstar Workshop
DLI Orientation: Concepts A Framework for Thinking about Statistical Information Train the Trainers Montreal, March 9, 2004 Chuck Humphrey Data Library.
Labour Force Historical Review Sandra Keys, University of Waterloo DLI OntarioTraining University of Guelph, Guelph, ON April 12, 2006.
Input Data Warehousing Canada’s Experience with Establishment Level Information Presentation to the Third International Conference on Establishment Statistics.
Enhancing Data Quality of Distributive Trade Statistics Workshop for African countries on the Implementation of International Recommendations for Distributive.
Workshop on Energy Statistics, China September 2012 Data Quality Assurance and Data Dissemination 1.
Data Access and Data Use: the Missing Link? Elizabeth Hamilton University of New Brunswick Chuck Humphrey University of Alberta Data and Knowledge Transfer.
Chuck Humphrey & Lynne Robinson University of Alberta Surviving Statistics Strategies for dealing with statistical questions on the reference desk.
United Nations Workshop on Revision 3 of Principles and recommendations for Population and Housing Censuses and Census Evaluation Amman, Jordan, 19 – 23.
Statistics and Data for Marketing Data Library, Rutherford North 1 st Floor Chuck Humphrey Data Library October 27, 2008.
EAS 293 Data Library, Rutherford North 1 st Floor Chuck Humphrey Data Library October 14, 2008.
United Nations Expert Group Meeting on Revising the Principles and Recommendations for Population and Housing Censuses New York, 29 October – 1 November.
An Integrated Approach to Economic Statistics “ The Canadian Experience” UNSD – IBGE Workshop on Manufacturing Statistics Kevin Roberts Rio de Janeiro,
STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2008 SURVEY LIFECYCLE Michel B. Séguin Atlantic DLI Training.
Time Use Surveys in Canada Jodi-Anne Brzozowski Statistics Canada International Seminar on Time Use September , 2010.
United Nations Economic Commission for Europe Statistical Division Applying the GSBPM to Business Register Management Steven Vale UNECE
United Nations Workshop on Revision 3 of Principles and recommendations for Population and Housing Censuses and Census Evaluation Amman, Jordan, 19 – 23.
Metadata: Integral Part of Statistics Canada Quality Framework International Conference on Agriculture Statistics October 22-24, 2007 Marcelle Dion Director.
FCM Quality of Life Reporting System Metadata By: Acacia Consulting and Research June 2002.
European Conference on Quality in Official Statistics (Q2010) 4-6 May 2010, Helsinki, Finland Brancato G., Carbini R., Murgia M., Simeoni G. Istat, Italian.
Using IPUMS.org Katie Genadek Minnesota Population Center University of Minnesota The IPUMS projects are funded by the National Science.
Health Statistics Information on STC website Calgary–DLI training–Dec 2003 Michel B. Séguin, Statistics Canada,
Michel Séguin DLI Chief December 2006 The Need to Liberate The Data.
Population Census carried out in Armenia in 2011 as an example of the Generic Statistical Business Process Model Anahit Safyan Member of the State Council.
Chuck Humphrey, University of Alberta Atlantic DLI Training, 2008 DLI Orientation: Concepts A Framework for Thinking about Data and Statistics.
Current and Future Applications of the Generic Statistical Business Process Model at Statistics Canada Laurie Reedman and Claude Julien May 5, 2010.
Framework of Statistical Information. This is a typology of the categories or classes of statistical information. Remember the relationship between statistics.
1 Improving Data Quality. COURSE DESCRIPTION Introduction to Data Quality- Course Outline.
ICES III - Montréal, Canada Listening to Respondents for Better Results Alexander Hays Distributive Trades Division Statistics Canada.
United Nations Economic Commission for Europe Statistical Division Mapping Data Production Processes to the GSBPM Steven Vale UNECE
Use of Administrative Data Seminar on Developing a Programme on Integrated Statistics in support of the Implementation of the SNA for CARICOM countries.
Statistics Canada Statistics Canada Statistique Canada Statistique Canada Disseminating gender statistics: The Canadian experience Heather Dryburgh, Ph.D.
New sources – administrative registers Genovefa RUŽIĆ.
Compilation of Distributive Trade Statistics in African Countries Workshop for African countries on the implementation of International Recommendations.
Editing of linked micro files for statistics and research.
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
Copyright 2010, The World Bank Group. All Rights Reserved. Principles, criteria and methods Part 2 Quality management Produced in Collaboration between.
JOINT UN-ECE/EUROSTAT MEETING ON POPULATION AND HOUSING CENSUSES GENEVA, MAY 2009 DETERMINING USER NEEDS FOR THE 2011 UK CENSUS IAN WHITE, Office.
Household Surveys: American Community Survey & American Housing Survey Warren A. Brown February 8, 2007.
Pilot Census in Poland Some Quality Aspects Geneva, 7-9 July 2010 Janusz Dygaszewicz Central Statistical Office POLAND.
Developing and applying business process models in practice Statistics Norway Jenny Linnerud and Anne Gro Hustoft.
Regional Seminar on Promotion and Utilization of Census Results and on the Revision on the United Nations Principles and Recommendations for Population.
National Boot camp Vancouver Heather Dryburgh and Michel B. Séguin May 31 st, 2011 Survey Life cycle.
United Nations Oslo City Group on Energy Statistics OG7, Helsinki, Finland October 2012 ESCM Chapter 8: Data Quality and Meta Data 1.
Developing the prototype Longitudinal Business Database: New Zealand’s Experience Julia Gretton IAOS Conference Shanghai, China, October 2008
Copyright 2010, The World Bank Group. All Rights Reserved. Recommended Tabulations and Dissemination Section B.
1 Working with Canadian Census Microdata Martine Grenier and Mokili Mbuluyo Census Operations Division, Statistics Canada December 2007.
1 Overview of Economic Statistics in Africa UNECA Andry Andriantseheno Regional Workshop on Basic Economic Statistics Addis-Ababa October 2007.
Overview and challenges in the use of administrative data in official statistics IAOS Conference Shanghai, October 2008 Heli Jeskanen-Sundström Statistics.
5.8 Finalise data files 5.6 Calculate weights Price index for legal services Quality Management / Metadata Management Specify Needs Design Build CollectProcessAnalyse.
Introduction to Statistics Estonia Study visit of the State Statistical Service of Ukraine on Dissemination of Statistical Information and related themes.
Metadata models to support the statistical cycle: IMDB
Information for marketing management
Quality assurance in official statistics
COMPILATION OF DISTRIBUTIVE TRADE STATISTICS IN UGANDA
Dissemination Workshop for African countries on the Implementation of International Recommendations for Distributive Trade Statistics May 2008,
Sharne Bailey, Tony Byrne UK, Office for National Statistics
Generic Statistical Business Process Model (GSBPM)
Organization of efficient Economic Surveys
Albania 2021 Population and Housing Census - Plans
Telling Canada’s story in numbers Marie-Josée Major
Mapping Data Production Processes to the GSBPM
Petteri Baer, Regional Adviser, UNECE
Technical Coordination Group, Zagreb, Croatia, 26 January 2018
Presentation transcript:

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY LIFECYCLE Vicki Crompton and Mike Sivyer Atlantic DLI Training

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM The financial perspective…. Core statistical programs dictated by statutes, regulatory instruments and contractual obligations account for 93% of STC budget $510 million in base budget supplemented by $130 million in supplementary revenues ($45 million in cost-recovery surveys) Social programs account for 65% of base budget, 75% of supplementary revenues

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM Weekly1 Monthly70 Quarterly52 Tri-annual2 Semi-annual8 Annual194 Biennial22 Triennial2 Quinquennial6 Occasional28 TOTAL385 ACTIVE SURVEYS Social Surveys: 93 Economic Surveys:292 The statistical perspective…

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM The rationale………. Produce information to support statutory and regulatory purposes Produce information on the economic and socio- demographic structure of Canada Produce information needed to support informed policy formulation, decision-making, research, and discussion Produce information required to meet emerging issues and new challenges

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM Maintaining relevance………. National Statistics Council Network of subject-specific professional Advisory Committees Federal Provincial Consultative Council Stakeholder consultations

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 STC SURVEY PROGRAM The deciding factors………. Relevance Public benefit Cost Response burden Public sensitivies

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 MAJOR COMPONENTS OF A SURVEY

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 What the heck is a survey?

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 WHAT IS A SURVEY? The collection of information From all or some units of a population Employing well-defined concepts, methods and procedures The compilation of such information into useful summary form

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CENSUSSAMPLE TYPES OF SURVEYS Includes all units of the population Population = total set of units to which survey results apply Includes some units of the population Sample = a representative part of the entire population used to reflect the entire population

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 In Statistics Canada we also categorize by: Business vs Household vs Institution Voluntary vs Mandatory Interviewer Administered vs Self-completed Administrative Records vs Direct Collection Cross-classified vs Longitudinal Ad Hoc vs On-going/Core NOTE: These are NOT mutually exclusive! TYPES OF SURVEYS

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 EXAMPLES OF SURVEYS WITHIN STC Regular (on-going/core) Surveys -Labour Force Survey - Retail Commodity Survey - Census of Population - Unified Enterprise Survey - Survey of Household Spending - Annual Survey of Manufactures - General Social Survey - Survey of Steel Inventory - Etc.

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 EXAMPLES OF SURVEYS WITHIN STC Special (ad hoc/irregular) Surveys - Youth Smoking Survey - Survey of Graduates Who Moved to the USA - Child Care Survey - Household Internet Use Survey - Survey of Approaches to Educational Planning - Adult Education and Training Survey - Survey of Importance of Wildlife to Canadians - International Adult Literacy and Skills Survey

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 EXAMPLES OF SURVEYS WITHIN STC Longitudinal Surveys - Survey of Labour and Income Dynamics - National Population Health Survey - National Longitudinal Survey of Children and Youth - Workplace and Employee Survey

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 User Consultation and Content Determination Questionnaire Development Data Collection Data Processing Data Quality Measurement Dissemination

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY OBJECTIVES Reasons for collecting the data Elaboration of information requirements How will the information/data be used? Identification of Population/Target Group What is the population to be studied? Are there any sub populations to be identified?

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY OBJECTIVES Identify Units of Analysis Individual Household Family - (Census or Economic) Business - Enterprise - Company - Establishment

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY OBJECTIVES Analysis Plan - Purpose/uses of the data - Statement of outputs Level of Geography Required for Analysis - National vs provincial - Urban/rural - Cities/economic regions Level of accuracy required

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY DESIGN ISSUES Feasibility of collecting data required Type of survey required (census/sample) Resources required Training Costs

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SURVEY DESIGN ISSUES Method of Collection Paper and Pencil Interview (PAPI) Computer Assisted Interviewing (CAI) By interviewer By respondent (incl. electronic transmission) Proxy vs Non-proxy Personal vs Telephone Interview vs Self-completed Special Testing Applications

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SAMPLE PLAN Target/survey population Sampling units Population must be divisible into distinctive non- overlapping units called sampling units Each member of the population should belong to just one sampling unit Size of sample required Sample selection methods Sample Frame

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SAMPLE FRAME Means to access survey population A list containing the survey population For Example: A printed telephone book A computer database of registered motor vehicles Business Register Address Register Should contain all the relevant sampling units and all necessary and relevant information

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 SAMPLE FRAME Sample of other surveys Labour Force Survey sample (supplements) Census of Population (post-censal) Canadian Community Health Survey Geographic Frames Lists of EAs or CMAs from census files

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT First perform a literature search Other sources which asked similar questions Other surveys in STC Surveys from other countries Define/develop concepts and definitions Similar to other STC surveys Specific to this survey Easy to understand, be accepted and used by interviewers and respondents.

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 COMPONENTS OF A SURVEY

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT Wording of questions Easily understood by interviewer and respondent Easy to read - sounds natural – friendly Appropriate level of language Flows and skip patterns Easy to follow Logical to interviewer and respondent Capture all relevant information from respondents

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT Questionnaire layout (print version) Easy to follow? Need additional instructions for Interviewer? CAI Application Ability to program Appropriate questions introduced at correct time User-friendly (for when completed by respondent)

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 CONTENT DEVLOPMENT Determine which type of question is best for each question asked: Closed question - Identify answer categories to include Open-ended question (write-in) - Coding of write-in responses

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005

TESTING ACTIVITIES An important part of all major components Test wording of questions (in both official languages) Are concepts and definitions understood? Level of language Sequence of questions (flows/skips) Length of interview

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 TESTING ACTIVITIES Focus groups Informal with colleagues Field procedures testing Pilot testing CAI Applications and other related programs

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 TESTING ACTIVITIES EXTREMELY IMPORTANT SOMETIMES SACRIFICED DUE TO TIME AND COST CONSTRAINTS

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DATA COLLECTION/FIELD ACTIVITIES Interviewer training Sample Selection and send to Interviewers Send questionnaires /CAI application to Interviewers Necessary equipment in place (laptops, phones, reference materials, etc.) Field collection/ Interviewing Data capture & transmission Respondent relations

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES STC Policy requires that we inform respondent of: Purpose of survey Confidentiality assurances Volunteer/mandatory aspects of survey Any projected linking of data Any projected sharing of data Normally start by sending a letter to respondents informing them of survey

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES Also have an introduction at the start of the interview process on the questionnaire

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES For business surveys there is a special program called the Key Provider Manager (KPM). The top 300 large complex enterprises in Canada account for one third of all economic activity. This can result in these enterprises being in many surveys leading to respondent burden and possible refusal to participate.

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 RESPONDENT RELATIONS ACTIVITIES Duties of KPM are: To co-ordinate and facilitate flow of information between these enterprises and STC; To negotiate reporting arrangements with enterprise; To explain concepts, definitions reporting requirements; To do follow-up; Etc.

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA There are a series of steps that include: Editing Imputation Coding Creation of derived variables Weighting calculations Preparation of metadata Final file creation

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Why edit the data? Data must be logical and consistent. Errors that may have been introduced into the data must be fixed. Errors can include: Interviewer errors Capture errors Respondent errors

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA What is included in editing? Inter-question relationships Paths and flows Coding of write-in responses Imputation for non-response or response error

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Imputation activities can include: Replacing response error with true value Imputation of non-response Imputation of missing values based on other responses from same record Imputation of missing values with responses from a similar record Replace values for an entire section or record with responses from similar record Remove record from file

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Imputation of Data from Other Records on File

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Imputation of Data from Other Records on File

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005

PROCESSING THE DATA During the editing process there is also the creation of Derived Variables. This can be the combination of responses of one question into a more useable format: e.g. Individual income into groups or ranges. Or the combination of two or more variables into a new variable: e. g. Usually smoke cigarettes with Usual number of cigarettes smoked to create Type of smoker: heavy, moderate, light.

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA It is during the editing stage that decisions and actions may be taken to suppress data This is done because of: Confidentiality risk Sensitivity of data Quality/reliability issues

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA One of the final activities of the processing stage is the calculation of the weights (sometime called the sample weight) The weight variable indicates the number of units in the population that are represented by each unit in the sample The sample weight corresponds to the inverse of the probability of selection of the unit in the sample

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA For example: In a survey of Ontario farms lets assume that: Farm population = 153,000 (farms) Survey sample required = 9,000 Probability of selection = 9,000 / 153,000 = 1/17 Sample weight = 17 However….

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Most samples are based upon a complex sample design, with stratification, multiple stages of selection, and unequal probabilities of selection of respondents Weighting MUST take this into account Weighting also has to take into account any non- response

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA After all the editing and weighting activities are completed the end result is what is called theMaster File. This file..: contains all (unsuppressed) data can use this file to produce tabulations for analysis For most business surveys this is the only file produced. This is because population is very small and the risk of disclosure is very high

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA Many household (social) surveys produce a Public Use Microdata File. This file: Contains suppressed data for public utilization May contain fewer variables than master file Must be approved by the Microdata Release Committee

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA The Microdata Release Committee reviews all of the procedures and activities done by the Survey Team related to Confidentiality Issues. This includes: Removal of all identifying information (name, address, etc.); Removal of any variables which allow linking of records across files; Suppression; Creation of derived variables; Etc.

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 PROCESSING THE DATA The survey team also produces all of the metadata required for the dissemination of the data files This includes: Data quality information Methodological information Questionnaires/questions asked in CAI applications User guides for microdata files Record layouts/codebooks

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION Must register all products and services Announcement in the Daily Release - availability of data Products released Highlights of the release (optional) Integrated Metadata Base (IMDB) is updated STC Online Catalogue is updated CANSIM updated

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION PRODUCTS: Statcan.ca (free) Publications ($ and free) Data Tables/ electronic files ($ and free) Canadian Statistics tables (free) Public Use Microdata Files ($$) CANSIM, Trade data, etc ($$) Custom Tabulations ($$) Census, Input/Output, etc

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION CHANNELS: Advisory Services All publications added to STC library collection Catalogued in BiblioCat Depository Services Program Full depository libraries – 52, Selectives libraries Legal Deposit of publications to LAC

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION CHANNELS: Data Liberation Initiative 67 subscribing Canadian universities and colleges All standard electronic products Research Data Centres 14 across the country Access to Master files – strict controls Secondary distributors

STATISTICS CANADA SURVEY LIFECYCLE WOLFVILLE, APRIL 2005 DISSEMINATION OTHERS: Secondary Analysis - Other government department(s) - Experts in field - Academics - Etc.