United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning and Implementation of Post Enumeration Surveys Pres. 4

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Some Requirements for a Good PES  Adequate planning  Technical personnel with requisite skills and experience in survey methodology and in designing and implementing the whole PES process Sample design, implementation, matching and estimation  Develop an efficient sample design A successful PES calls for a good sample design and survey implementation  The timing of PES should be soon after census  Maintain operational independence

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Some Requirements for a Good PES (Contd.)  Ensure quality assurance in all the stages of the PES, thus from planning to data processing  Control of non-sampling errors is essential like in any other surveys, as unlike sampling errors, they are difficult to measure, therefore one is better off controlling them  Credible estimation methodologies and procedures

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES  Judicious planning for the PES is critical in order to yield reliable results  Planning of PES should be: Preceded by a clear and unambiguous statement of objectives as this facilitates to articulate planning of PES activities Synchronized with planning for the census Start early and adequate resources devoted to it as part of overall census programme  Develop a comprehensive survey plan showing budgetary and manpower requirements

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES (contd.) Adequate Resources:  Financial allotment: Without sufficient funds the PES would be poorly implemented and therefore may not produce better results than the census. This would defeat the whole purpose of evaluating a census  Human resources: There is need to have requisite number of specialist staff such as survey methodologists; demographers; and analysts; qualified enumerators and supervisors

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES (contd.)  Commitment to the philosophy of a PES: it is very critical to have the top management of a statistical/census office committed to the philosophy of the PES, so that they can plan for resources and support its efficient implementation  Establishment of an independent unit: It is advisable that an independent PES planning and implementation unit is established. This also enhances the operational independence between the census and PES  Technical planning: It is necessary to have sub-technical teams to work on the sample design; plan for the implementation of the PES; the design of a Dual System of Estimation plan, matching and the reconciliation exercise

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES (contd.) Elements of planning a PES  Preparatory activities  Data collection related activities  Matching  Reconciliation (if undertaken)  Data processing  Estimation of coverage and content error  Report preparation and dissemination

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES (contd.) Publicity campaign  PES as a specialized data collection  Importance of not biasing potential sample population  Strategies used in different countries(??)

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES Pilot Test  This can be a dress rehearsal of the actual PES as the pilot census is a dress rehearsal of the census  It can be conducted in selected administrative divisions (taking into account costs)  The purpose of the pilot test is to test the adequacy of the entire PES plan and its organization  It should be conducted in conditions similar to the actual enumeration of the actual PES

Implementation of PES (contd.) Pilot Test (Contd.)  The pilot test should follow immediately the census pilot test  While it is not a source of usable data it provides insights into operational aspects of data collection that can contribute to a successful conduct of a PES and census  Ideally it should be taken a year before the actual PES just as a pilot census is taken a year before a census (UN, 2008 P&R) United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009

Implementation of PES (contd.) Pilot Test (Contd.)  Provides an opportunity to test questions and the overall field methodology before mounting the PES  It also offers a chance to test the matching and analytical procedures  Results of the pilot test contribute to establishment of matching rules, reconciliation procedures and logistical flows of documents between PES and census  Estimates of precision for the PES can be made based on pilot test results, such as sampling errors

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Data collection  Method commonly used in a PES is the face-to-face interview  Enumerators go to households, in selected EAS/clusters, and interview respondents  They collect information by asking questions from a PES questionnaire  This method provides an opportunity for probing  Enumerators can explain the objectives of the PES to respondents

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Questionnaire  Should be based on the final census questionnaire  The format and size of questionnaire are important for recording of responses and for data capture  Pre-testing of questionnaire is imperative  Selection of items for the questionnaire deserves careful consideration  Items to be used in matching process should ideally be considered for the questionnaire

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Questionnaire (Contd.)  Variables for which information is to be collected on the PES should have been included on the census as well  Commonly considered variables include – age, sex, relationship to reference person, marital status, educational level and type of housing unit

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Selection and training of field staff  Enumerators are the interface with respondents should be able to communicate effectively with respondents  Their work is critical to the success of the PES field work and therefore their selection is critical and should be done objectively  They should be thoroughly trained before being assigned field work  Main objective of training is to enhance uniformity and minimize measurement error during interviews

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Training  Qualified instructors well versed with the objectives of the PES should be responsible for training  It is advisable that the trainers should be part of the PES planning and implementation teams  Trainees should take turns in explaining to others various items in the questionnaire  Practical lessons are essential both in the classroom and the field. Based on performance trainees can be retained as enumerators or dismissed

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) The role of supervisors  Despite good training of enumerators, without good supervision, in the field, there may be inaccurate results  Need for dedicated and effective supervision  Supervisors should be more experienced and better qualified than enumerators  Like enumerators, they should undergo extensive training in all aspects of the PES  Supposed to organize work for enumerators by determining field assignments

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) The role of supervisors (Contd.)  They review completed work and maintain a high commitment of enumerators to the PES  A supervisor can make follow-up visits to non- respondents as they are better qualified and experienced  There should be a manageable ratio between enumerators and a supervisor (1 to 5 ratio should be okay)

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Field data collection  During data collection objective is to classify individuals by enumeration status relative to census night  Use of probing is necessary to adequately establish census night residence status

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES (contd.) Reconciliation (if done) (i) Developing instructions for field reconciliation visits (iii) Training of enumerators and supervisors (iv) Carrying out reconciliation visits (v) Final decisions on unresolved cases with respect to final status Data processing (i) Selection or development of computer programmes for data processing, tabulation & estimation (ii) Recruitment and training of data processing staff (iii) Carrying out data entry, editing and tabulation

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Planning of PES (contd.) Estimation of coverage and content error (i) Estimation of coverage error (ii) Estimation of content error Report preparation and dissemination (i) Analysis of PES results including producing and interpretation of sampling errors for key variables (ii) Preparation of PES analytical report (iii) Dissemination of results

Implementation of PES (contd.) Control of non-sampling errors  Should be controlled and reduced to the level that their presence does not compromise the usefulness of the PES results  Particularly harmful when they are non-random because they introduce bias in the PES estimates  Bias is difficult to measure  Best way to control non-sampling error is to follow the right procedures in all PES activities United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009

Implementation of PES (contd.) Control of non-sampling errors (Contd.)  Should be controlled and reduced to the level that their presence does not compromise the usefulness of the PES results  Particularly harmful when they are non-random because they introduce bias in the PES estimates  Bias is difficult to measure  Best way to control non-sampling error is to follow the right procedures in all PES activities

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Implementation of PES (contd.) Factors contributing to non-sampling error  Vague objectives of the PES  Duplication or omissions due to imprecise definition of boundaries of EAs  Inappropriate methods of interviewing  Lack of trained and experienced field interviewers and supervisors  Inadequate identification particulars of sampling units  Errors occurring in data processing

United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, September 2009 Thank You!