1. The Core Welfare Indicators Questionnaire (CWIQ)

2. CONTENTS 1.What is the CWIQ and how does it work? 2.Overview of the Pakistan pilot survey 3.Planning the full scale CWIQ

3. Part 1 What is the CWIQ and how does it work?

4. Quick data entry & validation Fixed core, flexible modules Simple reporting Easy data collection Short questionnaire Large sample Features of the CWIQ

5. Quick data entry & validation Fixed core, flexible modules Simple reporting Easy data collection Short questionnaire Large sample The CWIQ has been built around several key ideas That all work to complement each other Features of the CWIQ

6. Large sample Quick data entry & validation Fixed core, flexible modules Simple reporting Easy data collection Short questionnaire Features of the CWIQ

7. Large sample Quick data entry & validation Fixed core, flexible modules Simple reporting Easy data collection Short questionnaire Needed to provide estimates at sub-national levels and for particular target populations. Features of the CWIQ

8. Large sample Short questionnaire Quick data entry & validation Fixed core, flexible modules Simple reporting Easy data collection Features of the CWIQ

9. Large sample Short questionnaire Quick data entry & validation Fixed core, flexible modules Simple reporting Easy data collection The core questionnaire consists of 9 pages (5 sheets). Features of the CWIQ

10. Large sample Short questionnaire Easy data collection Quick data entry & validation Fixed core, flexible modules Simple reporting Features of the CWIQ

11. Large sample Short questionnaire Easy data collection Quick data entry & validation Fixed core, flexible modules Simple reporting A single interview takes about 40 minutes to complete Features of the CWIQ

12. Simple reporting Large sample Short questionnaire Easy data collection Quick data entry & validation Fixed core, flexible modules Features of the CWIQ

13. Simple reporting Large sample Short questionnaire Easy data collection Quick data entry & validation Fixed core, flexible modules The combination of short questionnaire, scanning and predefined validation procedures reduces processing time and reduces the range and number of non-sampling errors Features of the CWIQ

14. Large sample Short questionnaire Easy data collection Quick data entry & validation Simple reporting Fixed core, flexible modules Features of the CWIQ

15. Large sample Short questionnaire Easy data collection Quick data entry & validation Simple reporting Fixed core, flexible modules Basic tables are automatically tabulated to show frequency distributions for all the key indicators disaggregated by: urban and rural, region and socio-economic grouping. Features of the CWIQ

16. Large sample Short questionnaire Easy data collection Quick data entry & validation Simple reporting Fixed core, flexible modules Features of the CWIQ

17. Large sample Short questionnaire Easy data collection Quick data entry & validation Simple reporting Fixed core, flexible modules Features of the CWIQ A fixed core for rapid reporting. Flexible modules for customizing to country needs

18. Large sample Short questionnaire Easy data collection Quick data entry & validation Simple reporting Fixed core, flexible modules Features of the CWIQ

19. Large sample Short questionnaire Easy data collection Quick data entry & validation Simple reporting Fixed core, flexible modules Features of the CWIQ The CWIQ is not a complicated survey. It incorporates a package of features, which, when taken together, ensure wide coverage and a rapid turnaround time.

20. Part 2 The Pakistan Pilot Survey

21. OBJECTIVES Objectives of the pilot survey include: Testing and refining the modified questionnaire and software system; Testing and refining the modified questionnaire and software system; Training field and data processing staff in use of the CWIQ; Training field and data processing staff in use of the CWIQ; Gathering operational data for planning full scale survey; Gathering operational data for planning full scale survey; Providing end-users with examples of CWIQ output. Providing end-users with examples of CWIQ output. The CWIQ pilot objectives

22. OBJECTIVES The CWIQ modifications CWIQ is a generic package including computer software. Specialist software used for both scanning and data processing must be modified. Questionnaire is combination of the ‘generic’ CWIQ with modifications based on recommendations of technical committee.

23. OBJECTIVES The CWIQ modifications- the wider picture Scanned forms for capturing data from schools and health facilities have also been developed and tested. The tabulation system is being prepared. Scanned forms for capturing data from schools and health facilities have also been developed and tested. The tabulation system is being prepared. First time the CWIQ has been used in Asia. First time the CWIQ has been used in Asia. Strong emphasis on creating an instrument for Pakistani requirements. Strong emphasis on creating an instrument for Pakistani requirements. New features developed here should contribute to the ongoing development of the CWIQ. New features developed here should contribute to the ongoing development of the CWIQ.

24. THE TIMETABLE 3-6 NovemberTraining course in Islamabad 7-15 NovemberFieldwork in 4 provinces 20 NovemberAll data cleaned and tabulated 21 NovemberPresentation workshop The CWIQ pilot timetable

25. THE QUESTIONNAIRE some questions relate to the household as a unit of study (asset ownership, access to services etc.) some questions relate to the household as a unit of study (asset ownership, access to services etc.) other questions relate to each household member (demographic information, education, health and employment) other questions relate to each household member (demographic information, education, health and employment) additional data for children under five years old (immunisation history) additional data for children under five years old (immunisation history) The questionnaire

26. THE QUESTIONNAIRE % reporting diminishing/increasing assets (land and livestock)% reporting diminishing/increasing assets (land and livestock) % of literate adults% of literate adults Housing (mean number of persons per room).Housing (mean number of persons per room). % of adults unemployed in the last 7 days.% of adults unemployed in the last 7 days. Welfare indicators The questionnaire captures data for estimating:

27. THE QUESTIONNAIRE Access to clean waterAccess to clean water Access to sanitation facilitiesAccess to sanitation facilities Access to pre-primary, primary and secondary school.Access to pre-primary, primary and secondary school. Net pre-primary, primary and secondary enrolment rates (by gender)Net pre-primary, primary and secondary enrolment rates (by gender) Primary and secondary drop out rates (by gender)Primary and secondary drop out rates (by gender) Type of birth supervision for children less than five.Type of birth supervision for children less than five. Satisfaction rating with primary and secondary school services.Satisfaction rating with primary and secondary school services. Access to local health centre.Access to local health centre. Frequency of use of local health centre.Frequency of use of local health centre. Completeness of vaccination record for children under five years.Completeness of vaccination record for children under five years. Satisfaction rating with local health services.Satisfaction rating with local health services. Access, utilisation and satisfaction The questionnaire captures data for estimating:

28. THE QUESTIONNAIRE Designed for scanning

29. 4 day training course which included: 4 day training course which included: detailed presentation on the questionnaire and interview technique detailed presentation on the questionnaire and interview technique role play interviewing in class role play interviewing in class field interviews field interviews exercises to introduce techniques needed for scanned forms exercises to introduce techniques needed for scanned forms THE TRAINING COURSE The training course

30. Training GOOD DATA QUALITY

31. The scanner and computer software is critical to the training. The scanner and computer software is critical to the training. All practice interview forms are scanned and validated. All practice interview forms are scanned and validated. Errors and problems are identified and feedback is provided to trainees. Errors and problems are identified and feedback is provided to trainees. The system can be used for final selection of competent interviewers. The system can be used for final selection of competent interviewers. GOOD DATA QUALITY Preventing data quality problems

32. thorough training of interviewers to ensure an in-depth understanding of the CWIQ; thorough training of interviewers to ensure an in-depth understanding of the CWIQ; close supervision in the field; close supervision in the field; interviewers checking each other’s work; interviewers checking each other’s work; rapid data loading with extensive computerised validation checks; and rapid data loading with extensive computerised validation checks; and early feedback to interviewers in the case of problems. early feedback to interviewers in the case of problems. Data quality is achieved through: GOOD DATA QUALITY

33. 48 clusters each with 20 households; 48 clusters each with 20 households; total sample size of 960 households; total sample size of 960 households; sample was selected on a purposive basis, principally to test the methodology. Care should be exercised with the results from this survey. sample was selected on a purposive basis, principally to test the methodology. Care should be exercised with the results from this survey. THE SAMPLE FOR THE PILOT SURVEY The sample

34. RuralUrbanTotal Punjab240160400 Sindh 80 80160 240 240 NWFP120 80 80 200 200 Balochistan 80 80 40 40120 Total 520 520 440 440960 THE SAMPLE FOR THE PILOT SURVEY The sample Number of households by province

35. The CWIQ pilot fieldwork 27 interviewers with 8 supervisors 27 interviewers with 8 supervisors just over 9 days field work just over 9 days field work THE FIELDWORK

36. 4 interviews per enumerator per day should be possible THE FIELDWORK (continued)

37. Mean interview duration was about 40 minutes THE FIELDWORK (continued)

38. Mean household size was 6.6 persons THE FIELDWORK (continued)

39. team of 4 FBS staff plus 1 consultant; team of 4 FBS staff plus 1 consultant; uses standard Windows-based computers; uses standard Windows-based computers; requires two key software packages; requires two key software packages; TELEform for image processing of the scanned forms, converting the marked areas into data values; TELEform for image processing of the scanned forms, converting the marked areas into data values; Microsoft Access to build data structures, validate, correct, summarise and tabulate the data; Microsoft Access to build data structures, validate, correct, summarise and tabulate the data; objective is to start data processing as soon as possible after the start of fieldwork. objective is to start data processing as soon as possible after the start of fieldwork. THE DATA PROCESSING

41. Visual checks on questionaires Prepare forms for scanning Scan forms to create digital images in the computer Automatic creation of data records. Some user verification may be required THE DATA PROCESSING - stage 1

42. This shows how character recognition is verified and corrected using TELEform software THE DATA PROCESSING - stage 1 (continued)

43. An evaluated form showing the original on the right hand side THE DATA PROCESSING - stage 1 (continued)

44. Week 1 – waiting for forms to arrive. Week 2 – over 250 forms on 18th THE DATA PROCESSING - stage 1 (continued)

45. A single 9 page questionnaire can be scanned and converted into the database format in just over 1 minute. Three people in the data processing team can handle 300 questionnaires per day using the equipment used for the pilot survey. Faster scanners and computers as well as more staff would be used in the full scale survey. Capacity would be matched to the rate of fieldwork. The package includes comprehensive documentation on all aspects of the data processing.

46. THE DATA PROCESSING - stage 2 Once scanned the data are transferred to standard database package (MS-Access) which has been configured to: perform a number of validation checks to test logical consistency of the data; perform a number of validation checks to test logical consistency of the data; provide a means of editing erroneous records; provide a means of editing erroneous records; generate derived variables; generate derived variables; produce a range of formatted outputs including descriptive statistics, multi-way tables and standard errors; and produce a range of formatted outputs including descriptive statistics, multi-way tables and standard errors; and export the data to other data analysis packages. export the data to other data analysis packages.

47. Transfer data to Access Run validation checks Examine error listing and determine corrections to be made THE DATA PROCESSING - stage 2 THE DATA PROCESSING - stage 2 (continued) Enter corrections Summarise and generate tables

48. A small selection of the standard output of the CWIQ tabulation system has been provided as a handout. Please note the request that these results not be used for any purpose other than demonstrating the CWIQ system. EXAMPLES OF CWIQ STANDARD OUTPUT TABLES

49. Part 3 Scaling up to a national CWIQ survey

50. MOVING TO THE NATIONAL CWIQ The pilot survey has, hopefully, achieved its objectives. The pilot survey needs to be thoroughly reviewed, both internally and externally, in order to plan ahead. Prompt action is required in order to maintain momentum. The full set of modified tables and review document should be produced as quickly as possible. What are the next steps?

51. MOVING TO THE NATIONAL CWIQ The technology has worked well. The modified software has been successful as has optical scanning for data loading. The technology has worked well. The modified software has been successful as has optical scanning for data loading. Most interviewers conducted the interview satisfactorily. Most interviewers conducted the interview satisfactorily. The FBS team who carried out the scanning and cleaning have proved they can do this task. The FBS team who carried out the scanning and cleaning have proved they can do this task. Data from 960 households across Pakistan have been collected, scanned, cleaned and tabulated in less than 2 weeks. Data from 960 households across Pakistan have been collected, scanned, cleaned and tabulated in less than 2 weeks. First thoughts – what has worked?

52. MOVING TO THE NATIONAL CWIQ The training course was probably too short. The training course was probably too short. Some of the fieldwork and supervision was of poor quality. This is one of the most serious challenges. Some of the fieldwork and supervision was of poor quality. This is one of the most serious challenges. First thoughts – what needs to be improved?

53. MOVING TO THE NATIONAL CWIQ Thorough review of output and field experience of interviewing teams. Thorough review of output and field experience of interviewing teams. Comments of potential data users. Comments of potential data users. Formalisation of lessons learned at each stage of the pilot exercise – identifying best practice. Formalisation of lessons learned at each stage of the pilot exercise – identifying best practice. Further adaptation of the CWIQ to Pakistan’s requirements? – depends on the review. Further adaptation of the CWIQ to Pakistan’s requirements? – depends on the review. Review and revision of the questionnaire

54. MOVING TO THE NATIONAL CWIQ The course ahead will be determined by the answers to a number of questions. Does the output of the CWIQ meet the requirements of the data users (in terms of relevance, accuracy, timeliness etc.)? Does the output of the CWIQ meet the requirements of the data users (in terms of relevance, accuracy, timeliness etc.)? What are the resource implications? What are the resource implications? What is required for the FBS to implement this survey instrument in the time frame currently envisaged? (completed by Sept 2004) What is required for the FBS to implement this survey instrument in the time frame currently envisaged? (completed by Sept 2004) How does it fit into the programme of other activities for the FBS, especially the PIHS? How does it fit into the programme of other activities for the FBS, especially the PIHS?

55. MOVING TO THE NATIONAL CWIQ Proposal for moving to a national CWIQ – tasks... consolidation of lessons learned and revision of questionnaire and data processing system consolidation of lessons learned and revision of questionnaire and data processing system logistics and planning for a major exercise; logistics and planning for a major exercise; preparation of detailed budget for additional capital and recurrent expenditure; preparation of detailed budget for additional capital and recurrent expenditure; procurement; procurement; technical assistance for the first full survey; technical assistance for the first full survey; recruitment and training of interviewers; recruitment and training of interviewers;

56. MOVING TO THE NATIONAL CWIQ A complement of 300 interviewers A complement of 300 interviewers Say 4,000 interviews per week allowing time for travel, listing etc.; Say 4,000 interviews per week allowing time for travel, listing etc.; A sample of 80,000 would take around 6 months, A sample of 80,000 would take around 6 months, Data processing can keep pace with interview rate leading to early availability of results. Data processing can keep pace with interview rate leading to early availability of results. Extending the capacity within the FBS One possible example:

57. MOVING TO THE NATIONAL CWIQ How do we ensure that the CWIQ contributes to: How do we ensure that the CWIQ contributes to: Performance Related Budgeting; Performance Related Budgeting; Monitoring the impact of expenditures on priority areas; Monitoring the impact of expenditures on priority areas; Cross-checking facility (education and health) data with the HMIS and EMIS ; Cross-checking facility (education and health) data with the HMIS and EMIS ; Easier to achieve above objectives when CWIQ is firmly established in the FBS and fully owned by stakeholders. Easier to achieve above objectives when CWIQ is firmly established in the FBS and fully owned by stakeholders. Linking CWIQ to policy?

58. The End The End