1. PLANNING FOR AN APRM NATIONAL SURVEY: KENYA’S EXPERIENCE. BY WAFULA MASAI AFRICAN CENTRE FOR ECONOMIC GROWTH AND SCHOOL OF ECONOMICS, UNIVERSITY OF NAIROBI.

2. OUTLINE 1Overview 2Developing the Survey Instruments (4) 3Nature of the Training and Enumerators’ Field Manual 4Selection and Training of Enumerators 5Study Design (sampling approach, field logistics, the technical activities, the budget, field logistics) 6Data Collection and Processing 7Activities and Timing 8Indicative Budget

3. 1. Overview: National Coordinating Structures APRM NATIONAL GOVERNING COUNCIL APRM KENYA SECRETARIAT LEAD TECHNICAL AGENCIES Democracy & Political Governance African Center for Economic Growth (ACEG) Socio-economic Development Institute for Development Studies (IDS) Economic Management and Governance Kenya Institute for Public Policy Research and Analysis (KIPPRA) Corporate Governance Center for Corporate Governance (CCG) THEMATIC GROUPS Democracy and Political Governance Socio-economic Development Economic Management and Governance Corporate Governance MINISTRY OF PLANNING & NATIONAL DEVELOPMENT

4. 2Overview: Methodological Framework Pre-field Methodology comprised of information, education and communication activities and included roll-out sensitization activities at provincial administration level Self Assessment Methodology by Technical Agencies –Four Principal Instruments: Desk Research, Expert Panel, National Household Survey, Focus Group Discussions Syndicate reports (Trade Unions, Journalists, Professionals, Faith Based Institutions, Umbrella Organizations such as Private Sector Associations, Women’s Organizations, Youth Organizations, etc) Post-field Methodology (experts to interrogate the draft reports regarding integrating findings from the four instruments for each pillar, ensuring a flow in the reporting style, validating the contents of the draft report to ensure credibility, legitimacy, authority and acceptability. The last consultations included the Stakeholder Workshop and a retreat with Permanent Secretaries.

5. 3.Overview: Preliminaries(1) Kenya started by domesticating the NEPAD APRM Master Questionnaire (adding, subtracting, and clarifying some of the questions/issues) This was followed by recasting the domesticated Questionnaire into Survey Instruments (in this case four survey instruments), with each instrument containing questions on the four pillars The instruments included –Desk Research (using a good Desk Research Instrument, competent scholars to conduct the survey, and relevant literature providing factual and verifiable information). –Expert Respondents (using a good Expert Panel Instrument, carefully identified expert respondents based on criteria of age, education, gender, profession, ethnic background, region, etc., and carefully identified and trained interviewers)

6. Overview: Preliminaries(2) –National Household Sample Survey (using the Sample Survey Instrument) –Focus Group Discussions using FGD Checklist Carefully selected participants Competent facilitators –Memoranda, Reports, and other written submissions from a diverse range of stakeholders

7. 4Overview: National Survey Planning Team Subject Specialist (for the four pillars of governance) Senior Statistician preferably from the National Bureau of Statistics Data Analyst Logistician

8. NATIONAL GOVERNING COUNCIL T ECHNICAL COMMITTEE (ACEG KIPPRA, IDS, CCG AND CBS) FIELD SURVEY (UNDER STATISTICIAN & GOVERNANCE SPECIALIST) DATA PROCESSING (UNDER DATA ANALYST) LOGISTICS (UNDER LOGISTICIAN) NATIONAL HOUSEHOLD SAMPLE SURVEY (TEAM LEADER) SURVEY PLANNING/ DESIGN SURVEY INSTRUMENT DEVELOPMENT DESIGN FIELD STAFF TRAINING DESK RESEARCH AND DATA COLLECTION (DISTRICT STATISTICAL OFFICERS, FIELD SUPERVISORS AND FIELD ENUMERATORS) REPORT WRITING DATA EDITING DATA ENTRY (DATA ENTRY CLERKS) TABULATION (DATA ANALYSTS) OBTAINING AND DISBURSING FUNDS PROCUREMENT OF SUPPLIES COORDINATING TRANSPORT FIXING APPOINTMENTS AND ARRANGING VENUES

9. (II)Developing the Survey Instrument (1) This was done in close consultation with the Central Bureau of Statistics It was necessary to develop the National Household Sample Survey Questionnaire from the NEPAD APRM Master Questionnaire comprising of: –Identification and demographic information for each respondent –Questions designed to capture perceptions in a quantifiable manner (Yes/No, Ranking from highest to lowest, selecting options), with minimal open ended questions.

10. Developing the Survey Instrument (2) Questions were asked on each of the objectives in the four pillars (See copy of questionnaire in the folder) –Democracy and Political Governance (9 objectives) –Economic Governance and Management (5 objectives) –Corporate Governance (5 objectives) –Socioeconomic development (6 objectives)

11. (III) Training and Enumerators’ Manual (1)  The manual focused on principles of interviewing:  Role of Enumerator  Handling the respondent - including reluctant respondents  Confidentiality  Neutrality  Probing  Recording answers  Call back procedures  Language translation  Ending the interview

12. (IV) Training Enumerators and Pre-testing the Questionnaire Selection of Enumerators was on the basis of academic qualifications, experience, fluency in Kiswahili, and region of origin (knowledge of local dialect) Training of Enumerators was based on the Manual and centred around concepts and definitions, and how to ask each question in English, and Kiswahili and how to translate it into the local dialect Survey instrument pre-testing, sharing of experiences from the Pre-Test and revision of questions for clarity and simplicity  Learning how to refer to the Survey Manual in the event of not being sure of a question

13. (V)Study Design (1) The Sample for the survey was selected from the Sampling Frame - National Sample Survey and Evaluation Programme III (NASSEP III). The Sampling Frame is maintained and used by the Central Bureau of Statistics of Kenya to conduct household based sample surveys Sample Frame was developed from the 1999 Population and Housing Census and contained 1,133 clusters (of which 930 were rural and 203 were urban), with each cluster having approximately 100 households Each household in the cluster is identified by a number, the name of the household head and the exact village location. There were Cartographic maps to show location of each household in the cluster. The maps and household listings by name of household head are updated every 2 years and were therefore current.

14. Study Design (2) It was decided that all the 69 districts in Kenya were to be covered and hence stratification was on the basis of the districts Going by the district strata, 225 clusters were selected using the Probability Proportional to Size (PPS) Method. Then using systematic sampling with a random start, 10 households were drawn from each of the selected clusters. A sample size of 2250 households was estimated to be adequate for estimates at the national level. Each of the 69 districts was visited with some districts having more clusters and respondents than others depending on population size. District strata were further sub-stratified into urban and rural components taking into account rural-urban population proportions in the district

15. Study Design (3) Replacement of a household in the survey could only be done in consultation with Central Bureau of Statistics Headquarters. In each household, the household head was the respondent and if absent, the next senior member. From the data, it was possible to estimate the population parameters or indicators using proportions, ratios, means, and totals. The estimation process involved multiplying the weighting factor with the sample value per household and summing up the products for all the households in Kenya. Estimation of sampling errors and variance and statistical significance tests were done.

16. (VI)Data Collection and Processing (1) Administration of the interviews was through personal interviews of household heads by enumerators. Interviews were conducted using the language the respondent was more conversant with. Supervisors in the field edited the completed questionnaires (checking for completeness of the questionnaire and consistence of responses)

17. Data Collection and Processing (2) The questionnaires were then dispatched by Courier to the CBS (Nairobi) where they were received using a specifically designed checklist (district, cluster, sampled household) Data entry followed using data entry screens that had been developed and tested earlier. Information captured on the computer was verified and validated using inbuilt controls on variable ranges. Data was then weighted (using selection probabilities from the NASSEP III master frame) to enable estimation of population parameters. Necessary adjustments for population change and non response was then done Analysis involved production of frequency tables for selected variables. This information was then used during report writing

18. (VII) ACTIVITIESCOMPLETI ON TIME Preparatory Work (developing work plans, recruiting staff, project launch) Week 4 Research Work using Desk Research InstrumentWeek 8 Research Work on Expert Panel Instrument (identify experts, and train researchers, collect information, process the data (clean, validate and analyze the data), and write instrument specific report. Week 10 Research Work on National Sample Survey Instrument (questionnaire design, sample survey design including sample survey lists and maps, preparation of survey manual, preparation of data entry screen, training of enumerators, pre-testing the survey instrument, collecting data, processing the data (cleaning and editing and generating tables), and writing instrument specific report. Week 16 Focus Group Discussions (design guidelines and checklists for FGD, identify facilitators and participants, conduct FGDs and prepare FGD reports Week 14 Preparation of Country Report and National Programme of Action (review instrument specific reports, write thematic reports, consolidate thematic reports into full report, arrange for expert review, prepare final report) Week 22 National Workshop of Stakeholders (present findings to National Workshop and incorporate Workshop Views) Week 26 Country adoption of Report (after NEPAD APRM Secretariat Validation)

19. 8Indicative Budget 1 National APRM Secretariat175,000 2 Lead Technical Agencies and Conveners 323,600 3 National Bureau of Statistics (Data Collection) 140,000 4 Governing Council (Roll out meetings in the Provinces) 190,000 Total 828,600