Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Citizen Report Card Methodology THE BUILDING BLOCKS

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Overview of CRC Methodology Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling Design

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The CRC Journey Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling Design

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Examples of Research Objectives General-  Getting feedback from people on quality of public services Specific -  Finding out satisfaction with quality of service, satisfaction with behaviour of staff, extent of corruption, problem resolving capabilities of the agency

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The CRC Journey Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling Design

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Designing the Instrument Basic structure of a schedule/ questionnaire  Investigator information  Introduction  Filter questions  Demographics  The body of the schedule - sections if needed - sub- schedules if required

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Thumb Rules for a Schedule  All research objectives to be covered  Shortest length possible  Language - simple and convenient based on respondents’ and investigator’s level of understanding  Clear instructions for the investigator

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Basic Tools in Instrument Design  Different types of questions Open-ended and closed questions / combinations yes/ no questions  Coding - pre and post coding  Use of show cards  Scaling - 5 pt or 2 pt  Skips  Grids

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore CRC Process Chart Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Sampling Design  Why sampling?  Importance of representativeness  How to assure representativeness Fixing quotas by area or category or both Choosing appropriate sample size time constraints and cost constraints standard error level of sub classification required for analysis and conclusions

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore SAMPLING DESIGN BMP:  606 general households  812 slum households General Households:  10 wards from each of 3 zones selected  Number of households allocated according to population of ward  Mains and crosses selected from each ward  Proportionate number of households selected from each street MAIN SAMPLE: 2000 households from city and suburban areas:  800 general households  1200 slum households Slum Households:  Slums selected from each region using PPS  Proportionate allocation according to slum type & size within regions  Households selected using right-hand rule in each slum through systematic sampling

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The Research Journey... Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Field Issues...  Method of data collection  Right field team  Training/Briefing of investigators  Regular reporting  Spot checks and back checks  Time schedule  Adherence to budget

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The Right Team How do we decide?  Familiarity with locality, language  Able to relate to target audience  Intelligent  Available for the entire duration of fieldwork  And, INTEGRITY

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The Right Team How many?  f {Sample size, Timeline, Productivity }  Sample size : 2000  Timeline : 20 days  Productivity per person/ day : 5  Team size = [(2000/20) / 5] = 20 Investigators

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The Final Team... Field Supervisor Field Supervisor 5 Interviewers Field Manager 5 Interviewers Field Coordinator Field Coordinator Field Supervisor Field Supervisor Field Supervisor Field Supervisor Field Supervisor Field Supervisor

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Data Collection  Random sampling  Starting points selection Sample Size : starting points 10 listings around each starting point  Right hand rule  Boosters-- phone call, exit interviews

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Quality Checks At all stages  Before launching fieldwork  During fieldwork  After fieldwork

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Quality Checks Before fieldwork  Pilot exercise  interviews  Check for Length Comprehension Flow Sensitiveness

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Quality Checks During fieldwork  Accompanied interviews  Sample of each interviewer’s work spot checked by field supervisor  10% back checks by the field coordinator  100% scrutiny of schedules

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Quality Checks After fieldwork  Sample sizes  Quotas  Representation

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Cost Heads in Fieldwork  Cost is a function of Nature of project Type of respondent Selection criteria Sample size, sampling  Two broad types Overhead Expenses

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Cost Heads in Fieldwork  Overheads Time cost, personnel Communication: telephone, computers Courier / mailings

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Cost Heads in Fieldwork  Expenses Printing of questionnaires Briefing and piloting Interviewer fees Supervisor fees Local conveyance Outstation travel : Field officer, interviewers Others: Venue, equipment, moderation, gifts, transcriptions Translation

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The Research Journey... Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Data Analysis  Preliminary analysis by research agency  Further analysis by PAC

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Framework of Analysis  Select computer package - Foxpro, SPSS  Run basic demographics  Check if this is in line with population  If yes, proceed with further analysis  If no, weight data to represent population

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Framework of Analysis  What is the depth of analysis required?  What are the relevant analysis heads? By demographics By agency By responses to other questions : satisfied vs. dissatisfied had problem vs. did not have problem paid bribe vs. did not pay bribe  Simple measures used- percentages and mean scores. Recently regression models

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore The Research Journey... Defining Research Objectives Development of Instrument Data collection Analysis Interpretation Sampling

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore  Compare across agencies - rank them on satisfaction, responsiveness and corruption- using percentages and means  Within agencies - specific information - timings of water supply, quality of water - again using percentages and means  Two ways of interpreting same information - eg - number of visits to agency Interpretation of Data

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore Interpretation of Data  Assimilation of scattered findings into cohesive whole  Translation of findings into diagnostic statements  Conversion of diagnosis into conclusions  Formulation of recommendations based on conclusions

Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore From Findings to Recommendations  Satisfaction with behaviour of staff  Objective statement of finding - percentage satisfied, dissatisfied, ambivalent.  Conclusion - general / high/low satisfaction/dissatisfaction  Recommendation - training for staff to improve their behaviour