1. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 HOLDING A MIRROR TO THE GOVERNMENT! EXPERIENCES WITH CITIZEN REPORT CARDS 1

2. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 OVERVIEW OF PRESENTATION The Context – Why Citizen Report Cards? The Concept – What is a Citizen Report Card? The Method – How Citizen Report Cards are designed? The Impact – Major outcomes of Citizen Report Cards The Learning – Critical lessons & experiences 2

3. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 WHY CITIZEN REPORT CARDS? Budget Formulation – How public resources are allocated Budget Review & Analysis – Diagnosing the implications of the budget when formed Expenditure Tracking – Seeing where the money goes Performance Monitoring – Even after the money is spent, see how the output/service is performing 3

4. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 A School Report Card Name: Gopa; Class: IX 4

5. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 LEARNINGS FROM A SCHOOL REPORT CARD… Power of MEASUREMENT Power of COMPARISON An Opportunity for REFLECTION A trigger for CHANGE & IMPROVEMENTS 5

6. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 Citizen Report Cards: Defining Features Pioneered by Public Affairs Centre as an independent assessment in 1993 Credible user feedback on public services Uses the power of measurement & comparison Communicates findings in a very easy and focused manner PUBLIC DOMAIN Report always in PUBLIC DOMAIN Not a one-off effort – continued benchmarking 6

7. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 CONTENT OF CITIZEN REPORT CARDS  FEEDBACK FROM ACTUAL USERS OF SERVICES REGARDING AVAILABILITY, ACCESS & USAGE OF SERVICES QUALITY & RELIABILITY INCIDENCE OF PROBLEMS & RESPONSIVENESS OF SERVICE PROVIDERS COSTS - CORRUPTION & FORCED INVESTMENTS SATISFACTION WITH SERVICE REASONS FOR DISSATISFACTION SUGGESTIONS FOR IMPROVEMENTS 7

8. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 STAGES IN CITIZEN REPORT CARD DEFINING SCOPE OF ACTION PREPARATIONS AND COLLECTING CITIZEN FEEDBACK RATING OF SERVICES DIALOGUE AND RESPONSE OF AGENCIES CITIZEN ENGAGEMENT IN REFORM PERIODIC BENCHMARKING AND PUBLIC REVIEW 8

9. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 THE “3” BIG QUESTIONS…  WHAT DO WE WANT TO KNOW? ASSESSMENT OF SERVICES WHAT COMMUNITY HAS TO SAY SINGLE SERVICE OR COMPARISON OF SERVICES  HOW CAN WE OBTAIN THE REQUIRED INFORMATION? WHICH POPULATION CAN GIVE REQUIRED INFO WHAT QUESTIONS WE NEED TO ASK WHAT WILL BE THE SAMPLING METHOD WHAT LEVEL OF SKILL SHOULD FIELDWORKERS HAVE  HOW WILL WE USE THE INFORMATION? WHAT CONCLUSIONS WILL BE DRAWN WHO WILL SEE THE REPORT WHAT ACTIONS WILL FOLLOW 9

10. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 METHODOLOGY FOR CARRYING OUT CITIZEN REPORT CARDS IDENTIFYING ISSUES THROUGH FGDs (PROVIDERS & USERS) IDENTIFYING ISSUES THROUGH FGDs (PROVIDERS & USERS) DESIGNING THE SURVEY INSTRUMENT DESIGNING THE SURVEY INSTRUMENT FRAMING A SCIENTIFIC SAMPLE FRAMING A SCIENTIFIC SAMPLE CONDUCT OF SURVEY CONDUCT OF SURVEY CODING, ANALYSIS & INTREPRETATION CODING, ANALYSIS & INTREPRETATION PRESENTATION OF FINDINGS PRESENTATION OF FINDINGS ADVOCACY & PARTNERSHIPS FOR SERVICE IMPROVEMENTS ADVOCACY & PARTNERSHIPS FOR SERVICE IMPROVEMENTS 10

11. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 CITIZEN REPORT CARDS ARE … BLENDING THE “ SCIENCE ” OF SURVEYS WITH THE “ ART ” OF ADVOCACY/REFORMS 11

12. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 CITIZEN REPORT CARDS IN BANGALORE THE POWER OF MEASUREMENT & COMPARISON 12

13. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 First Report Card (1993): small experiment with focus on survey & minimal advocacy  made quality of service a key governance issue Second Report Card (1999): planned effort of PAC, survey followed by advocacy which enabled  wide range of agency and govt. responses  growth in scope/scale of civil society action Third Report Card (2003): PAC effort  major improvement in service quality  major advocacy in progress CITIZEN REPORT CARDS IN BANGALORE 13

14. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 OVERALL SATISFACTION OVERALL SATISFACTION All round overall improvement % Satisfied 14

15. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 OVERALL SATISFACTION – Poor Vs Rest OVERALL SATISFACTION – Poor Vs Rest % Satisfied 15

16. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 AGENCY WISE SATISFACTION AGENCY WISE SATISFACTION % Satisfied agency 16

17. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 AGENCY WISE SATISFACTION - SLUM AGENCY WISE SATISFACTION - SLUM % Satisfied agency 17

18. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 PROBLEM INCIDENCE % Satisfied Significant reduction in problems while using services 18

19. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 STAFF BEHAVIOUR STAFF BEHAVIOUR Major improvement in satisfaction with staff behaviour 19

20. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 CORRUPTION 20

21. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 APPLICATIONS OF CRCs As a “ Diagnostic Tool ” As an “ Accountability Tool ” As a “ Benchmarking Tool ” 21

22. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 22 Background  The pilot CRC project in Tajikistan was designed and implemented by United Nations Development Programme (UNDP) through a local partner –Zerkalo.  Public Affairs Foundation (PAF), Bangalore, provided technical assistance for the pilot project, which was funded by the UNDP.  Following a workshop with participation of UNDP Democratic Governance Programme, government agencies, and selected research company (El-Pikir Center for Public Opinion Study), a task force was set up to carry out the study in Kyrgyzstan.

23. Tajikistan- Opinions of Health Services by Aspect CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 23

24. Tajikistan- Overall Opinions on Health Services CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 Percent Very good 48.8 Somewhat good 28.8 Somewhat poor 2.0 Very poor 20.5 24

25. Kyrgyzstan- The Overall Scorecard on the Services of the Oblast Hospitals CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 25

26. Bangalore- Public Hospitals 1999 (general households) CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 Client satisfaction with agency interactions Percent satisfied (1994)25 Percent satisfied (1999)43 Percent dissatisfied (1994)19 Percent dissatisfied (1999)6 The bribery matrix Proportion in sample claiming to have paid a bribe (%) (1994)17 Proportion in sample claiming to have paid a bribe (%) (1999)24 Average payment per transaction in Rupees (1994)396 Average payment per transaction in Rupees (1999)289 26

27. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 Bangalore- Public Hospitals 1999 (general households) 27

28. Bangalore- Public Hospitals 2003 (general households) CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 IndicatorService provider Type of hospital visited in last I yearState government CorporationPrivate % respondents (multiple responses possible)891012 Government hospitals (% respondents) Quality of staff service% satisfied Time to attend83 Overall behaviour of doctors86 Overall behaviour of nurses100 Overall behaviour of other staff83 Helpfulness of staff75 Service quality for in-patients% satisfied Cleanliness of room31 Quality of food12 Cleanliness of linen12 Overall satisfaction with service% satisfied Completely satisfied37 Partially satisfied36 Dissatisfied2 Bribe demanded100%* Bribe paid2% 28

29. Bangalore- Public Hospitals 1999 (slum households) CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 29 Service provider (Public Hospitals)Indicator Speed Money Phenomenon Percentage of those who paid53 Average amount paid (in Rupees)640 Percentage of persons from whom bribe was demanded *63 Willingness to pay more for improved service Percentage willing to pay more for better services53 Percentage willing to pay over and above the current payment32 Improvement in services over last three years –peoples’ perception Overall Quality of Services (%)79 Behaviour of Staff (%)74 Ease of interaction (%)67 Client satisfaction with agency interactions Percent satisfied 199480 Percent satisfied 199973 Primary health centre available Percentage of respondents saying it is available56 Percentage of respondents saying they use it84

30. Bangalore- Public Hospitals 2003 (slum households) CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 30 IndicatorService provider Type of hospital visited in last I yearState government Corporation Private % respondents (multiple responses possible)8616 14 Government hospitals (% respondents) Quality of staff service% satisfied Time to attend69 Overall behaviour of doctors72 Overall behaviour of nurses73 Overall behaviour of other staff70 Helpfulness of staff67 Service quality for in-patients% satisfied Cleanliness of room80 Quality of food73 Cleanliness of linen70 Overall satisfaction with service% satisfied Completely satisfied55 Partially satisfied33 Dissatisfied12 Bribe demanded85%* Bribe Paid9%

31. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 Bangalore Government Hospitals-Quality/Reliability Some Findings  99% slum respondents report doctor’s chamber as clean.  34% report presence of doctors at the time of visit.  37% said all the required medicines were available.  11% report issue of expired medicines.  69% completely satisfied with the time taken. by the government hospital staff to attend to them. Follow up  Board of visitors constituted.  Help desk for patients.  Constant internal reform from the health department and health authorities. 31

32. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 POWER OF DIAGNOSIS 32

33. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 Corruption & The Urban Poor (SURVEY IN BANGALORE, 1999) 33

34. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 POST SURVEY ACTIONS Pre-launch presentations to the providers & elected representatives Inter agency workshop to stimulate learning and sharing good practices Public release of findings & public forum – “open house” with agency heads Sustained media advocacy Supporting local champions 34

35. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 INSTITUTIONAL FORMS Individual Civil Society Organizations – PAC & TIB Civil Society Partnerships – People’s Voice Project, PANE Independent Multi-Stakeholder Consortiums – Kenya, Tanzania Governments – Internal ( Vietnam ) & Open ( Delhi ) 35

36. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010  Counting and discussing instead of just shouting  Presenting strengths and weaknesses – “pat” & “slap”  Enables policy makers set policy priorities  Helps agency managers assess service efficiency  Provides a bridge for civil society to dialogue on citizens’ priorities  Transmits the voice of the poor without intermediation by representatives MAJOR POINTS OF INFLUENCE 36

37. CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 VARIETY IN IMPACT OF CRC’S AGENCIES DISCUSS PERFORMANCE WITH CITIZENS IN OPEN FORA IN BANGALORE SYSTEMATIC CITIZEN WATCH DOG ROLE IN LOCAL GOVT. IN TERNOPIL, UKRAINE FORCING POLITICAL ACCOUNTABILITY IN MUMBAI SLUMS INDEPENDENT APPROACH TO MONITOR PRO-POOR SERVICES IN ZANZIBAR, ETHIOPIA & TAJIKISTAN LOWER LEVEL OFFICIALS CITE FINDINGS TO SEEK FUNDS & SUPPORT IN MUMBAI, INDIA POLITICAL LEADERSHIP ASKS FOR MORE DIRECT FEEDBACK IN DELHI, INDIA REGULATORY BODIES SEEK INDEPENDENT VOICES IN KENYA BENCHMARK LOCAL SELF GOVERNMENTS IN INDIA 37

38. PLANNING FOR A CRC ACTIVITYREQUIRED COMPETENCYTIME FRAME Identifying Scope Knowledge of service provision Access to technical resources 2 Months Conducting the survey Field work management Trained pool of enumerators Supervising quality of survey 2 weeks - 2 months Post survey analysis Data Entry & Analysis Analytical Report writing 2-3 months Dissemination of findings & Advocacy Stakeholder analysis Communicating to different groups 1-2 months Improving Services Ability to work with different stakeholders Imagination & creativity 2-6 months CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010 38