Client Satisfaction Survey what’s possible Belgrade, July 2010 Report on the findings of the CSAI Success Grants Applicants’ Satisfaction with the Grant Application Process and Procedures

Introduction Research Objectives Key Findings Analysis Content

ISC/CSAI announced an RFA for its Success Grants Program on March 17, The purpose of this RFA was to solicit project proposals from CSOs operating in Serbia for the purpose of using mass media tools and new technologies to successfully promote their results and impacts in order to increase the overall visibility of CSOs. In the second halh of May 2010, ISC/CSAI organized an on-line survey to find out what Success Grants Applicants think and feel about CSAI’s application process and procedures. The main purpose of the survey was to collect feedback in order to change and adjust future CSAI Requests for Applications. The survey was anonymous; respondents did not have to leave any records regarding their organizations or themselves. Also - as it was noted to respondents - their participation in the survey, as well as their opinions and attitudes, did not in any way affected the evaluation of their project proposals. Introduction/ Background

Method of data collection: Internet Based Questionnaire Population: 111 CSAI Success Grants Applicants Targeted Sample: All Applicants Sample Size: N=102 (92% of the Total Population) Data collection was conducted from May 10 – 27, 2010 Introduction/ Methodology

The objective of this research was to collect and analyze applicants’ feedback regarding: 1.Manner of RFA announcement and channels used; 2.Application Documents (including ISC Application Form and ISC Budget Form); 3.RFA timelines; 4.Project Proposal method of submission to ISC; 5.General attitudes about the Application process and procedures. Research Objectives

How would you rate the overall application process? Key Findings/ Overall Satisfaction with the Application Process It is fair to say that almost all respondents are satisfied with the Application Process in general, with 46% of respondents stating Absolutely satisfied, and 44% stating Satisfied. Base: N=100 Average Score (on a scale of 1- 5)

Key Findings/ Information (Re)sources Where did you first find out about ISC/CSAI’s Success Grants RFA? Top Channels Other Channels ISC Serbia website, ISC mailing list and CDNPS mailing list seem to be the most valuable sources of information for CSO’s interested in ISC/CSAI RFAs. Receiving inputs from other CSO’s regarding funding opportunities seems to be another way of acquiring the information. Base: N=102

Key Findings/ RFA Narrative Did the Narrative Section of the RFA clearly articulated what kind of projects ISC/CSAI intends to support trough the Success Grants Program? Average Rate (on a 1-4 rating scale) The Success Grants RFA Narrative was highly rated among all respondents, with average rate of 3.44, on a 1 -4 scale. Half of the respondents consider the RFA Narrative to be Absolutely Clear. Base: N=101 6%6%

Key Findings/ YouTube video How did you feel about having the option of using YouTube videos as part of the application? Average Score (on a scale of 1- 4) Submitting YouTube videos as part of the application for the Success Grants RFA was highly rated amongst a vast majority of respondents, with the average scores of 3.42 on a 1-4 point scale. More than half of the respondents Liked very much that option. Base: N=101

Key Findings/ Application in English How difficult was it to complete the application in English? Completing the application in English was easy for the majority of respondents. Only 3% of respondents found it to be somewhat difficult. Base: N=101

Key Findings/ Relevance of ISC Application Form Questions Where questions in different sections of the ISC Application Form relevant? More than 90% of respondents feel questions in the Application Form were Absolutely Relevant or Relevant. Seems that follow-up activities and Org’s experience in media campaigns caused some minor misunderstandings. Base: N=101 96% 99% 96% 95% 99% 95% 94% 98% 94% RELEVANCE

Key Findings/ Budget Form I How difficult was it to complete the ISC Budget Form? More than two thirds (77%) of respondents didn’t find it difficult to complete the budget form, in general. However, some respondents had some difficulties regarding different features of the ISC Budget Form. Classification of the findings can be found on the next slide.on the next slide Base: N=101

Key Findings/ Budget Form II Total number of comments: 18 Positive: 2 comments Negative: 16 comments Defining and Understanding Budget Lines 7 Comments Understanding Cost Share: 2 Comments Time Constraint: 1 Comment Language: 1 Comment General Looks of the Budget Form: 1 Comment Miscellaneous: 4 Comments Base: N=18 Was there anything particularly difficult about the ISC Budget Form?

Key Findings/ Budget Instructions I Did the Budget Instructions provided clear directions as to how ISC funds should be used by grantees? Half of the respondents responded that the budget instruction were Mostly Clear. In general, 88% of the respondents feel that the budget instructions were clear. However, a number of respondents had some difficulties to clearly understand specific Budget Instructions. Classification of these difficulties are presented on the next slide.next slide Base: N=101

Key Findings/ Budget Instructions II Total number of comments: 13 Positive: 1 comments Negative: 12 comments Ratio between project and Other costs: 6 Comments Unclear Budget Categories: 2 Comments Instructions Need Additional Clarifications: 2 Comments The Way Budget Categories are Calculated: 1 Comment Definition of the In Kind: 1 Comment Base: N=13 Where there any specific budget instructions you found hard to understand?

Key Findings/ Time Was the 1-month timeframe sufficient to complete the application? More than three quarters of respondents had enough time (6% had more than enough) to complete the application. However, a significant number – almost one quarter of all respondents - needed more than a month, mostly because it took time to reach agreements with media outlets (key partners in this program). Base: N=101

How does your organization prefer to submit applications? Key Findings/ Application Submission Using is the most preferred way to submit project proposals. Base: N=100 Average Score (on a scale of 1- 4)

A high response rate to the survey is the first “result” that merits attention, and safely allows for conclusions to be drawn. The fact that 102 organizations, out of 111 (92%), took part in the survey, leads to the conclusion that CSOs are very interested and motivated to participate in these kinds of surveys and that they appreciate the fact that somebody (no other than donor organization) wants to hear their opinions and attitudes. Like one of the respondents noticed: “It’s nice of you to ask about these kinds of things. I have a feeling that most of the donors are not doing this (client satisfaction surveys), which leaves their application procedures non-transparent for those using them.” It can be argued that organizations felt pressured to take part in the survey, given that the survey was organized during the project proposal evaluation process, despite the fact that they were, on several occasions, assured that the survey was anonymous; and that their participation in the survey, as well as their opinions and attitudes expressed within, would in no way affect the evaluation of their proposal. This “pressure”, and the general type of relationship existing between donors and CSOs is something that should be taken in consideration when interpreting results. Also, this high response rate allows us to safely make conclusions in regards to the entire population. Results Analysis

Overall satisfaction: Having 90% of all respondents satisfied with the overall Application Process (46% of respondents feeling Absolutely satisfied; and 44% feeling Satisfied) is an impressive achievement. Application Form: More than 90% of respondents felt that the questions in the Application Form sections (Justification and Background, Project Goal and Objectives, Expected Results, Target Groups, Description of Activities, Management Team, Follow up Activities, Brief description of organization, Organization’s experience in media campaigns) were Absolutely Relevant (score ranges from 64% to 85%, depending on the section) and/or Relevant (14% to 34%). It seems that only follow-up activities and organization’s experience in media campaigns caused some minor misunderstandings. RFA Announcment: The Success Grants RFA Narrative was highly rated by all respondents, with an average scores of 3.44 (on a scale of 1 – 4). One half of the respondents consider the RFA Narrative to be Absolutely Clear. Results Analysis

Budget Form & Instructions: The budget form did not cause too many difficulties to respondents, at least according to the survey results. However, compared to the Application form, some features caused misunderstandings or where not clear enough. “It was not elaborated enough how detailed budget items should be, so we used our own judgment; It would be good to adjust budget form to the specific media production costs, in order to make it clearer; It was not clear which costs go under which budget line. We were not able to fully understand the form without additional consultations with ISC staff; We were not sure whether honoraria for consultants we assigned for the project activities, should be placed in budget line no.2 or no 9, etc.” Exactly half of all respondents think that the budget instructions were Mostly Clear. In general, 88% of the respondents feel that the budget instruction were clear. However, some respondents had difficulties to clearly understand specific budget instructions: “It was mostly clear in general, but it was not clear and precisely defined what is air time, does it refer to TV advertisement only, or to general advertisement trough official channels, the usual channels of media communication; In the budget instructions it is asked for a reasonable ratio between administrative and program costs, and the preference will be given to those proposals that do not exceed 40% of administrative costs compared to program costs. It is not logical to have personal salaries as administrative costs. Not clear!!!-Confusing!!!; The ratio between project and non-project costs is not in accordance with the needs of the project, etc. Results Analysis

Getting the word out: The ISC Serbia website, ISC mailing list and CDNPS mailing list seem to be the most valuable sources of information for CSOs interested in ISC/CSAI RFA, with 21%, 13% and 12% respectively. Receiving inputs from other CSO’s regarding funding opportunities seems to be another effective way of acquiring the appropriate information (17% of organizations stated that other organizations were their primary source). It is safe to assume that distribution channels amongst organizations which served as primary sources of information was similar to distribution amongst survey participants. Results Analysis

Thank You