1. The EVA Barometer Signe Ploug Hansen Special Adviser at the unit for methodology and IQA The Danish Evaluation Institute (EVA) sph@eva.dk

2. Overview of the presentation What is a barometer good for? Why did EVA go for a barometer? What is the content of the EVA Barometer? What kind of information does a barometer provide? What does it take to use the barometer concept? How to get started?

3. What is a barometer good for? Can summarise external (as well as internal) feedback by providing performance indicators Provides you with overall expressions of the performance of your agency Represents a strong communicative tool for informing external stakeholders about the performance Validates the information you get from individual surveys

4. Why did EVA go for a barometer? Reduce information overload created by a growing number of internal and external feedback surveys Benefit from the fact that many isolated surveys had much overlapping in terms of themes Achieve an overall view of EVA s performance in terms of these common themes Be able to provide clear and simple information about the performance of the agency

5. The content of the EVA barometer 1/2 The seven themes: The quality of information provided by EVA EVAs abilities in terms of cooperation EVAs abilities in terms of process management EVAs knowledge & abilities in terms of evaluation methods EVAs sector specific knowledge and understanding The usefulness of the quality assurance process The quality of the final evaluation report.

6. The content of the EVA barometer 2/2 Questions of the theme information (translated versions): How satisfied or dissatisfied were you with EVAs information about: 1.the process of the project? 2.the content of the project? 3.the method(s) applied? 4.your workload? Possible answers: Satisfied, Largely satisfied, Largely dissatisfied, Dissatisfied

7. What kind of information does a barometer provide? The information theme as an example: External experts Quality assured institutions/programs Givers of assignments Receivers of EVAs quality assurance surveys 12341234 12341234 12341234 80 % 83 % 87 % Information. Barometer: 83% Identical questions regarding information Percentage of positive answer (satisfied + largely satisfied) (ex) Barometer

8. What does it take to use the barometer concept? A number of equally relevant surveys focusing on the same themes related to the performance of the agency Several questions related to each theme in each of the surveys Identical questions in each of the surveys Some years of stability concerning the content of the surveys and thus the activities of the agency Methodological and technical competencies to construct valid surveys and analyse the data

9. How to get started? Identify relevant surveys for the barometer Identify relevant themes Formulate a minimum of four questions related to each theme. Ensure that the questions make sense in all the surveys by testing them among potential respondents Start using the new questions in all the relevant surveys at the same time