1. Quality & Impact in DE defined: a selection
An overview of selected informants
Harm-Jan Fricke
October 2011

2. DARE Forum so far (May 2011) Q&I of DE depends particularly on
Quality of organisation and management of work
Quality of ways of working with audiences, partners, beneficiaries
Specific to DE are issues of:
Coherence:
between values and actions, between actions and relevance to the own organisation and the audience/beneficiaries/partners
Partnerships:
explicitly defined and operated
Participation:
In methods used and in participants defining work done
Perspectives:
Plurality of viewpoints

3. Johannes Krause: Global Education quality standards
Project/ programme design
based on quality standards
Monitoring& evaluation
(a) M&E instruments, (b) M&E practice
Official policies:
quality standards translated into grant guidelines, funding criteria
Organisational Quality Management
Sharing good practices/ lessons learnt within the org. & with networks; peer reviews (e.g. Norway), etc.
Work on quality frameworks
(a) in the organisation, (b) in networks, (c) for policy level

4. Irish DEA: Quality and Measuring the Impact of DE
Attention to the context in which work takes place: the relationship between the ‘development agenda’ and the ‘learning agenda’
Distinguish between process and impact evaluation: show progress against defined milestones or progress markers
Suggested criteria and indicators at output level:
# of schools/groups/individuals participating in a programme/course/event
# of sessions/meetings held
% of participants who complete a programme
# of publications/resources/webpages produced
# of visits to a website
# of posts to a blog, facebook page etc.
# of people who join an organisation
Criteria and indicators at outcome level:
% of participants who take an action/ join relevant activist groups or campaigns
% of participants who go on to further study/training in the area
% of participants who change their consumer behaviour and/or lifestyle choices
Progress towards ‘markers’ of desired behavioural changes

5. Irish DEA: example of Q&I at national levels
Output criteria
Indicator
Cooperation with DES is enhanced
Number of meetings with the DES
Number of times DE is mentioned at the cross-departmental working group
Educational policies contain DE messages
Number of educational policies containing dev ed messages
Quality Research
Number of research articles on DE
Outcome criteria
Indicator
Cooperation with DES is enhanced
(Enhancement is defined as increased awareness and buy-in from people at senior management level and increased commitment to integrate messages)
Senior members of DES attend DE events and are open to conversations and dialogue
DE becomes visible in DES communications and policies
Educational policies contain DE messages
Quality and depth of dev ed messages contained in policies
Quality Research
Indicators for research should follow academic standards (peer- reviews, quality of publications, etc.) but also bear in mind that relevance for practice is an important indicator of success for quality DE research.

6. ACODEV – COPROGRAM: Quality in Development Education
Looking for quality in DE means the following:
awareness of the relationship DE action - context, culture, reality, power relationships (society, north-south relationships, institution, funding,…)
To act in coherence with our values.
Quality on 2 main levels,
Level 1 : the 3 main groups of actors involved in a DE action : participants, educators (and the institution to which they belong) and partners
Level 2 : the 3 main phases in the setting up and development of any DE action: action design, action process or processes, results and outcomes

8. Teles and Schmitt: Elusive craft of evaluating advocacy
“Successful advocacy efforts are characterized not by their ability to proceed along a predefined track, but by their ability to adapt to changing circumstances”
Attribution of effect and of effectiveness is difficult
Therefore:
Assess the (policy-social) context within which interventions take place
Use a "spread betting" approach to advocacy, and thus to evaluation since success or pay-off is rarely due to any one project/intervention alone
Adopt a Long Time Horizon
Pay Attention to Policies’ Impact on Politics, Not Just on Target Populations
Evaluate organisations as a whole, not the single project

9. DESS 2011: Q&I in advocacy, campaigns and events
Clarify concepts and definitions:
Advocacy v. Campaigns
Advocacy v. Awareness Raising
Advocacy/Campaigning v. Education
Q&I shown through:
Strategic approach to do with for instance:
Choice of topic/theme/issue;
Problem analysis
Stakeholder analysis
Definition of objectives
Critical path/Intended change process
Participation including to do with:
Vision
Values
Reflection
Theory of change made explicit

10. DESS 2011: Q&I in adult education
Criteria:
Participation of audience/target group and relevance of issues to learners’ needs
Methodology to promote engagement, motivation, critical reflection
Engagement as outcome: with skills useful in various situations

11. DESS 2011: Q&I in youth work Issues to address include:
Plan for impact
Address relevance
Show coherence with vision
Be based on explicit values
Use participatory approach
Show different perspectives
Stimulate reflection (on what is being gained)
Be sensitive to gender and other power relations

12. DESS 2011: Q&I in resources
Does it engage the user in questioning, discussion and action?
Is it relevant and educational
Does it show different perspectives on the issue – including minority and southern
Does the resource enforce/reinforce stereotypes?
Is it relevant to the target group
Is it accessible to the user
Is the information accurate and up-to-date

13. OECD-DAC & EC: evaluation framework for projects
Relevance
Effectiveness
Efficiency
Sustainability
Impact
DG DevCo: the above plus
Coherence
Added value

14. Next steps for the Q&I working group?
Given the foregoing and
given the DEEEP objective (a pan-European quality framework on Development Education):
What can the Q&I group do next?
What is needed?
What is useful?