1. 1 Contact: claudia.bergmueller@ewf.uni-erlangen.de From Evaluation to Research Description of a continuum in the field of Global Education

2. 2 Contact: claudia.bergmueller@ewf.uni-erlangen.de 1.Evaluation and Research 2.From Evaluation to Research – a continuum 3.Models of Evaluation in the field of Global Education and their relevance for Research 4.The analysis of learning effects as a challenge for Evaluation and Research Overview

3. 3 Contact: claudia.bergmueller@ewf.uni-erlangen.de Evaluation and Research

4. 4 Contact: claudia.bergmueller@ewf.uni-erlangen.de “The activity of scientific research is to establish rates or systems of propositions and to systematically review, in the empirical sciences in particular hypothesis, theory, systems which are installed and checked against experience by observation and experiment." (Popper, 1966: 3) Research … 1.is a systemic activity (about a "system of records" or a "system of knowledge"); 2.is directed towards reality (empiricism); 3.works with the help of hypotheses, theories, and other means, and 4.arrives at general statements ("systems theory" be established and verified, or "causal relationships and regularities") Research

5. 5 Contact: claudia.bergmueller@ewf.uni-erlangen.de Standards: 1) The results need to be provable. 2) The results need to be intersubjectively verifiable. Research

6. 6 Contact: claudia.bergmueller@ewf.uni-erlangen.de Evaluation is …  is a systemic activity  is directed towards reality (empiricism);  works with the help of pre-determined quality standards  arrives at project-related statements (assessment of strengths and weaknesses of a certain project reality) Standards:  Utility  Feasibility  Propriety  Accuracy Evaluation

7. 7 Contact: claudia.bergmueller@ewf.uni-erlangen.de Evaluation and Research EvaluationResearch ConcernValuation of practice based on verifiable criteria Generation of knowledge about a field of practice FocusFocus on a caseGeneralization as perspective StandardsStandards of carrying out an evaluation on a case Intersubjectivity of the valuation needs to be verifiable Sufficient data base for the generalization Generation of knowledge needs to be intersubjectively verifiable

8. 8 Contact: claudia.bergmueller@ewf.uni-erlangen.de From Evaluation to Research

9. 9 Contact: claudia.bergmueller@ewf.uni-erlangen.de From Evaluation to Research EvaluationFrom Evaluation to Research Research Concern Valuation of practice based on verifiable criteria Using the knowledge from the evaluation without inheriting the attitude to improve Generation of knowledge about a field of practice Focus Focus on a caseSeveral similarily arranged evaluations Generalization as perspective Standards Standards of carrying out an evaluation at a case Intersubjectivity of the valuation needs to be verifiable Sensitivity for the evaluation‘ s limitations Data collection with methods that permit intersubjectivity Sufficient data base for the generalization Generation of knowledge needs to be intersubjectively verifiable

10. 10 Contact: claudia.bergmueller@ewf.uni-erlangen.de Models of Evaluation and their relevance for Research

11. 11 Contact: claudia.bergmueller@ewf.uni-erlangen.de We can distinguish three models… 1.Concept Evaluation 2.Performance Evaluation 3.Impact Evaluation Models of Evaluation

12. 12 Contact: claudia.bergmueller@ewf.uni-erlangen.de Examples: 1)Evaluating the quality of an educational programme’s concept … 2)Evaluating the quality of teaching material … …with normatively defined quality standards/criteria Possible aspects of concept evaluations: -Objectives -Content -Educational Methodology... Concept Evaluation

13. 13 Contact: claudia.bergmueller@ewf.uni-erlangen.de Possible questions: Concerning the objectives: Do the educational objectives of an activity relate adequately to the target group? Are these objectives expressed in an understandable way? Concerning the content: Does the content relate appropriately to the objectives / the target group? Does the chosen content and its didactic format comply with the quality criteria of Global Education? Concept Evaluation

14. 14 Contact: claudia.bergmueller@ewf.uni-erlangen.de Examples: 1)Evaluating the implementation of a Global Education project in a school 2)Evaluating the organisational structures of an institution Possible aspects of performance evaluations: -Project management -Transparency -Educational Methodology.... Performance Evaluation

15. 15 Contact: claudia.bergmueller@ewf.uni-erlangen.de Possible questions: Concerning the project management: Does the allocation of responsibilities correspond to the working conditions? Concerning the educational methodology: Is the learning material used properly / adequately for the particular target group? Performance Evaluation

16. 16 Contact: claudia.bergmueller@ewf.uni-erlangen.de Examples: 1)Evaluating the widespread impact of a certain material 2)Evaluating the impact of an activity / project on a particular target group Possible aspects of impact evaluations: -Spread -Satisfaction -Learning effects … Impact Evaluation

17. 17 Contact: claudia.bergmueller@ewf.uni-erlangen.de Possible questions: Concerning satisfaction: In which aspects are the stakeholders / target groups content with the performance / conduction of the project? Concerning the learning effects: How do the target groups assess their own attainment of knowledge? What real learning effects can be recognized after a certain project / activity? Impact Evaluation

18. 18 Contact: claudia.bergmueller@ewf.uni-erlangen.de Concept Evaluation and its relevance for Research Concept Evaluation: comparison of concept and quality standards Research:comparison of concept A and quality standards comparison of concept B and quality standards comparison of concept C and quality standards To which extend do the practical concepts comply with the theoretical discourse or the normative quality standards/ criteria in the field of GE/DE? Sociological interest in the correspondance of norms of a practical field

19. 19 Contact: claudia.bergmueller@ewf.uni-erlangen.de Performance Evaluation and its relevance for Research Performance Evaluation: How is the project being carried out? Has the performance been aim-oriented? Research:Performance Evaluation A Performance Evaluation B Performance Evaluation C Organizational Development Project Management Interest from the perspective of theories on organizational development and theories of project management

20. 20 Contact: claudia.bergmueller@ewf.uni-erlangen.de Impact Evaluation and its relevance for Research In the field of GE/DE 3 different types of impact can be distinguished: 1.the widespread impact through a project or activity, e.g. the number of people reached, the number of material distributed, the number and type of media coverage etc., 2.the impact on the institutional level, e.g. the strengthening of a field of activity or the extension of projects in GE/DE within a NGO, 3.the impacts on the level of the target groups; in this case, another distinction can be made between the effects on the level of satisfaction statements, self- mentioned/reported learning and real learning effects.

21. 21 Contact: claudia.bergmueller@ewf.uni-erlangen.de Impact Evaluation and its relevance for Research Impact Evaluation: Widespread impact Impact on institutional level Self-identified/reported learning (Cognitive) Learning effects Research:Impact Evaluation A Impact Evaluation B Impact Evaluation C distribution/acceptance self-reflective competencies real learning effects Interest from the perspective of communication and media research Interest from the perspective of research into teaching and learning

22. 22 Contact: claudia.bergmueller@ewf.uni-erlangen.de The analysis of learning effects as a challenge for Evaluation and Research

23. 23 Contact: claudia.bergmueller@ewf.uni-erlangen.de Analysis of learning effects as a challenge Input e.g. quality of material Impact Learning activities e.g. active use of time Individual preconditions Context Mediators e.g. motivation Teacher/ Trainer/ Educator

24. 24 Contact: claudia.bergmueller@ewf.uni-erlangen.de Analysis of learning effects as a challenge 1)Learning is very complex. There is no linearity of teaching and learning. Learning (esp. change of human behaviour) isn’ t always a logical response to a certain activity. 2)Learning has different dimensions. - Knowledge - Competencies concerning values and judgement - Competencies/Strategies concerning communication and action

25. 25 Contact: claudia.bergmueller@ewf.uni-erlangen.de Analysis of learning effects as a challenge

26. 26 Contact: claudia.bergmueller@ewf.uni-erlangen.de Analysis of learning effects as a challenge Analysing the acquisition of knowledge:

27. 27 Contact: claudia.bergmueller@ewf.uni-erlangen.de Analysis of learning effects as a challenge Analysing the acquisition of competencies concerning values and judgement: Decision and action alternatives by giving short tales concerning topics, which the participants have to weigh against each other or state reason for their choice Focus: To what extent does an activity create the necessary cognitive preconditions / qualifications (!) for the intended change in values and judgement to become most probable?

28. 28 Contact: claudia.bergmueller@ewf.uni-erlangen.de Analysis of learning effects as a challenge Analysing the acquisition of competencies concerning communication and action / strategies: Scenic challenge-situations for which future solutions have to be found Focus: Examination of certain prevailing conditions like e.g. - the capability / ability to predict future developments - the capability / ability to set individual goals - the capability / ability to shape change processes

29. 29 Contact: claudia.bergmueller@ewf.uni-erlangen.de Conclusion We need … … to gain experience with the synopsis of evaluations which are carried out according to similar criteria. … to complement evaluation with research by taking out small areas that can be dealt with according to the standards of research. To do so it is necessary … … to find more adequate methods concerning different evaluation targets … that evaluators and researchers get into contact with each other.

30. 30 Contact: claudia.bergmueller@ewf.uni-erlangen.de Thank you very much!