Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together Research project on Evaluation Evaluation team: Michael Bommes and Ulrike Wolff- Jontofsohn (University of Education Freiburg)

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 2 Betzavta-Evaluation The Research Process: ◦The Programme ◦The Funders: Demands and Expectations ◦The Research team: basic assumptions about evaluation ◦The Evaluation Design Main Findings and Conclusions ◦Programme ◦Implementation –the Organisational Framework ◦Training Quality: Staff and Multipliers ◦Success and/ or Failure??

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 3 Betzavta-Together Betzavta-a Democracy and Human Rights education Programme -Developed by the Adam- Institute in Jerusalem/Israel(1986) Closely connected to the Israeli Situation Educational Goals: educate democratic citizens, educate all members of a community to accept equality and freedom as basic democratic principles Approach: Social and moral education by creating dilemmas and conflictual situations(Piaget, Kohlberg) Facilitation Techniques: focus on group dynamics and conflict elaboration

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 4 Betzavta-Methods: Two Types of Activities Type 1: Creating a Dilemma Structure: ◦group-dynamic activity ◦Perception and reflection ◦Conflict elaboration within normative aspects Type 2: Learning the Democratic Discourse Structure: ◦individual task of opinion forming ◦Search for consent in small groups ◦Market Place - Exchange of Opinions

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 5 German Version of Betzavta-Together 1996: Adaptation initiated by the Bertelsmann- Foundation and the University of Munich (CAP) Modular combination of three different programs: ◦Betzavta (1886), ◦Peace Education Program (1995) ◦Building Blocks of Democracy (1996). Since 1997: Training of multipliers Implementation Process

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 6 The Funders: Demands and Expectations Target Group: “ The Multipliers from 1997 to 2001” Innovative Research Combination of qualitative and quantitative methods A democratic and participatory process of evaluation corresponding to the spirit of the programme Valid assessment of progress or failure Inherent Definition of Success: cognitive learning, socio-moral education, attitude change, action planning, systemic changes

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 7 Basic assumptions about evaluation of the research-team (Part I) Complexity-Learning processes are complex phenomena Validity:Control groups are limited in their testimony Reliability: Effects and effect-attribution are not identical

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 8 Our Basic assumptions -Part II Systemic View: Evaluation is an attempt to measure the outcomes and effects of an individual learning experience within a specific system Perception of effects and effect-attributions are context-related Interpretation of Goals is dependent on the occupational environment Interpretation of Effects/Outcomes dependent on the definition of progress and failure in a certain milieu

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 9 Evaluation design Database: 250 addresses Partly standardized questionnaires- 11 individual interviews Group discussion Cooperation with a practitioner’s group Interviews with experts

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 10 Main results (Part I) Programme : enthusiastic acceptance of its methodology and interactive learning strategies Effect-attributions are very positive But: positive judgment did not depend on duration of the personnel and/or professional experience Modest degree of distribution and implementation Circularity in specific milieus and institutions

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 11 Main results (Part II) Trainers and multipliers: majority from pedagogical and psychological occupations One Third complain about training quality High commitment to the issues-certain lack of professionalism Rare use of the whole Programme Preference for group-dynamic activities Modest integration into school curricula Educational practice shows tendancy towards a specific interpretation of the programme Rather depoliticized usage as programme for Social Learning;

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 12 Recommendations and conclusions Improvement of staff and organisation Theory-input Clarification of goals and objectives Monitoring and consulting Context-orientation -different trainings for different target groups Improvement of implementation- strategies

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 13 Discussion: What might be useful for us?? Our understanding of political education? Our goals and expectations concerning our work with a specific programme? Do we pay attention to needs and requirements of different target group? Long-term or short- term intervention? Follow-up? Our indicators for success or failure?

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 14 Obstacles and conflicts during the evaluation process The practitioner‘s group ◦Fluctuation of members ◦Collisions of interests ◦Problems with solidarity Negotiation processes ◦Different perspectives of Stakeholders ◦Conflict of interests

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 15 Preconditions for Success Quality of the programme Quality of the facilitators Structure of the learning process Reference to the Participants´context Integration into Curricula Learning arrangements Long-Term Intervention Continuity of the process Follow-up

Research project on Evaluation Michael Bommes and Ulrike Wolff-Jontofsohn Betzavta-Together 27 May, 2016Slide Nr. 16 Workshop in Feldafing-Overview Introduction Evaluating a Peace Education Programme:” Betzavta-Together” Evaluation Process and Main Findings Discussion Closing round