1. Professional Coach Training Evaluation Research Beyond Participants’ Self-Reports Arthur Drexler University of Innsbruck Department of Communication in the Professional Field and Psychotherapy Schöpfstrasse 3 A-6020 Innsbruck Heidi Möller University of Kassel Department 04 Arnold-Bode-Straße 10 D-34109 Kassel

2. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Overview Background The Research Focus Characteristics of a Coach Evaluation Design and Instruments Results Outlook „How can hard facts for soft skills be collected?“

3. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Background: Lack of Evaluation Tools No tradition of evaluating coach trainings No valid and practicable specific tools for evaluation and quality assurance No benchmarks available

4. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Background: The Coach Programme 9 weekend seminars (=190 training units) Duration of 16 months Each seminar focused on specific topic conflicts communication skills group dynamics etc.

5. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 The Research Focus The research addresses three main questions:  What theoretical knowledge, skills and personality traits does a coach need to successfully meet the clients’ needs?  Do the participants have the knowledge and skills at their disposal after coach training?  What instruments can make these factors visible?

6. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Characteristics of a Coach  Motivation  Personality factors  Concept competence  Key qualifications

7. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Sample Description Sample size: 17 Gender: 6 men, 11 women Mean age: 44 years (s = 6.6) Profession: 23.5% work in a social profession, 76.5% in other professions (e.g. business consulting, HR, medicine)

8. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Evaluation Design and Instruments Pre-measurement at the beginning of the training Questionnaires Case Study Affect Recognition Tests Business Knowledge Test Sculpting Continuous measurements after each seminar Self and trainers’ assessments of learning progress through questionnaires Post- measurement Questionnaires Case Study Affect Recognition Tests Business Knowledge Test

9. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Initial Motivation Results Goal of their own personal development through coach training Desire for a change to improve their professional lives – not necessarily as coaches Motivation to help clients (and themselves) reach professional goals

10. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Results Personality FPI-ScalesTdfSig. Satisfaction/Contentedness,89113,389 Social Orientation-6,20413,000* Performance Orientation-5,26413,000* Inhibition2,10413,055 Excitability-,90013,385 Aggression-,30613,765 Pressure1,23613,238 Physical Complaints10,26213,000* Concerns about Health-1,00013,336 Openness-2,91813,012* Extraversion-4,95413,000* Emotionality1,12713,280

11. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Results Affect Recognition No significant changes for anxiety, joy, surprise, loathing, mourning and anger

12. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Business Knowledge Overall-changes are not significant (p = 0.119) Individual changes (blue bars: pre / red bars: post-measurement)

13. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Results Case Studies Case analyses improved –Considered different levels (relationship, competence, emotional) –Considered psychodynamics –Considered group dynamics in the organisation Hypotheses improved –More hypotheses –More complex hypotheses Problem-solving skills improved –Beyond mere description –Concrete interventions

14. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Self-Assessments Self-assessments of improvement, after the conflict management seminar

15. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Self-Assessments Self-assessments of improvement after completion of the programme

16. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Summary  Motivation to attend dominated by expectations of personal and professional development  Social orientation, performance orientation, extraversion and openness increase  Tendencies of improvement in recognising emotions  Tendencies of improved business knowledge  Better development of hypotheses based on interpretive frameworks and improved problem-solving skills  Seminar-related improvements correlate mostly with the seminar topic  Strong improvements of analytical skills and self-reflection

17. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Summary Personal and professional development More Social Orientation More Performance Orientation Extraversion Openness + Motivation Coaching skills

18. University of Kassel Department 04 University of Innsbruck Department of Communication in the Professional Field and Psychotherapy ICTP/ St. Petersburg 2008 Outlook  The multi-methodological approach to evaluate coach training can be prototypic for adult education and training evaluation in general.  Results allow the comparison of the development of competencies in different education and training contexts.  Participants and course designers can benefit from the feedback provided by the evaluation process.  “Hard facts” can be collected for quality assurance and quality development in training.