1. The National Human Services Training Evaluation Symposium: - A 10 Year Retrospective -

2. 10 Yrs of Picturesque Settings The Faculty Club

3. 10 Years of Enlightening Presentations

4. Looking back and moving forward……..

5. 1998: The First In the beginning

6. 1999: The 2 nd Annual Keynote: Bernadette McCarthy—Using Training Evaluation to Improve Services Program Examined: Training outcomes: performance monitoring Delphi Techniques Nominal Group Techniques Knowledge testing Promoting & evaluating skill development Using training to influence policy Levels of competence TOL

7. 2000: The 3 rd Annual Kirkpatrick’s model, but what about Level Four? Keynote: Icebreaker on the first evening instead Program examined: Evaluation of Interdisciplinary training: Child Welfare workers and supervisors Strategic training evaluation Embedded evaluation TOL/measuring behavioral change Skills assessment

8. 2000

9. 2001: The 4 th Annual The year of “Affinity Action Groups” Keynote: Dave Wegenast on “Return on Investment” of the Evaluation Symposium Program examined: Assessing underlying competencies Models for evaluating core curriculum Evaluation of statewide training program The logic model Integration of TOL Evaluating group behavior Blackboard—E Learning

10. 2001

11. 2002: The 5 th Annual Outcomes in Child Welfare Keynote: Mark Courtney–Evaluation in Child Welfare Services: Big Questions Program examined: Training Evaluator Competency SPrA CPS system Compliance with practices standards Needs assessments

12. 2002

13. 2003: The 6 th Annual Measuring outcomes, the long and winding road… Keynote: Elena Cohen—Getting the whole picture: The CFSR, Training and Frontline Practice Links Program Examined: Chain of evidence Panel presentation on each of Kirkpatrick’s levels Project briefings

14. 2003

15. 2004: The 7 th Annual Improving Evaluation Capabilities Keynote: Wayne Stevenson—Training as an Avenue to Support Practice Improvement Program Examined: National Frameworks: National Evaluation of Child Welfare Training & North American Certification Project TOL Staff Retention Organizational Culture

16. 2004

17. 2005: The 8 th Annual Expanding our horizons: Enhancing current methods, creating new methods… Keynote: Eileen Gambrill Program Examined: Future trends in child welfare training Addressing fairness & equity in training evaluation Training evaluation in large systems Linking training outcomes to children and families Instruments to measure skill acquisition Practice assessments in a large system

18. 2005

19. 2006 – The International House

20. 2006: The 9 th Annual Evidence-based training: What does it mean when we say training is effective? Keynote: Kurt Kraiger – Alternative Frameworks for Training Evaluation Program Examined: How evaluation brings attention to gaps in training Teaching & training methods that contribute to knowledge retention Logic models Measuring knowledge/skill Evaluating assumptions, values and beliefs

21. 2006

22. 2007: The Berkeley City Club

23. 2007: 10 th Annual Looking back and moving forward… Keynote: Peter Watson—Training Evaluation as a tool for organizational development Program Examines: Findings from the National Evaluation of Child Welfare grants Measuring TOL Revisiting Level 1 The future of training evaluation Using tests to predict job performance Evaluation of training content

24. Lessons Learned… (NSDTA Journal Article) 1. You can teach a turkey to climb a tree, but it's easier to hire a squirrel. 2. Effective utilization of training is influenced by individual, organizational, and training design factors. 3. Training evaluation can help administrators differentiate between training and non- training solutions.

25. Lessons Learned, cont’d 4. We know what we don’t know. 5. Sometimes the training evaluation “tail” wags the training content “dog”. 6. Be careful what you wish for, the focus on outcomes cuts both ways.

26. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

27. Increased funding, value, and visibility of training and development

28. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT Increased funding, value, and visibility of training and development Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career

29. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT Increased funding, value, and visibility of training and development Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career Increased accountability of child welfare/human services

30. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT Increased funding, value, and visibility of training and development Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career Increased accountability of child welfare/human services Introduction of litigation into the HSTD arena

31. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level

32. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level Technology – An increase in distance models and alternative delivery systems

33. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level Technology – An increase in distance models and alternative delivery systems Agency-University partnerships

34. THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level Technology – An increase in distance models and alternative delivery systems Agency-University Partnerships Development of the field and emerging profession of HSTD (e.g., Code of Ethics, roles and competencies, journal, landmark events 10 years NHSTES & 20 years NSDTA conference)