1. Carol M. Trivette, Ph.D. Orelena Hawks Puckett Institute Asheville and Morganton, North Carolina Presentation Prepared for the 2012 Early Childhood Inclusion Institute, University of Chapel Hill, Chapel Hill, NC, May 14 – 16, 2012

2. 2

3. 3 Participant Chat This “word cloud” highlights the words participatory and reflection. When considering your professional development work, what do you think about when you see the words participatory and reflection?

4. 4 Purposes of the Presentation  First, I will describe the key characteristics of an evidence-based approach to the implementation of professional development.  Then you can share some of your own solutions to the challenges of providing “good” professional development.

5. 5 A Lesson Learned from More Than 40 Years of Experience No intervention practice, no matter its evidence base, is likely to be adopted and used if the implementation methods used to teach or train practitioners to use the practice are themselves ineffective. Therefore, concern for the characteristics of implementation practices that are associated with optimal learner and practitioner outcomes should be of paramount importance as part of implementation research.

6. 6 Child and Family Benefits

7. 7 Two Types of Evidence-Based Practices  Evidence-Based Intervention Practices Early childhood intervention practices  Evidence-Based Implementation Practices Adult learning methods

8. 8 Definition of Terms  Implementation practices include methods and procedures used by implementation agents (trainers, coaches, instructors, supervisors, etc.) to promote interventionists’ use of evidence-based intervention practices.  Intervention practices include methods and strategies used by intervention agents (teachers, therapists, clinicians, parents, etc.) to effect changes or produce desired outcomes in a targeted population or group of recipients (e.g., infants and toddlers with disabilities).

9. 9  Participatory adult learning  Coaching  Mentoring  Just-in-time training  Guided design  Accelerated learning  Early child contingency learning  Interest-based child learning  Natural environment practices  Preschool classroom practices  Communication and language learning  Early literacy learning  Family systems intervention practices Implementation PracticesIntervention Practices Relationship Between the Two Types of Practices

10. 10 Principles  Evidence-based intervention practices are a necessary—but not sufficient—condition to ensure optimal outcomes for children and families.  Practitioners must be trained on intervention strategies using evidence-based adult learning implementation practices.

11. 11 Participant Chat Professional Development Formats Thinking about the professional development (TA, training) that you do, what training formats/contexts do you use most often?

12. 12

13. 13 Research Synthesis of Adult Learning Studies a  Research synthesis of studies of accelerated learning, coaching, guided design, and just-in-time training  58 randomized control design studies  2,095 experimental group participants and 2,213 control or comparison group participants  Combination of studies in university and non-university settings  Learner outcomes included learner knowledge, skills, attitudes, and self- efficacy beliefs  The influence of the adult learning methods on the learner outcomes was estimated by weighted Cohen’s d effect sizes for the differences on the post test scores for the intervention vs. nonintervention group participants a Dunst, C.J., Trivette, C.M., & Hamby, D.W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1), 91-112.

14. 14 Efficacy of the Adult Learning Methods Effect size for the intervention vs. nonintervention group comparisons is d =.42 (95% Confidence Interval =.36 to.47).

15. 15 Efficiency of the Four Adult Learning Methods

16. 16 Translational Findings Process for unpacking and unbundling the key characteristics of the adult learning methods

17. 17 Effect Sizes for the Six Adult Learning Characteristics Planning Application Understanding Average effect sizes and 95% confidence intervals for the relationships between the six adult learning method characteristics and the study outcomes.

18. 18 Most Effective Adult Learning Method Practices CharacteristicPracticeMean Effect Size IntroductionOut-of-class learner activities/self-instruction0.64 Classroom/workshop presentations0.63 Pre-class learner exercises0.54 IllustrationTrainer role playing/simulations0.55 Learner-informed input0.53 PracticingReal-life learner application0.94 Real-life learner application/role playing0.86 EvaluationSelf assessment of strengths/weaknesses0.94 ReflectionIdentify performance improvement goals1.27 Journaling/behavior suggestions0.82 MasteryStandards-based assessment0.86

19. 19 Cumulative Effects of Different Combinations of the Most Effective Adult Learning Method Practices Average Cohen’s d effect sizes and 95% confidence intervals for the relationship between different combinations (clusters) of adult learning methods characteristics and the study outcomes.

20. 20 PALS (Participatory Adult Learning Strategy) PLAN APPLICATION RECYCLE Active Learner Involvement Reflection and Mastery Practice and Evaluate Introduce and Illustrate Identify Next Steps in the Learning Process INFORMED UNDERSTANDING

21. 21

22. 22 Adult Learning Method Practices

23. 23 PALS PhasesTrainer RolesTrainee Roles IntroductionPreview learning topicComplete pre-training preview Describe key elementsPre-class/workshop exercises Provide examplesProvide input on the learning topic Include trainee inputIn-class/workshop warm-up exercises Illustrate application Demonstrate application ApplicationFacilitate applicationProvide examples of application Observe trainee applicationTrainee role playing, games, etc. Provide in vivo feedback/guidanceImplement/practice use of the subject matter Facilitate learner assessment of optionsEvaluate use of the knowledge or practice Informed Understanding Establish learning standardsStandards-based evaluation Engage learners in self-assessmentConduct self-assessment Provide guidance to learnersTrainer-guided learner reflection Provide behavioral suggestionsJournaling Group discussions of understanding Repeat Learning Process Joint planning Trainer guidanceIdentify needed information/experiences Trainer/trainee mentoring Trainer and Trainee Roles in the Different Phases of PALS

24. 24 Practice Reflection Checklists  Specify the practice by creating indicators for each project component  Help practitioners understand key characteristics of the practices  Serve as the standards against which practitioners examine and improve their practices

25. 25 CELL Early Literacy Learning Model Literacy-Rich Environments Everyday Literacy Activities Responsive Teaching Early Literacy Learning Outcomes Child Interests

26. 26 Practice Reflection Checklists

27. 27 Caregiver Responsive Teaching Checklist Center for Early Literacy Learning

28. 28 Child Interests Checklist Center for Early Literacy Learning

29. 29 Participatory Adult Learning Strategy (PALS) Checklist

30. 30 Principles  Active participation of the learner in all phases of the training is essential.  Individualize training to support learner mastery of the practice.  Encourage practitioners’ self-reflection at every phase of the process (Specific checklists can be helpful) to increase the rate of change in practitioners’ practices.  The more training strategies (introduce, illustrate, practice, etc.) used during training, the greater the change in practitioner practices.  Checklists are effective tools for engaging practitioners in self-reflection.

31. 31 Relationships Among the Two Types of Fidelity and Child/Parent Outcomes Implementation Fidelity Intervention Fidelity Child/Parent Outcomes

32. 32 Methods of Assessing Implementation Fidelity  Implementation fidelity can be gathered through observation to determine if the trainer uses the correct training strategies.  Implementation fidelity data can be collected from trainees about whether the trainer provides the appropriate training content and strategies.  Implementation data ask the trainee to indicate the extent to which they feel competent with the practice content and training strategies they are supposed to use.

33. 33

34. 34 CELL Trainer Feedback Form Content Items 1.The importance of literacy-rich environments was well explained by the trainer. 2.The key characteristics of responsive teaching were described and illustrated in ways that made the instructional practice easy to understand. Training-Method Items 3.The importance of active participation of trainees as a method was made explicitly clear. 4.The training method was described in enough detail to understand its key elements. Practice Items 5.A sufficient amount of time was devoted to each component of the CELL literacy model. 6.The activities for engaging trainees in learning the CELL literacy practices were especially helpful.

35. 35

36. 36 What was the most fun you have ever had doing professional development? Why?

37. 37 What was the most successful professional development you have ever done? How did you know it was successful?

38. 38 In order to do PD well, what do you need to do that you cannot do? Different attitudes One-to-one talk time Ability to observe practice Better technology Face-to-face tim e Jump the digital divide Flexible work schedules Less staff turn over

39. 39 What else is hard to do well? What suggestions do others have to help?

40. 40 Thank you for your participation!

41. 41 References Dunst, C. J., & Trivette, C. M. (2009). Let’s be PALS: An evidence-based approach to professional development. Infants and Young Children, 22, 163-175. Free download at http://dept.washington.edu/isei/iyc/22.3_Dunst.pdf http://dept.washington.edu/isei/iyc/22.3_Dunst.pdf Dunst, C. J., Trivette, C. M., & Hamby, D. W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1), 91-112. Retrieved from https://w5.hkuspcae.hku.hk/journal/index.php/ijcell/article/view/111. https://w5.hkuspcae.hku.hk/journal/index.php/ijcell/article/view/111 Trivette, C. M., Dunst, C. J., Hamby, D. W., & O’Herin, C.E. (2009). Characteristics and consequences of adult learning methods and strategies [Winterberry Research Syntheses, Vol. 2, No. 2]. Asheville, NC: Winterberry Press. Free download at www.wbpress.comwww.wbpress.com www.puckett.org

42. 42