1. Carol M. Trivette, Ph.D. Orelena Hawks Puckett Institute Asheville and Morganton, North Carolina Presentation prepared for Helping Extend Learning and Practice (HELP) University of Louisville, Louisville, KY, May 29-30, 2012

2. 2

3. 3 Purposes of the Presentation Describe the key characteristics of an evidence-based approach to the implementation of professional development. Share your solutions to the challenges of providing good professional development.

4. 4 A Lesson Learned from More Than 40 Years of Experience No intervention practice, no matter its evidence base, is likely to be adopted and used if the implementation methods used to teach or train practitioners to use the practice are themselves ineffective. Therefore, concern for the characteristics of implementation practices that are associated with optimal learner and practitioner outcomes should be of paramount importance as part of implementation research.

5. 5 Child and Family Benefits

6. 6 Two Types of Evidence-Based Practices Evidence-Based Intervention Practices Early childhood intervention practices Evidence-Based Implementation Practices Adult learning methods

7. 7 Definition of Terms Implementation practices include methods and procedures used by implementation agents (trainers, coaches, instructors, supervisors, etc.) to promote interventionists use of evidence-based intervention practices. Intervention practices include methods and strategies used by intervention agents (teachers, therapists, clinicians, parents, etc.) to effect changes or produce desired outcomes in a targeted population or group of recipients (e.g., infants and toddlers with disabilities).

8. 8 Participatory adult learning Coaching Mentoring Just-in-time training Guided design Accelerated learning Early child contingency learning Interest-based child learning Natural environment practices Preschool classroom practices Communication and language learning Early literacy learning Family systems intervention practices Implementation PracticesIntervention Practices Relationship Between the Two Types of Practices

9. 9 Principles Evidence-based intervention practices are a necessarybut not sufficientcondition to ensure optimal outcomes for children and families. Practitioners must be trained on intervention strategies using evidence-based adult learning implementation practices.

10. 10 Professional Development Formats Thinking about the professional development (TA, training) that you support, what training formats/contexts are used most often? Participant Chat

11. 11

12. 12 Research Synthesis of Adult Learning Studies a Research synthesis of studies of accelerated learning, coaching, guided design, and just-in-time training 58 randomized control design studies 2,095 experimental group participants and 2,213 control or comparison group participants Combination of studies in university and non-university settings Learner outcomes included learner knowledge, skills, attitudes, and self- efficacy beliefs The influence of the adult learning methods on the learner outcomes was estimated by weighted Cohens d effect sizes for the differences on the post test scores for the intervention vs. nonintervention group participants a Dunst, C.J., Trivette, C.M., & Hamby, D.W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1), 91-112.

13. 13 Efficacy of the Adult Learning Methods Effect size for the intervention vs. nonintervention group comparisons is d =.42 (95% Confidence Interval =.36 to.47).

14. 14 Efficiency of the Four Adult Learning Methods

15. 15 Translational Findings Process for unpacking and unbundling the key characteristics of the adult learning methods

16. Six Characteristics Identified in How People Learn a Were Used To Code and Evaluate the Adult Learning Methods Planning Introduce Engage the learner in a preview of the material, knowledge or practice that is the focus of instruction or training Illustrate Demonstrate or illustrate the use or applicability of the material, knowledge or practice for the learner Application Practice Engage the learner in the use of the material, knowledge or practice Evaluate Engage the learner in a process of evaluating the consequence or outcome of the application of the material, knowledge or practice Deep Understanding Reflection Engage the learner in self-assessment of his or her acquisition of knowledge and skills as a basis for identifying next steps in the learning process Mastery Engage the learner in a process of assessing his or her experience in the context of some conceptual or practical model or framework, or some external set of standards or criteria a Donovan, M. et al. (Eds.) (1999). How people learn. Washington, DC: National Academy Press.

17. 17 Effect Sizes for the Six Adult Learning Characteristics Planning Application Understanding Average effect sizes and 95% confidence intervals for the relationships between the six adult learning method characteristics and the study outcomes.

18. 18 Most Effective Adult Learning Method Practices CharacteristicPracticeMean Effect Size IntroductionOut-of-class learner activities/self-instruction0.64 Classroom/workshop presentations0.63 Pre-class learner exercises0.54 IllustrationTrainer role playing/simulations0.55 Learner-informed input0.53 PracticingReal-life learner application0.94 Real-life learner application/role playing0.86 EvaluationSelf assessment of strengths/weaknesses0.94 ReflectionIdentify performance improvement goals1.27 Journaling/behavior suggestions0.82 MasteryStandards-based assessment0.86

19. 19 Cumulative Effects of Different Combinations of the Most Effective Adult Learning Method Practices Average Cohens d effect sizes and 95% confidence intervals for the relationship between different combinations (clusters) of adult learning methods characteristics and the study outcomes.

20. 20 PALS (Participatory Adult Learning Strategy) PLAN APPLICATION RECYCLE Active Learner Involvement Reflection and Mastery Practice and Evaluate Introduce and Illustrate Identify Next Steps in the Learning Process INFORMED UNDERSTANDING

21. 21

22. 22 PALS PhasesTrainer RolesTrainee Roles IntroductionPreview learning topicComplete pre-training preview Describe key elementsPre-class/workshop exercises Provide examplesProvide input on the learning topic Include trainee inputIn-class/workshop warm-up exercises Illustrate application Demonstrate application ApplicationFacilitate applicationProvide examples of application Observe trainee applicationTrainee role playing, games, etc. Provide in vivo feedback/guidanceImplement/practice use of the subject matter Facilitate learner assessment of optionsEvaluate use of the knowledge or practice Informed Understanding Establish learning standardsStandards-based evaluation Engage learners in self-assessmentConduct self-assessment Provide guidance to learnersTrainer-guided learner reflection Provide behavioral suggestionsJournaling Group discussions of understanding Repeat Learning Process Joint planning Trainer guidanceIdentify needed information/experiences Trainer/trainee mentoring Trainer and Trainee Roles in the Different Phases of PALS

23. 23 Practice Reflection Checklists

24. 24 Practice Reflection Checklists Specify the practice by creating indicators for each project component Help practitioners understand key characteristics of the practices Serve as the standards against which practitioners examine and improve their practices

25. 25 CELL Early Literacy Learning Model Literacy-Rich Environments Everyday Literacy Activities Responsive Teaching Early Literacy Learning Outcomes Child Interests

26. 26 Caregiver Responsive Teaching Checklist Center for Early Literacy Learning

27. 27 Child Interests Checklist Center for Early Literacy Learning

28. 28 Participatory Adult Learning Strategy (PALS) Checklist

29. 29 Principles Active participation of the learner in all phases of the training is essential. Individualize training to support learner mastery of the practice. Encourage practitioners self-reflection at every phase of the process (Specific checklists can be helpful) to increase the rate of change in practitioners practices. The more training strategies (introduce, illustrate, practice, etc.) used during training, the greater the change in practitioner practices. Checklists are effective tools for engaging practitioners in self-reflection.

30. 30 What was the most successful professional development you have ever experienced? Why was it so successful? Participant Chat

31. 31 Relationships Among the Two Types of Fidelity and Child/Parent Outcomes Implementation Fidelity Intervention Fidelity Child/Parent Outcomes

32. 32 Methods of Assessing Implementation Fidelity Implementation fidelity can be gathered through observation to determine if the trainer uses the correct training strategies. Implementation fidelity data can be collected from trainees about whether the trainer provides the appropriate training content and strategies. Implementation data ask the trainee to indicate the extent to which they feel competent with the practice content and training strategies they are supposed to use.

33. 33

34. 34 CELL Trainer Feedback Form Content Items 1.The importance of literacy-rich environments was well explained by the trainer. 2.The key characteristics of responsive teaching were described and illustrated in ways that made the instructional practice easy to understand. Training-Method Items 3.The importance of active participation of trainees as a method was made explicitly clear. 4.The training method was described in enough detail to understand its key elements. Practice Items 5.A sufficient amount of time was devoted to each component of the CELL literacy model. 6.The activities for engaging trainees in learning the CELL literacy practices were especially helpful.

35. 35

36. 36 What do trainers need to do professional development well? Participant Chat

37. 37 In order to do PD well, what do you need to do that you cannot do? Different attitudes One-to-one talk time Ability to observe practice Better technology Face-to-face tim e Jump the digital divide Flexible work schedules Less staff turn over

38. 38 Thank you for your participation!!

39. 39 References Dunst, C. J., & Trivette, C. M. (2009). Lets be PALS: An evidence-based approach to professional development. Infants and Young Children, 22, 163-175. Free download at http://dept.washington.edu/isei/iyc/22.3_Dunst.pdf http://dept.washington.edu/isei/iyc/22.3_Dunst.pdf Dunst, C. J., Trivette, C. M., & Hamby, D. W. (2010). Meta-analysis of the effectiveness of four adult learning methods and strategies. International Journal of Continuing Education and Lifelong Learning, 3(1), 91-112. Retrieved from https://w5.hkuspcae.hku.hk/journal/index.php/ijcell/article/view/111. https://w5.hkuspcae.hku.hk/journal/index.php/ijcell/article/view/111 Trivette, C. M., Dunst, C. J., Hamby, D. W., & OHerin, C.E. (2009). Characteristics and consequences of adult learning methods and strategies [Winterberry Research Syntheses, Vol. 2, No. 2]. Asheville, NC: Winterberry Press. Free download at www.wbpress.comwww.wbpress.com www.puckett.org ctrivette@puckett.org