Evaluating Impact of Training related to Implementation of Early ACCESS in Iowa Presenter: Janet Melby Co-authors: Chris Rubino, Neil Rowe, Jiong Yang Iowa State University National Human Services Training Evaluation Symposium Pittsburgh, PA May 23, 2013

About the Training Training topic: Power of Teaming: Department of Human Services and Early ACCESS, Allies for Infants and Toddlers Training objectives: 1. Understand impact of trauma on infants and toddlers 2. Identify how federal law supports child well-being 3. Understand purpose of CAPTA, ASFA, and IDEA law 4. Determine action steps between local DHS and EA providers to facilitate improved collaboration. Delivery: Nine, in-person, one-day, around Iowa, February-April, Process: Content delivery, plus DHS/EA discussion and action steps Evaluation: (a) Post-training paper survey-13 items, (b) action steps, (c) on-line training feedback

Evaluation Focus  Effect of training on improvement in trainee:  Knowledge  Approach to work with children  Collaboration  Link between training effects and trainee description:  DHS or Early ACCESS  Role length  Age, gender, race  Current collaboration status

Sioux City Council Bluffs Fort Dodge 31 Waterloo Cedar Rapids Ottumwa Burlington Des Moines 15 Davenport DHS Early ACCESS Community Provider FSRP (Family, Risk, Safety & Permanency) Other About the Trainees Registered for training: N = 212 Paper post-training survey: N = 187 DHS = 61, Early ACCESS = 112, Other = 14 Non-Caucasian = 3 Males = 5 On-line follow-up: N = 132

Impact Work α=.615 Impact Knowledge α=.747 Impact Collaboration α=.625 Post-training Survey Items and Domains

Survey Numeric Results Overall: EA higher than DHS on impact of training on Future Work with young children and on Collaboration (p <.01). Within Knowledge, improvements in CAPTA, child trauma, and IDEA. Current collaborators: EA and DHS, both higher on Future vs. Current Collaboration; EA sig. higher than DHS on both Future and Current Collaboration (p <.01)

Action-Step Coding Results Participants grouped by local areas (counties):  Discussed local issues and protocols  Developed action steps with the goal of enhancing collaboration  Action steps synthesized by trainer and coded by researcher Main themes identified:  Meetings “Invite DHS/EA to FTM and IFSP meeting and attend if at all possible.” “Quarterly staffing for DHS, EA, and FSRP agencies (face to face or conference calls).”  Communication “Better communication through .” “Get more information from DHS worker on family goals in case plan to possibly use as family goal for IFSP.” “Make sure EA has family interaction schedule and FSRP worker contact.”

On-line Training Feedback Results Trainee item responses (% agree or strongly agree; N = 132):  Training material informative and relevant (92%)  Communication methods effective (92%)  Appropriate pace (94%)  Trainer knowledgeable on topic (94%)  Gained better understanding (87%) Open-ended comments:  What did you like best about training?

Challenges  Training development:  Matching DHS and EA participants for groups by geographic location and balancing the groups by individual roles within each agency  Developing a method to provide information about each agency’s work to the other agency in an engaging way  Getting people to sign up for the training  Evaluation design:  Post-training survey  Developing survey items  Revising survey form  Inability to distinguish among DHS roles  Obtaining on-line evaluation data  Inability to link paper survey to on-line evaluation

Conclusions and Implications  Early ACCESS “Power of Teaming” training is effective in impacting the knowledge, future work with young children, and future collaboration of participants.  Current work role of trainees (specifically DHS or Early ACCESS) is a major factor in the effect of the training in these three domains, and age is also an important factor regarding the impact of training on future work with young children.  Adding training-specific evaluation tools enhances assessment of training impact.  Being intentional in designing evaluation approaches has potential benefits.