1. Training, Evaluation and Developmental Change in Learning Systems Katharine Briar-Lawson, PhD University at Albany

2. Journey of Enormous Impacts Partnerships began in 1985,over two decades of innovations Evolved into robust, increasingly data driven training, education initiatives Learning communities formed within/ across states, universities Survived threats to cuts in IV-E, 426 and related funding streams

3. Stocktaking: New Opportunities Ten years of collaboration and research exchanges Clearinghouse functions Many fugitive studies (Barbee,2003) New Public Child Welfare Journal National Partnership Center: research repository for cross state learning

4. Challenges Over $275 million in IV-E training and $80 million in IV-E education dollars spent each year Training transfer generally about 10-15% Turnover high in many states and counties Aggregate impact of our work not synthesized on annual basis, nor collective education or training efforts across nation studied

5. Training Versus Learning Systems Lectures, traditional training models Training transfer problem Curriculum is trainer “proof” Participatory, collaborative action research and learning Collaborative team members jointly construct learning Team members adapt to context

6. Work Units as Learning Systems Work units as incubators of innovative practice models and service improvements Front line units as design teams with inventive, self correcting, data driven learning systems with consensus based helping practices Supervisors as capacity builders and case consultants rather than solely administrative managers

7. Design Teams Utah, Nevada, New Mexico, Colorado New York (high turnover counties and borough) Front line staff create logic models for retention strategies addressing root causes Design teams as learning and action systems

8. Design Teams and Change Design teams effective in retention with team effects saturating entire district office (Strollin, 2006) Training is embedded and localized, co- production of the training so ownership Design team concept congruent with teaming in CPS so new models of practice being tested

9. Next Round of Challenges Consumer perspectives, cultural competence Trauma informed training and evaluation Value, attitude assessment Competency vs. quality of treatment issues Mixed workforce issues Every training and evaluation act- linked to re-professionalization, PIP, EBP

10. Research challenges Randomization Quasi-experimental designs Evidence based knowledge building, dissemination Cost of failure, ROI analyses

11. Evidence Based Movements Disconnect between training evaluators and evidence based initiatives in SSW Expert knowledge among training evaluators on organizational barriers to “evidence based” practices Connect SSW dialogue and capacity building on evidence based practice and field placements to training evaluators

12. Traditional vs. Developmental Research (Springer & Phillips) Traditional model: Intervention controlled, stable Hypotheses do not change Interventions imposed Outcome measures standardized," given” Single research design matched to need Developmental model: Intervention evolves, changes Hypotheses change, iterative research Stakeholders are co- designers Developed in context, refined iteratively Multiplism in methods

13. Model comparison and CQI fit: Traditional model: Experimental and quasi-experimental logic RCT as gold standard Developmental model: Develop and/or seize on naturally occurring quasi-experimental conditions Theory driven evaluation models (Weiss)

14. CQI Developmental model and CQI: turning each barrier into the next goal to be addressed with successive new interventions, including training, data Journey charting for lessons learned Training (however defined) linked to PIP, workforce development, practice improvements, client and system outcomes