1. What Works ? What Works? A Webinar Series on Effective Early Childhood Mental Health Consultation Presented by the Georgetown University Center for Child and Human Development with sponsorship from The A.L. Mailman Family Foundation Moderated by Kathy Hepburn, GUCCHD Technical support by Kevin Enright, GUCCHD

2. What Works ? Use the Chat (Q&A) pod to ask a question or respond to a general question to the audience.

3. What Works ? Logistics Close all file sharing applications and streaming music and video Send questions during the webinar that will be held for Q&A periods After the webinar: –Evaluation –Archives, support materials, and unanswered questions at : http://gucchd.georgetown.edu/90642.html http://gucchd.georgetown.edu/90642.html

4. What Works ? Webinar Series Webinar Dates in 2010 An Introduction to Effective Early Childhood Mental Health ConsultationApril 28 Preparing and Supporting the Work Force, Part 1May 26 Preparing the Supporting the Work Force, Part 2June 30 High Quality ServicesJuly 28 Relationships and ReadinessSept. 29 Evaluating Early Childhood Mental Health ConsultationOct. 27 Building Capacity, Part 1Nov. 17 Building Capacity, Part 2Dec. 16 For brief descriptions of each webinar, go to http://gucchd.georgetown.edu/90642.html http://gucchd.georgetown.edu/90642.html

5. What Works ? Webinar #6 Evaluating Early Childhood Mental Health Consultation Presenters: Deborah Perry Georgetown University Center for Child and Human Development Rosalind Kirk Michigan State University Walter Gilliam Yale University October 27, 2010

6. What Works ? Today’s Webinar Revisit key study findings and essential features of high quality ECMH consultation services Recognize the role of research and evaluation in effective early childhood mental health consultation Focus on: –Key features and challenges of effective evaluation –Three state-wide efforts to evaluate early childhood mental health consultation (MD, MI, CT) –Strategies, tools and resources to support effective evaluation and research

7. What Works ? Evaluation Helps answer critical questions about your program: –What is your program doing? –What effects are you having on whom? –What parts work well? –What parts need improvement? Continuous Quality Improvement (CQI) Helps promote sustainability and expansion

8. What Works ? An Emerging Model for ECMHC

9. What Works ? Evaluation Efforts in Study Sites All selected sites had positive outcomes All programs have rigorous evaluations underway Mixed methods being used: quantitative and qualitative Variety of formative and summative outcomes Comprehensive table of tools (p. 91-93)

10. What Works ? Positive Outcomes Child Outcomes –Decreased expulsion rates –Increased number of children in placements that meet their needs –Decreased problematic behavior –Increased pro-social behavior –Improved school readiness

11. What Works ? Positive Outcomes Family and Early Care and Education Provider Outcomes –Implementation of techniques and strategies –Enhanced sense of self- efficacy –Increased knowledge –Reduced stress –Improved interaction with child –Strong provider, parent, consultant collaboration –Satisfaction with services –Positive attitude toward mental health services –Application of strategies to other children –Increased ability to identify and address social/emotional concerns early

12. What Works ? Positive Outcomes Early Care and Education Program Outcomes –Increased quality of early childhood settings –Decreased job turnover –Improved linkages with community services Other Considerations –Cost efficiency of the program –Fidelity to the model –The “dosage” of services needed for positive outcomes –The longitudinal impact of consultation services –The capacity of the service system to serve infants /young children needs

13. What Works ? Guidance for Continuous Quality Improvement Program/services status Service /training delivery and fidelity Program and provider(s) accountability/performance outcomes Supervision Technical assistance support Collaborative activities and engaging partners Program /services planning and management

14. What Works ? Support for Sustainability and Expansion Program/services –Accountability –Outcomes –Fidelity –Expansion –Replication Leveraging funds and resources Influencing policy

15. What Works ? Expanding the Evidence Base Two reviews of the best available evidence –Staff & Program Level Outcomes –Brennan, E. M., Bradley, J. R., Allen, M. D., & Perry, D. F. (2008). –Child Behavioral Outcomes –Perry, D.F., Allen, M.D., Brennan, E.M., & Bradley, J.R. (in press) Increasing rigor in state level evaluations underway Use of valid and reliable tools Well articulated theory of change

16. What Works ? Question Review and Response

17. What Works ? Research Syntheses Brennan, E. M., Bradley, J. R., Allen, M. D., & Perry, D. F. (2008). The evidence base for mental health consultation in early childhood settings: Research synthesis addressing staff and program outcomes. Early Education and Development. Duran, F.B.; Hepburn, K.S.; Kaufman, R.K.; Le, L.T.; Allen, M.D.; Brennan, E.M.; & Green, B.L. Research synthesis: Early childhood mental health consultation. Available at Center for Social Emotional Foundations of Early Learning (CSEFEL) website http://www.vanderbilt.edu/csefel/pdf/rs_ecmhc.pdf http://www.vanderbilt.edu/csefel/pdf/rs_ecmhc.pdf Perry, D.F., Allen, M.D., Brennan, E.M., & Bradley, J.R. (in press). The evidence base for mental health consultation in early childhood settings: Research synthesis addressing children’s behavioral outcomes. Early Education and Development

18. What Works ? Resources Crafting Logic Models for Systems of Care: Ideas Into Action http://cfs.fmhi.usf.edu/publications/detail.cfm?id=32 Evaluating Early Childhood Programs: Improving Quality and Informing Policy http://www.zerotothree.org/site/DocServer/vol23-6a.pdf?docID=2470 Families in the World of Evaluation http://www.rtc.pdx.edu/PDF/pbP8Course1Report.pdf http:// www.ffcmh.org/what-we-do/training/world-of-evaluation-parts-i-ii-and- iii/ Participatory Action Research Involving Families from Underserved Communities and Researchers: Respecting Cultural and Linguistic Diversity http://www.beachcenter.org/research/FullArticles/PDF/PAR3PARInvolvingFa milies.pdf http://www.beachcenter.org/research/FullArticles/PDF/PAR3PARInvolvingFa milies.pdf http://www.beachcenter.org/families/partnerships/participatory_action_resear ch.aspx?JScript=1

19. What Works ? Resources Assessing Cost and Benefits of Early Childhood Intervention Programs http://www.rand.org/pubs/monograph_reports/MR1336/ What Works? study & Resource Compendium http://gucchd.georgetown.edu/78358.html http://gucchd.georgetown.edu/78358.html http://gucchd.georgetown.edu/products/78366.html Logic Models Measurement Tools & Data Codebooks Indicators/Benchmarks Cost benefit analysis Early Childhood Mental Health Consultation: An Evaluation Toolkit (2007) http://gucchd.georgetown.edu/72407.html

20. What Works ? For More Information Deborah Perry –dfp2@georgetown.edudfp2@georgetown.edu Rosalind Kirk John Carlson –roskirk@comcast.net carlsoj@msu.eduroskirk@comcast.netcarlsoj@msu.edu Walter Gilliam –walter.gilliam@yale.eduwalter.gilliam@yale.edu

21. What Works ? Join us for next webinar…. Building Capacity, Part 1: Program Infrastructure Wednesday, November 17, 2010 2-3:30 PM EST, 11 AM-12:30 PM PT Presenters: Beth Green Center for the Improvement of Child and Family Services Portland State University Al Zachik and Rolf Grafwallner Maryland Department of Health & Mental Hygiene and Maryland State Department of Education

22. What Works ? After the Webinar Evaluation Unaddressed questions/archives on webpage For any webinar connection/participation issues, contact Kathy Hepburn, ksh@georgetown.edu Thank You