1. Maryland Early Childhood Mental Health Consultation (ECMHC) Evaluation
Webinar Presentation
by Deborah F. Perry Ph.D.
Georgetown University
October 27, 2010

2. Overview of Presentation
Overview of statewide ECMHC evaluation in Maryland
Maryland’s approach
Logic model and components
Levels of data collection and other features
Future directions

3. Maryland ECMHC Evaluation
Funding:
Maryland State Department of Education (MSDE), Division of Early Childhood
Involves 12 sites across Maryland
Three year evaluation ( ), with two option years
Principal Investigators:
Sharon Stephan: University of Maryland
Bruno Anthony, PhD & Deborah Perry, PhD: Georgetown University
Minority Business partner: CKD Communications, LLC

4. Early Childhood Mental Health Consultation (ECMHC)
Collaborative, capacity building services provided to early care and education (ECE) providers and family members
Can be initiated in response to concerns about an individual child’s social-emotional or behavioral development
The impact can be measured in the domains of child, parent, staff, and family 4

5. Approach to Evaluation
Collaborative
Participatory
Comprehensive
Sensitive to respondent burden
No site comparisons
Technical assistance related to evaluation and data collection provided

6. Funded Projects and Evaluation Liaisons

7. Evaluation Logic Model
Skills and Qualifications
Satisfaction
Relationship
Early Childhood Mental Health Consultation
Early Care and Education
- Attitudes, Beliefs, and Stress
- Classroom Environment
Parenting
- Behavior and Attitudes
- Stress
Young Children
↓Challenging Behavior
↑Social Skills
Expulsions

8. Evaluation Components
ECE Providers
SDQ
TOS
RQS
DECA
Parents
PSI
PBI
Consultants
Climate
ECE Directors
GAS
Implementation Factors
Core Outcomes
Program Directors
Model
Description,
Quarterly
Interview
Knowledge
& Skills,
Service Log,
Exiting Child Form,
Roster, Preschool
Climate Scale
Exiting Child Study
Consultants,
ECE Directors,
ECE Providers,
and Parents
Qualitative
Interviews
Note: instruments in pink will only be collected for child-specific referrals

9. Levels of Data Collection
Program-level outcomes
Measures assess changes in: ECE provider behaviors and beliefs; job stress; structural and interpersonal characteristics of the ECE setting; and the quality of the relationship between the MHC and ECE providers.
Child-/Family level outcomes
For child-specific referrals:
Measures assess changes in the child’s behavior (rated by teacher and parent), parenting stress and behaviors; and the quality of the relationship between the MHC and ECE providers.

10. ECMHC Program Directors
Tools
Construct
Frequency
Model Description
Program Structure
Baseline, Annual
Quarterly Site Interviews
Program Changes
Quarterly

11. Mental Health Consultants
Tools
Construct
Frequency
Knowledge and Skills Inventory
Consultant Characteristics
Baseline, Annual
Service Log
ECMHC Services
Ongoing
Classroom Roster
Administrative Tool
Preschool/Infant MH Climate Scale
Classroom Environment
Baseline, every 4 months (ongoing)
Consultant Final Perceptions Form
Quality of Relationship
4 months, or at Discharge
Exit Study (Qualitative Interviews)
Perceptions and predictors of expulsion
Following an exit due to behavior issues

12. Preschool/Infant Climate Scale
These tools are completed by the mental health consultants for every child-specific and programmatic consultation they initiate.
Preschool Mental Health Climate Scale (Gilliam, undated)
Infant Climate Scale (FSU, 2009)
The instruments assess multiple dimensions of the classroom environment and ECE providers’ behaviors.
Consultants complete these scales at baseline, at 4 months or at discharge; and if ongoing, every 4 months thereafter.

13. Child Care Directors Tools Construct Frequency Goal Achievement Scale
Opinions and Attitudes
Baseline, 4 months or at discharge
Exit Study Interviews
Perceptions of Expulsion
When child exits due to behavior

14. Early Care and Education Providers
Tools
Construct
Frequency
Strengths and Difficulties Questionnaire
Child Functioning
Baseline, 4 months or at discharge
Teacher Opinion Survey
Teacher Attitudes
Provider Job Stress Index
Job Stress
Teacher Final Perceptions Form
Quality of Relationship
Exit Study Interviews
Perceptions of Expulsions
When child exits due to behavior

15. Parents * For those who consent to long-term follow along Tools
Construct
Frequency
Parenting Stress Index-Short Form
Parent stress
Baseline, 4 months or at discharge
Parent Behavior Inventory
Parenting practices
Parent Final Perceptions Form
Quality of relationship
Devereux Early Childhood Assessment
Child functioning
Baseline, 4 months, 12* & 24* months
Exit Study Interview
Perceptions of expulsions
If their child is expelled
* For those who consent to long-term follow along

16. Exit Study Interviews
Motivated by MSDE’s interest in this policy-relevant (but infrequent) outcome
Always interview the consultant; seek the additional perspectives of ECE Director, provider and parent
Includes open-ended questions such as:
What behaviors led to the child’s exit?
What was done to try to maintain the child in the placement?
Did you expect this child to be expelled? Why or why not?

17. Professional Activity Units
Working with MSDE, established process for ECE Directors and Providers to get professional activity units when complete baseline and follow-up measures for evaluation
Increase participation and help with follow-up data

18. Current Status and Future Directions
Data collection in the field: good participation from all sites
Exit interviews completed with consultants, and a few ECE providers
Comparison group
Addition of CLASS as objective measure of change
Paper on Preschool Mental Health Climate Scale with Walter Gilliam