1. MIECHV Benchmark Data Collection and Evaluation
Mary Anne Wilson, M.S.W.
Center for Prevention Research and Development
Institute of Government and Public Affairs
University of Illinois

2. Overview What are the benchmarks and why are they necessary?
Federal requirements
Evaluation and data collection
Performance measures
Continuous quality improvement (CQI)

3. Purpose of Presentation
Increase understanding for role of Benchmarks, Benchmark Reporting Requirements
Understand the reporting and data collection requirements for the MIECHV programs
Identify the roles that each organization involved in Benchmark reporting will play, and what additional roles that CPRD can play for MIECHV and home visiting services
Identify key performance measures for MIECHV programs, both for Benchmarks and continuous quality improvement efforts
Understand the role of the field data collectors

4. And what is not the purpose of this presentation?

5. Maternal, Infant, Early Childhood Home Visiting
What is MIECHV?
Maternal, Infant, Early Childhood Home Visiting
Over time, families and home visitors build partnerships and work together to:
Improve health and development.
Prevent child injuries, child abuse, neglect, or maltreatment, and reduce emergency department visits.
Improve school readiness and achievement.
Reduce crime, including domestic violence.
Improve family economic self-sufficiency.
Improve the coordination and referrals for other community resources and supports.

6. How do we define success?
Providing high-quality home visiting services that positively impact the lives of children and families enrolled in the MIECHV programs
Meeting and exceeding Benchmarks
Findings ways to pool our knowledge collectively to improve statewide delivery of home visiting services.

7. How does MIECHV fit in?
MIECHV is designed to expand HV services into priority populations:
Reside in communities in need of services & have low income
Include pregnant women who have not attained the age of 21
Have a history of child abuse
Have a history of substance abuse
Have users of tobacco products
Have a history of, or have children with, low student achievement
Have children with developmental delays or disabilities
Include members of the military

8. MIECHV Priority Communities
Rockford- Winnebago County
65.1% White, 20.5% African American, 15.8% Hispanic
Vermilion County
83.6% White, 13.3% African American, 4.4% Hispanic
Chicago- Cook County
45% White, 32.9% African American, 28.9% Hispanic
Elgin City ( border of Cook and Kane Counties)
69.5% White, 7.4% African American, 43.6% Hispanic
Cicero Town- Chicago
51.9% White, 3.8% African American, 86.6% Hispanic
Macon County
79.8% White, 16.6% African American, 2.0% Hispanic
Doula Sites: N. Lawndale, E. Garfield Park, Vermilion County, Rock Island County and Waukegan

9. The program goals, per the HRSA website:
Strengthen and improve the programs and activities
Improve coordination of services for at-risk communities
Identify and provide comprehensive services to improve outcomes for families who reside in at-risk communities

10. MIECHV Benchmarks There are a total of 6 Benchmark areas
These reflect overall domains in which we expect home visiting services to have an impact.
Each benchmark has a set of constructs, and these total 37 constructs across the 6 benchmarks
Benchmarks were developed in the Legislation, and the constructs were developed by HRSA to address the benchmarks.

11. Why do we have Benchmarks?
The legislation funding MIECHV required quantifiable, measurable improvements for the populations receiving services. Programs must demonstrate improvement in the following Benchmark areas:
Improved maternal and newborn health
Prevention of child injuries, child abuse, neglect, or maltreatment, and reduction of emergency department visits
Improvement in school readiness and achievement
Reduction in crime or domestic violence
Improvements in family economic self-sufficiency
Improvements in the coordination and referrals for other community resources and supports

12. Where do the Benchmarks come from?

13. Now, let’s look at the Benchmarks individually
The Benchmarks contain constructs, specific measures to address each concept
Reflect outcomes of home visiting services

14. Improved Maternal and Newborn Health
Constructs:
Prenatal Care
Prenatal use of alcohol, tobacco, or illicit substances
Postpartum use of contraception and Interpartum Interval
Screen for Maternal Depressive Symptoms
Duration of breastfeeding, well-child visits, and maternal health insurance coverage

15. 2. Prevention of Child Injuries, Child Abuse or Maltreatment and Reduction of Emergency Department Visits
Constructs:
Visits for child and mother to the emergency department from all causes
Information provided or training of participants on prevention of child injuries
Incidence of child injuries requiring medical treatment
Reports of suspected maltreatment for children in the program, reported substantiated maltreatment, and first time victims of maltreatment

16. 3. Improvements in School Readiness and Achievement
Constructs:
Parent support and knowledge for children’s learning and development
Parenting behaviors and parent-child relationship
Parent emotional well-being or parenting stress
Child’s communication, language and emergent literacy, cognitive skills
Child’s positive approaches to learning and attention
Child’s social behavior, emotional regulation and emotional well-being
Child’s physical health and development

17. 4. Domestic Violence Constructs: Screening for domestic violence
Of families identified for the presence of domestic violence, number of referrals made to relevant domestic violence services

18. 5. Family Economic Self-Sufficiency
Constructs:
Household income and benefits
Employment or education of adult (mother and father) members of the household
Health Insurance Status

19. 6. Coordination and Referrals for Other Community Resources and Supports
Constructs:
Number of families identified and referred to available community resources
The number of families who complete referrals to available community resources
Number of agencies with which the home visiting provider has a clear point of contact in the collaborating community agency
Number of agencies with which the home visiting provider has established a formal memorandum of understanding

20. What the Evaluation is and is not
A way to measure and address the challenges in scale-up implementation of HV services
A way to show Congress that tax dollars spent on additional HV services is important and useful
A way to introduce statewide continuous quality improvement efforts
A goal setting mechanism
The Evaluation is not:
A performance assessment to be used for funding decisions
A method to weed out the poor performing HV sites
A way to collapse the variety of model HV programs into a single government-managed HV organization

21. Illinois MIECHV: A Partnership Approach
CPRD is involved in the following areas:
In partnership with the state, developing measurement procedures and systems which adequately address the Benchmark reports
Coordinating Benchmark data collection with state and sites
Work together with home visitors to gather benchmark data
Query data from MIECHV data system and prepare reporting to the feds annually
Utilize data to track individuals through time in services
Collect home visiting services satisfaction information
Help clarify possible barriers to engagement
Lead Home Visiting Continuous Quality Improvement efforts in MIECHV and throughout the Home Visiting State System

22. What if we don’t make improvements on the benchmarks after 3 years?
Would need to develop a plan of action
CPRD, the Governor’s Office, and individual community programs would work together to develop a plan and next steps
The CQI process is crucial to next steps

23. What is CQI and how will CPRD aid in the CQI efforts?
I can tell you with 95% confidence that there is less than a 65.6% possibility that the CQI will simply generate 34.8% more meaningless statistics.

24. What are the cohorts mentioned in the Benchmarks?
The term “cohort” refers to the idea of a group of families who enter services together.
But families are entering and exiting HV services all the time
Cohorts in MIECHV follow one group of people and compare them to another group enrolled a year later
Cohorts help us measure program change and effectiveness over time
How will we know if individuals are changing?
Individual change is reviewed in CQI process

25. Who are Field Data Collectors?
What does it mean to be a field data collector?
How will we all stay in contact?
How will we ensure regular communication and updates?
Working with the home visiting sites and home visitor
Will use a shared calendar through Google Calendar (E.g. , call and text.

26. The Role of Field Data Collectors in the Benchmarks
Some Measures require rigorous approach for reliability
The Benchmarks created a large and taxing load on home visitors
Some benchmarks require that the parent not be hindered by watching their child, such as needing to fill out a longer assessment

27. On-site Data Collection
Field data collectors will work with home visitors to collect measures from families
Assess the families at three time periods
2-4 weeks after enrollment
1 year and 2 year follow-ups
Informed Consent – depending on age or emancipation status
Parental consent – if mother is less than 18 years old, written parental/guardian consent is required and self-assent
Self-consent – emancipated or 18 years or older, written assent is required
Gift cards are provided to participants O

28. What is the PICCOLO?
Parenting Interactions with Children: Checklist of Observations Linked to Outcomes
PICCOLO items describe positive parenting
PICCOLO helps practitioners identify parenting strengths to encourage parents to do more

29. What does PICCOLO measure?
From the research literature, there are identified
four important kinds of “developmental parenting” that support child development:
• Affection
• Responsiveness
• Encouragement
• Teaching

30. What does PICCOLO look like?

31. 29 THINGS PARENTS DO THAT PREDICT SCHOOL READINESS
Speak warmly
Encourage child to do things with toys
Smile at child
Support child’s choices
Praise child
Help child do things on his or her own
Stay physically close to child
Verbally encourage child’s efforts
Say positive things to child
Offer suggestions to help child
Interact in positive ways with child
Show enthusiasm about what child does
Show emotional warmth
Explain reasons for something to child
Pay attention to what child is doing
Suggest activities to build on what child is doing
Change activities to meet child’s interests or needs
Repeat or expand child’s words or sounds
Be flexible when child changes interests
Label objects or actions for child
Follow what child is trying to do
Engage in pretend play with child
Respond to child’s emotions
Do activities in a sequence of steps
Look at child when child talks or makes sounds
Talk about characteristics of objects
Ask child for information
Reply to child’s words or sounds
Wait for child’s response after making a suggestion

32. Knowledge of Infant Development Inventory
What is the KIDI?
Knowledge of Infant Development Inventory
The theoretical base for the KIDI was the theory that parental knowledge of infant development can influence parental behavior and child rearing practices.
The KIDI is a 58-item instrument that was designed to obtain comprehensive information on parents’ factual knowledge of parental practices, child developmental processes, and infant norms of behavior.
The KIDI is designed to be easily accessible to persons with limited education and to be culturally neutral.

33. What is the PSI? The Parenting Stress Index - Short Form
The PSI-SF consists of 36 items derived from the PSI which comprise three scales: Parental Distress, Difficult Child Characteristics, and Dysfunctional Parent-Child Interaction.
Designed to evaluate the magnitude of stress in the parent-child system
The PSI-SF is commonly used as a screening and triage measure for evaluating the parenting system and identifying issues that may lead to problems in the child's or parent's behavior.
Asking parents to complete the PSI—SF can be uncomfortable. Field Data Collectors explain that it will provide information that will help identify the sources and different types of stress that every parent can experience.

34. What is the HOME?
Home Observation for Measurement of the Environment (HOME) is a descriptive profile which yields a systematic assessment of the caring environment in which the child is reared.
The primary goal of the instrument is to measure, within a naturalistic context, the quality and quantity of stimulation and support available to a child in the home environment.
Its focus is on the experience of the child in the home environment.

35. Other Resources
Visit Tracker has various recorded webinars available. New staff should watch Illinois MIECHV Training for New Staff.
The Governor’s Office webpage has numerous up-to-date documents posted including a MIECHV Data Collection Manual.
Field Data Collection brochures for each MIECHV community (contact Mary Anne Wilson for copies)
Contacts
Teresa Kelly, Project Director, Strong Foundations Partnership- Governor's Office of Early Childhood Development
Lesley Schwartz, Manager of Program Evaluation- Governor's Office of Early Childhood Development
Joanna Su, Community Systems & Capacity Building Manager-Governor's Office of Early Childhood Development
Audrey Moy, Manager of Compliance- Governor's Office of Early Childhood Development
Peter Mulhall, Director , CPRD-University of Illinois
Mary Anne Wilson, MIECHV Research Project Specialist at CPRD, University of Illinois
Deborah Kemmerer, MIECHV CQI Specialist at CPRD, University of Illinois