Objectives Understand the purpose behind the revision of the benchmarks Understand how these benchmarks contribute to the larger early childhood system,

MIECHV Performance Measurement Information System Public Comment Webinar November 4, 2015

Objectives Understand the purpose behind the revision of the benchmarks Understand how these benchmarks contribute to the larger early childhood system, not just Maternal, Infant, Early Childhood Home Visiting program funded sites. Understand the benefits and drawbacks of each indicator. Understand the steps to make public comment.

Need and Proposed Use of the Information
Demonstrate program accountability, as well as continuously monitor and provide oversight. Provide quality improvement guidance and TA to grantees. Inform the development of early childhood systems at the national, state, and local level. HRSA is seeking to collect: Demographic, Service utilization, and Select clinical indicators for participants enrolled in home visiting services. Standardized performance and outcome indicators that correspond with statutorily identified benchmark areas.

Public Comment Expectations
Include comments on: The necessity and utility of the proposed information collection for the proper performance of the agency’s functions; The accuracy of the estimated burden; Ways to enhance quality, utility, and clarity of the information to be collected; Use of automated collection techniques and other forms of information technology to minimize the information collection burden.

Overall Impression We agree that most of these are important indicators for home visiting. There is need for clarification about time of collection (point-in-time, cumulative) and other definitions to make the indicators more clear. It would be helpful for several constructs to be limited to only those children enrolled prenatally so that programs have the opportunity to influence the measures. (e.g., breastfeeding, well-child visits, pre-term birth). HRSA has included System Outcomes, for which MIECHV grantees are NOT held accountable, but upon which we must report.

Overall Impression New number of constructs is reduced from current requirements. However, 21 is still a large number is more manageable and still consistent with HRSA’s goals. Many of the indicators are maternal and child health centric. This may create an issue for models that do not focus specifically on maternal or child health. There is a lack of definition for interpretation of time of collection as well a lack of guidance on the frequency of collection for many of these measures.

Demographic, Service Utilization, and Clinical Indicator Data

Demographic, Service Utilization, and Clinical Indicators Data
Describe population served by MIECHV program including items such as: Unduplicated count of new and continuing participants by age, ethnicity, race, marital status, educational attainment, and insurance status. We already collect most of the required information See DRAFT – FY17 Home Visiting Form 1

Additional demographic information to be collected: Housing Status – Adults That this could be added to the demographic information already collected by MHVI grantees without a significant time burden. Not currently included in most data systems. There is concern that this may not be as meaningful considering the highly mobile population we work with. Consider aligning with the McKinney-Vento Act definition of homelessness.

Number of FTE Home Visitors Already collected by MHVI LIAs for our IPR process, and is now required for quarterly reports to HRSA. We do not anticipate this will result in a significant time burden. Nationally, clarification is needed on whether FTE includes supervisors and others, and is only MIECHV funded or all employed FTE HV. Child’s Primary Language The choices are collapsed into a simple English, Spanish, or other. The categories used by Project Launch and the ACA are more inclusive. Labeling the table as “Primary Language Exposure” would better account for the young and bilingual children served in HV.

Clinical/Insurance Information Usual source of medical care – children Usual source of dental care – children We will request clarification on this indicator. We aren’t sure what is meant by “usual source” (most recent, the most frequent, preferred, etc.). Additionally, many children enrolled in home visiting are too young to have begun dental care. Models focused on prenatal and infant care could be penalized for reporting large numbers of children without dental care. The recommendation is to be more consistent with the American Academy of Pediatric Dentistry’s recommendations.

Performance and Outcome Benchmark Data

For MIECHV - These data will be used to demonstrate accountability with legislative and programmatic requirements. Will be used to monitor and provide performance oversight. Future MIECHV funding decisions may be allocated based on grantee performance, including benchmark areas.

For the Early Childhood System – Similar data is being collected for the legislatively required PA 291 report, incorporating this national level work into a set of indicators that demonstrate success for families served by home visiting. All state Departments who fund evidence-based or promising home visiting programs participate in shared reporting of outcomes for all state-funded home visiting services. Future funding decisions may be based on grantee performance, including outcome indicators.

Maternal and Newborn Health
Programmatic Indicators

Depression Screening Indicator: Percent of primary caregivers enrolled in HV who are screened for depression using a validated tool within 3 months of enrollment. Numerator - Number of primary caregivers enrolled in HV who are screened for depression within the first three months since enrollment Denominator - Number of primary caregivers enrolled in HV for at least three months

Indicator Pros/Cons Pros Cons
The switch to the term “caregiver” highlights HRSA’s understanding that HV serves fathers or other caregivers. We collect this data now at 6 months. This would be a change in interpretation, but aligns better with recommended practice. Cons There is not a clear definition of who is included in the term “caregiver”. If it is meant to include others besides the mother, a new tool will be required. The EPDS is only designed for mothers. The switch to primary caregivers is a significant change that would increase the reporting burden and cost for programs.

Well-Child Visit Indicator: Percent of children enrolled in HV who received the recommended number of age-specific well-child visits according to AAP guidelines. Numerator: Number of children (index child) enrolled in HV who receive: 5 visits by 6 months; 7 visits by 12 months; 9 visits by 18 months; 10 visits by 24 months Denominator: Number of children (index child) enrolled in HV reaching specified time frame

Pro More specific information about how many visits a child receives are kept. Cons This appears to be four constructs collapsed into one. It is unclear if data systems can collect data on four separate time frames in a single field. Data collection and reporting burden will be increased. Currently we report if last visit was completed. This would require tracking each visit. Missed visits before children enroll would be considered even though they couldn’t be influenced by LIAs. There is little room for families to get back on track with their visits should they fall behind one month.

Preconception/Interconception Care Indicator: Percent of mothers enrolled in HV at the time of delivery who received a postpartum visit within 8 weeks of delivery. Numerator: Number of mothers enrolled in HV at or before delivery who received a postpartum visit within 8 weeks of delivery Denominator: Number of mothers who enrolled in HV at or before delivery

Pro This indicator is one that Michigan would like to include in PA 291 reporting and was recommended by PEW. This has been demonstrated to be important related to both maternal and child health outcomes. Con We do not collect this now. This would require the collection of a different data point. There are many indicators related to the role of home visiting supporting access to services. Do we need them all? Concern that the term “at delivery” differs by state. A clearly defined window for “at delivery” could improve data comparability across states.

System Indicators

Preterm Birth Indicator: Percent of infants (among mothers who enrolled in HV during pregnancy) who are born preterm following program enrollment. Numerator: Number of live births (index child or subsequent children among mothers who enrolled in HV during pregnancy) born before 37 completed weeks of gestation and after enrollment Denominator: Number of live births after enrollment

Pros Cons Indicator Pros/Cons
Pre-term birth is a key indicator for overall health of a community. We don’t collect this now, but it will come from Vital Records. There will not be a change in collecting burden. Cons Concern that this construct measures an outcome home visiting cannot influence. Those HV models that do not have evidence of improvements in maternal health might be negatively impacted. The dosage requirement for this construct should include families enrolled prior to birth only in order to properly measure a program’s effectiveness in improving preterm birth rates.

Breastfeeding Indicator: Percent of infants (among mothers who enrolled in HV at or before delivery) exclusively breastfed at 6 months. Numerator: Number of infants (index child among mothers who enrolled in HV at or before delivery) exclusively breastfed (given nothing but breast milk) through 6 months of age Denominator: Number of infants (index child among mothers who enrolled in HV at or before delivery) enrolled in HV for at least 6 months

Pro Cons Indicator Pros/Cons
We do collect this data, although we look at initiation rates and do not include ‘exclusive’. We would need to collect it differently. Cons Allowances for dosage should be built in. HV services need to begin prior to birth to influence a mother’s decision to breastfeed.. Exclusive breastfeeding rates at 6 months is more stringent advice than most mothers receive from their pediatricians, who recommend introducing cereals at 4 months. A construct measuring either any breastfeeding at 6 months or exclusive breastfeeding at 3 months would be a good step.

Depression Indicator: Percent of primary caregivers with a negative screen for depression (at second time point) among those enrolled in HV who had a positive screen at the first time point. Numerator: Number of primary caregivers enrolled in HV who had a positive screen for depression at baseline and have a negative screen for depression at follow-up Denominator: Number of primary caregivers enrolled in HV who had a positive screen for depression at baseline.

Indicator Pros/Cons Pros Cons
We don’t have an expectation about multiple depression screenings (but your model may), therefore this would require a change in practice, but data collection systems are in place. Cons Treating adult depression is outside the scope of the home visitors’ influence. Severity of depression can vary widely, leading to less meaningful data on a national scale. The second point in time for screening should be defined to ensure nationally comparable data. The most widely used screening tool is not designed for fathers or other caregivers. Adopting a new tool will mean a significant cost for program and increased reporting burden.

Inter-pregnancy Interval Indicator: Percent of mothers enrolled in HV who had an inter-pregnancy interval of 6 months or less. Numerator: Number of mothers enrolled in HV who become pregnant within 6 months of delivering their previous birth Denominator: Number of mothers who are less than 6 months postpartum or who were pregnant at the time of enrollment

Pros Cons Indicator Pros/Cons
Interpregnancy intervals is a strong measure of maternal and child health. Cons In order to influence inter-pregnancy intervals, they need to be working with mothers who enroll prenatally and receive a specified dosage. If the indicator were inter-birth intervals, we could access the information from vital records, but because the focus is on inter-pregnancy intervals, we have to ask LIAs to collect this information. There is concern that this outcome is outside the scope of many home visiting models.

Tobacco Use Indicator: Percent of primary caregivers enrolled in HV who quit using tobacco or cigarettes within 12 months of enrollment. Numerator: Number of primary caregivers enrolled in HV who were using tobacco or cigarettes at baseline and quit by follow-up (up to one year following enrollment) Denominator: Number of primary caregivers enrolled in HV who reported using tobacco or cigarettes at baseline

Pro Con Indicator Pros/Cons
We currently collect information about tobacco use but we would have to collect this differently. Con Quitting is a dynamic process and needs to be more clearly defined in the numerator, or the numerator should focus on how many are smoking at 12 months out of those that were smoking at baseline. Clear guidance on whether e-cigarettes and vaporizers constitute tobacco use under this construct.

Child Maltreatment, Injuries and ED Visits

Safe Sleep Indicator: Percent of infants enrolled in HV that are always placed to sleep on their backs, without bed-sharing and soft-bedding Numerator: Number of infants (index child aged less than 1 year) enrolled in HV whose mothers report that they are always placed to sleep on their backs, without bed-sharing and soft-bedding Denominator: Number of infants (index child aged less than 1 year) enrolled in HV

Safe sleep is an important construct to measure. Cons
Indicator Pros/Cons Pro Safe sleep is an important construct to measure. Cons There is no indication of how often administrators need to collect this data. This needs to be more clearly defined. There are no standardized tools for collecting this information, would be dependent upon parent report. The term “always” is a very strong descriptor in the definition.

System Indicators

Child Injury Indicator: Rate of injury-related visits to the ER among children enrolled in HV. Numerator: Number of parent-reported nonfatal injury-related visits to the ER among children (index child) enrolled in HV Denominator: Number of children (index child) enrolled in HV

This would require only a minor modification in what we already collect. Con There needs to be consideration that ED visits tend to rise as children become more mobile. Appropriate use of the ED is not a negative. This construct should only reflect the ED visits that occurred during each reporting year.

Indicator: Percent of children enrolled in HV with investigated cases of maltreatment following enrollment. Numerator: Number of children (index child) enrolled in HV with an investigated case of maltreatment Denominator: Number of children (index child) enrolled in HV

Con Indicator Pros/Cons
Nationally, there is no clear consensus among whether adverse impacts are best indicated by reported, investigated, or substantiated cases. Partners within MDHHS recommend using ‘substantiated’ as the best data available related to these measures. This will require a change in what we collect. We currently collect “reports referred for investigation” and “substantiated cases”, not investigated. Recommend reporting this on a yearly basis may be too short a time frame to adequately measure improvement.

School Readiness and Achievement

Parent Child Interaction Indicator: Percent of primary caregivers enrolled in HV who receive an observation of caregiver interaction using a validated tool. Numerator: Number of primary caregivers enrolled in HV who receive an observation of caregiver interaction using a validated tool Denominator: Number of primary caregivers enrolled in HV

Indicator Pros/Cons Con We would need clarification on this. Is there a specific tool that has been identified? We use the HOME but it is more focused on the caregiving environment.

Early Language and Literacy Activities Indicator: Percent of children enrolled in HV with a family member who reads, tells stories, and/or sings songs with child daily in the past week. Numerator: Number of children (index child) enrolled in HV with a family member who reads, tell stories, and/or sing songs to them daily in the past week Denominator: Number of children (index child) enrolled in HV

This is not something we collect. We would need guidance on how to collect it. Is this to be assessed and reported after each visit, monthly, quarterly or annually. Would need clarification on the term “daily, in the past week”. There are no standardized tools to collect this, would be dependent upon parent report. It would be burdensome for administrators and home visiting programs to ensure that each model used consistent documentation on this construct.

Developmental Screening Indicator: Percent of children enrolled in HV with a timely screen for developmental delays using a validated parent-completed tool. Numerator: Number of children (index child) enrolled in HV screened once by 9 months, twice by 18 months, and/or three times by 30 months Denominator: Number of children (index child) enrolled in HV reaching the specified time frame

MHVI sites already collect information about developmental screening. Con This would be difficult to report as currently written. This would require a new way to pull out the data within what we already collect, but would happen at the evaluator level. The reporting burden is the same as three separate constructs. This should also require a minimal program dosage before assessing outcomes.

Behavioral Concerns Indicator: Percent of primary caregivers who are asked if they have any concerns regarding their child’s development, behavior, or learning. Numerator: Number of primary caregivers enrolled in HV who are asked at every visit if they have any concerns regarding their child’s development, behavior, or learning Denominator: Number of primary caregivers enrolled in HV

We would need guidance on this. What exactly is meant by “asked at every visit”. Note that this assumes you use that exact phrase: “…development, behavior or learning.” Would be burdensome to collect this at every visit that is completed. Would be burdensome for administrators and home visitors to ensure that each model uses consistent documentation on this construct.

System Indicators

Parenting Stress Indicator: Percent of primary caregivers enrolled in HV whose parenting stress scores improved from being above the cutoff for normal levels of parenting stress (at first time point) to within the normal range (12 months after enrollment) using a validated tool. Numerator: Number of primary caregivers enrolled in HV who score within normal range of parenting stress at follow-up (and met the conditions specified in the denominator) Number of primary caregivers enrolled in HV who scored above the cutoff for normal levels of parenting stress at baseline

We would need to add a validated tool to measure parenting stress. States would benefit from a defined list of validated tools. Many parents report honest answers about stress levels after they have established trust with their home visitors. Baseline measures often underreport parenting stress. The denominator should only include primary caregivers who receive a baseline and follow up screening.

Crime or Domestic Violence

Crime or Domestic Violence
Completed Depression Referrals Indicator: Percent of primary caregivers enrolled in HV who are screened for IPV using a validated tool. Numerator: Number of primary caregivers enrolled in HV who are screened for IPV using a validated tool within 6 months of enrollment Denominator: Number of primary caregivers enrolled in HV for at least 6 months

Pro Con Indicator Pros/Cons MHVI sites do collect this currently.
However, language would now read “primary caregiver”, suggesting LIAs are screening men as well as women, or other caregivers. This would need clarification and definition of “primary caregiver”.

Family Economic Self-Sufficiency

Insurance Indicator: Percent of a) children and b) primary caregivers (enrolled in HV) who had continuous health insurance coverage in the past 12mos. Numerator: 1) Number of children (index child) enrolled in HV who reported having health insurance coverage for all 12 months 2) Number of primary caregivers enrolled in HV who reported having health insurance coverage for all 12 months Denominator: 1) Number of children (index child) enrolled in HV for at least 12 months 2) Number of primary caregivers enrolled in HV for at least 12 months

We would need to change how we collect this. We currently don’t ask about continuous insurance coverage. Many states do not have the capacity to track continuous insurance coverage with their data systems. As currently defined, does not credit the program for assisting the family to access new insurance if their previous insurance ends for some reason. Clarification on whether parents and children should be reported separately would help create a comparable data set at the national level.

System Indicators

Maternal Education Indicator: Percent of primary caregivers who enrolled in HV without a high school (HS) degree or GED completion who subsequently enrolled in or completed HS or equivalent during their participation in HV. Numerator: Number of primary caregivers who enrolled in or completed a high school degree or GED after enrollment into HV (and met the conditions specified in the denominator) Denominator: Number of primary caregivers without a high school degree or GED completion at enrollment

MHVI LIA’s currently collect this information. Con Many home visiting models enroll teenage mothers too young to have graduated from high school. To ensure these participants are counted, the numerator should include those primary caregivers who remain enrolled in high school or GED program.

Coordination and Referrals

Completed Depression Referrals Indicator: Percent of primary caregivers referred to services for a positive screen for depression who receive one or more service contacts. Numerator: Number of primary care givers enrolled in HV who received recommended services for depression (and met the conditions specified in the denominator) Denominator: Number of primary caregivers enrolled in HV who had a positive screen for depression at baseline and were referred for services

This is an indicator the MHVI LIA’s already collect. Con Would need clarification or definition of the term “primary caregiver”. Service Contacts would also need to be defined. This indicator, many times, is outside the purview of home visiting programs as there are many reasons a family might not follow through on a referral. A more accurate assessment of HV impact on adult depression would be a process measure that counts the number of referrals made following a positive baseline screen.

Completed Developmental Referrals Indicator: Percent of children enrolled in HV with parental concerns and/or positive screens for delays (measured using a validated tool) who receive services in a timely manner. Numerator/Denominator – Next slide

Completed Developmental Referrals Numerator: Number of children enrolled in HV who received individualized developmental support from a home visitor; b) were referred to EI and received evaluation within 60 days; OR c) were referred to other community services who received services within 30 days (and met the conditions specified in the denominator) Denominator: Number of children enrolled in HV with parental concerns and/or positive screens for developmental delays (measured using a validated tool)

This appears to be several constructs in one and includes three different scenarios for reporting. Referral for services in many states is dependent upon a positive screen. States who do not refer based on parental concern alone would need guidance. Would this limit the national comparability of this indicator? Would need to have the term “timely manner” defined.

Questions Questions?

Making Public Comment Comments are due November 9, 2015
Submit comments to: or HRSA Information Collection Clearance Officer Room 10C-03, Parklawn Building 5600 Fishers Lane Rockville, MD, 20857 Include the Information request collection title for reference: The Maternal, Infant, and Early Childhood Home Visiting Program Performance Measurement Information System.

Home Visiting Contact Information Tiffany Kostelec – Nancy Peeler – Julia Heany –

Objectives Understand the purpose behind the revision of the benchmarks Understand how these benchmarks contribute to the larger early childhood system,

Similar presentations

Presentation on theme: "Objectives Understand the purpose behind the revision of the benchmarks Understand how these benchmarks contribute to the larger early childhood system,"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Objectives Understand the purpose behind the revision of the benchmarks Understand how these benchmarks contribute to the larger early childhood system,

Similar presentations

Presentation on theme: "Objectives Understand the purpose behind the revision of the benchmarks Understand how these benchmarks contribute to the larger early childhood system,"— Presentation transcript:

Similar presentations

About project

Feedback