Building Processes for Conducting and Managing Data Collection Erin Geary Tribal MIECHV Annual Grantee Meeting May 2015

Objectives To introduce tools for improving timeliness and quality of data collection** To present best practices for institutionalizing data collection in your program To hear from you what strategies or tools are helping you improve data collection **Tools come from TEI’s Data Collection Toolkit– Coming Soon!

What data are collected? Who collects the data? When are data collected? How are data entered? How is data collection monitored? How are data used?

Why “systematize” data collection

Collecting data on time

Considerations “On-time” window can look different for different measures Ticklers and alerts can be helpful but…. It is helpful to talk through scenarios with staff

Discussion How have you addressed data collection scheduling? What’s worked (tips, tools, etc.)?

Data Collection Tracking Tool Screenshot of Tool and Timepoint entry screen

Data Collection Tracking Tool Screenshot of Scheduling Template

Monitoring Data Quality

In many programs, quality happens (or doesn’t) when… 1) data are collected and 2) when data are entered

Monitoring data collection Training/role play Shadow Collection while being observed Independent data collection Monitoring data collection

Monitoring data entry- knowing what to look for Data entry “red flags” Lots of missing data Values outside the normal range Lack of variation Score patterns that seem out of the ordinary Is the error part of a larger issue or a one time mistake? Are there patterns related to certain staff? What happens when you sort by date? Are certain measures or certain questions being skipped?

Discussion How have you monitored data quality? What issues are you seeing show up regularly? What’s worked for monitoring quality (tips, tools, etc.)?

Quality Assurance Form

Institutionalizing Data Collection

Best Practices- Write it down… all of it Policies and Procedures Manuals are your friend! EXAMPLE- Quality checks How frequently are data checked? By who? Sample of data or all of it? What will they look for? How does the review come back to staff?

Data Collection Protocol Outline

Best Practices- Consistent messaging Training, in-services, weekly meetings, supervision, written memos are all good platforms for communicating data collection procedures BUT…. It is important to ensure that all staff are getting the same messages

Sample Training Schedule

Best Practices- Be supportive Home visitors play a key role in whether data collection is institutionalized in a program Acknowledge that data collection can be challenging AND emphasize why it matters Highlight good work Ask for Home visitor’s input and leadership (form redesign, issues with measures, training new staff, etc.)

Final thoughts? Has this generated other ideas or potential strategies? Are there tools you might take back and use?

Thank you for coming!

For more information on TEI contact: Nicole Denmark Kate Lyon The Tribal Home Visiting Evaluation Institute (TEI) is funded by the Office of Planning, Research and Evaluation, Administration for Children and Families, Department of Health and Human Services under contract number HHSP23320095644WC. TEI is funded to provide technical assistance to Tribal Home Visiting grantees on rigorous evaluation, performance measurement, continuous quality improvement, data systems, and ethical dissemination and translation of evaluation findings. TEI1 was awarded to MDRC; James Bell Associates, Inc.; Johns Hopkins Bloomberg School of Public Health, Center for American Indian Health, and University of Colorado School of Public Health, Centers for American Indian and Alaska Native Health. For more information on TEI contact: Nicole Denmark Kate Lyon Federal Project Officer Project Director Office of Planning Research and Evaluation James Bell Associates, Inc. nicole.denmark@acf.hhs.gov lyon@jbassoc.com The Tribal Evaluation Institute is funded by the Office of Planning, research and Evaluation within the Administration for Children and Families. TEI was awarded to James Bell Associates in partnership with the University of Colorado’s Centers for American Indian and Alaska Native Health and Michigan Public Health Institute. For more information, contact the individuals on this slide.