Seeing Where We are Going: Why Evaluating the Revised BIH Model Is Important and BIH MCAH MIS Update David Dodds, PhD, MPH Nichole Sturmfels, MPH BIH Statewide Meeting—March 15, 2011

What is the foundation for the BIH program evaluation? Why is evaluation important? What are the roles of BIH staff in evaluation? What are the plans for data collection and the BIH MCAH MIS? Key Questions BIH Statewide Meeting—March 15, 2011

Tools for Evaluation BIH MCAH MIS Client Data Book Conceptual Framework BIH Statewide Meeting—March 15, 2011

DRAFT To improve African American infant and maternal health in California and decrease Black:White health disparities and social inequities for women and infants BIH Activities Ultimate Goal INDIVIDUAL Increased health knowledge Increased healthy behaviors Increased empowerment through improved life skills and coping skills Increased receipt of quality medical, social & mental health services Decreased unplanned pregnancy Increased social support both for & among the women Decreased stress by mobilizing resources and services Improved parenting Increased bonding between mother & infant Improved infants’ developmental milestones COMMUNITY Increased community and provider knowledge & cultural competence Increased community partnerships & linkages among service agencies Decreased stressors in the community through community & provider/agency action Improved understanding among community and providers of influence of social inequities on health INDIVIDUAL Services provided to African-American women, infants, and their families & partners include: Referrals to medical, social & mental health services Health education Social and group support Identifying resources Self-advocacy COMMUNITY Promote community and provider engagement and advocacy Educate community and providers about influence of social inequities on health GUIDING PRINCIPLES 1. Comprehensive and integrated: Address multiple risk factors and use multiple strategies 2. Multi-level: Address individual, community, service systems and societal levels, with empowerment focus 3. Collaborative: Partner with community providers and agencies with similar activities 4. Community-driven: Developed, implemented and evaluated by local communities 5. Evidence-based: Developed from proven or promising strategies; impact is measurable 6. Culturally competent: Designed & implemented in a culturally-competent manner 7. Staff training and professional development: Conducted to ensure the BIH activities are provided effectively Problem Poor birth outcomes Social isolation Lack of health knowledge Lack of access to quality health care Poverty Racism Environmental stressors Maximizing impact of the program (numbers served & effect size) Lack of cultural awareness and skills among providers Conceptual Framework Intermediate Outcomes

Outcomes Measured in the BIH Revised Model are Based on Scientific Literature BIH Statewide Meeting—March 15, 2011 Psychosocial Outcomes Health Behavior Outcomes Infant and Maternal Health Outcomes Social support Mastery/Self-efficacy Self-esteem Stress management & coping Tobacco, alcohol & drug use Physical activity Healthy eating (nutrition) Multivitamin use Family Planning Maternal weight gain Low birth weight Preterm birth Mode of delivery Infant mortality

Why are intermediate outcomes important? Intermediate outcomes are on the critical pathway toward the ultimate goal of improving African American infant and maternal health in California and decreasing Black:White disparities. Measuring these outcomes is important for quantifying the good work being done at sites—the previous BIH model did not adequately measure these outcomes. BIH Statewide Meeting—March 15, 2011

Transactional Model of Stress and Coping* Key Concepts Evaluation of stressors Coping efforts: Information Seeking Optimism Social Support Positive Psychology *Lazarus, Folkman, and Moskowitz BIH Curriculum Topics and Activities Understanding and handling stress Relaxation techniques Gratitude circle Strengths and virtues Data Book Tool for Measuring Outcomes Assessment Scale C in Prenatal and Postpartum Assessments 1 and 3 BRIEF RESILIENCE SCALE: Resilience refers to one's ability to bounce back or recover from stress Example: Scientific Concepts in the BIH Curriculum and Data Book BIH Statewide Meeting—March 15, 2011

Why is evaluation important? To measure program progress toward the ultimate goals of BIH. To document the effectiveness of the revised program model. –Provide the best possible intervention for clients within program resources. –Demonstrate the program’s usefulness to federal agencies, state legislators, county superintendents, and other policy makers. –Improve accountability for program funding. BIH Statewide Meeting—March 15, 2011

What is the BIH Staff role in the evaluation? Program Fidelity –Ensure that all BIH sites are conducting the same standardized model. –Clearly document any modifications to the curriculum or case management process. Data Quality –Collect all data in client data book. –Ensure quality of data collection (paper data book) –Ensure quality of data entry (BIH MCAH MIS) Program Feedback –Group Sessions: Complete Facilitator Feedback & Summary after every session –Case Management: Complete FHA Feedback Form monthly BIH Statewide Meeting—March 15, 2011

BIH MCAH MIS – Timeline 2009 BIH Data and Evaluation Sub-Committee – 6 initial conference calls with 22 participants (BIH, MCAH, UCSF) Report at June 2009 Statewide BIH Meeting concluded: “The current BIH MIS will need to be extensively revised or a new MIS adopted in order to improve usability and to measure outcomes of the redesigned BIH program.” BIH Statewide Meeting—March 15, 2011

BIH MCAH MIS – Timeline 2010 MCAH reviewed three options for developing a new MIS: –Revise an existing MCAH program databases (e.g., BIH MIS, AFLP Lodestar). –Purchase and customize off-the-shelf database. –Design a new database at MCAH. Decision: Design a new database at MCAH. MCAH managers worked on budget and IT issues. MCAH MIS programming team and EPI team worked on new MIS. BIH Statewide Meeting—March 15, 2011

BIH MCAH MIS – Timeline 2011 (Working Plan Subject to Change) Winter 2010/2011: Data base structure –Programmers build database where Data Book information will be stored. Spring 2011: Beta testing and training –Beta testing by MCAH, Branagh Information Group, and some BIH users –Training by Branagh Information Group Summer 2011: MIS Release 1 –Data collection and some standard reports Winter 2011/2012: MIS Release 2 –Case management and more standard reports TBD 2013: MIS Release 3 BIH Statewide Meeting—March 15, 2011

BIH MCAH MIS - Features Release 1—Data Entry –Centralized, secure database hosted on a state server –Users will login through an internet browser –Data entry screens are modeled after Data Book forms –7 standard reports Release 2—Case Management –Additional standard reports and user functionality Release 3—Analysis –All standard reports complete (approximately 30) –Ad hoc queries for any information in the database –Data export to Excel BIH Statewide Meeting—March 15, 2011

Data Entry Site BIH Clients Statewide (ALL) Alameda Berkeley Contra Costa Fresno Family Health Advocate (FHA) All FHA Danielle Katie Jane Jackie Choose site Choose data entry mode Choose FHA Planned Navigation BIH Statewide Meeting—March 15, 2011

BIH Clients All clients Dnielle Katie Jane Jackie First Name PNA1PNA2PNA3Birth Outcome PPA 1 Danielle Vicky Sara Jackie Kim FHA all enrolled clients view: form status Choose all enrolled clients All clients All enrolled clients Danielle Vicky Sara Jackie Kim Navigate to specific client form: White/empty = no data entered into form Yellow = form data entered but not validated Red = form data has not passed validation Green = form data validated and saved (finalized) BIH Statewide Meeting—March 15, 2011

BIH MCAH MIS Screenshot BIH Statewide Meeting—March 15, 2011

BIH MCAH MIS Screenshot BIH Statewide Meeting—March 15, 2011

Computing Requirements for BIH MCAH MIS Policy Alert # 2010/11-01 (Effective May 1, 2011) System requirement details: –High-speed internet connection (DSL, etc.) –Silverlight (Free Microsoft operating system update) –.NET (Free Microsoft operating system update) –Internet explorer 7.0+ –2GHz+ CPU, 2GB + RAM, 1024 x 768 or better display May need your local IT to allow software updates Branagh Information Group tech support: BIH Statewide Meeting—March 15, 2011

Data submission to MCAH before the BIH MCAH MIS is online Policy Alert 2010/11-02—Group 1 sites –Forms completed on or before February 28 th, 2011 Client Recruitment Prenatal Assessment 1—Sections 1 & 2 Postpartum Assessment 1 (for postpartum entry)— Sections 1 & 2 Group Sessions: Facilitator Summary & Feedback Policy Alert 2010/11-03—Group 2 sites –Forms completed on or before February 28 th, 2011 –Prenatal Supplemental Form BIH Statewide Meeting—March 15, 2011

BIH MIS – Use and Plans How to use the BIH MIS now: –For clients under the “old” model, use Greenbook and BIH MIS –When implementing the new model, use the hard copy Data Book and eventually the new BIH MCAH MIS –If your client is transitioning from old to new Close out the client using the Greenbook and BIH MIS Enroll the client again using the Data Book BIH MIS will eventually be phased out. BIH Statewide Meeting—March 15, 2011

Summary The evaluation will measure progress towards the ultimate goals of the BIH program. –LHJs, MCAH, and UCSF each have a role in the BIH Program evaluation. The conceptual framework, Data Book, and BIH MCAH MIS are all tools for evaluation. –These tools were developed with input from LHJs. –These tools are based on scientific literature. All LHJs prepare for the BIH MCAH MIS by ensuring compliance with system requirements. All LHJs submit some initial data to CDPH/MCAH for data entry. BIH Statewide Meeting—March 15, 2011

Questions? For questions about the Data Book, submitting data to MCAH, and the BIH MCAH MIS, contact: Nichole Sturmfels For Help Desk support for the BIH MIS or BIH MCAH MIS, contact: Branagh Information Group BIH Statewide Meeting—March 15, 2011